Diagnosing Random Packet Loss Across a 30-Year-Old Network

Legacy networks are common. Businesses grow, offices expand, and new equipment gets added wherever it is needed. Over time, what began as a structured network becomes something much harder to understand — and much harder to fix. The first step in solving a problem like this is not replacing hardware. It is gaining visibility.

The Problem

A customer was experiencing random packet loss across a large and complicated network. The environment had grown organically over around 30 years and now supported more than 300 devices.

Over time, additional switches had been added wherever connectivity was needed. Some linked different offices and buildings, often because there was not enough structured cabling available. The result was a network that had become difficult to understand and even harder to troubleshoot.

Symptoms included random packet loss affecting multiple devices, intermittent performance problems, no obvious single point of failure, numerous unmanaged switches throughout the network, very limited visibility of where devices were connected, and a topology that had evolved over decades.

Replacing everything would have been the cleanest technical solution, but it was not a practical starting point. We needed a way to understand what was happening without checking more than 300 devices individually.

Our Investigation

The first priority was visibility. Instead of immediately replacing equipment, we introduced managed switching into strategic points within the network.

A managed switch can provide information that an unmanaged switch cannot: which MAC addresses are visible through each port, transmit and receive errors, broadcast traffic, multicast traffic, port utilisation, and potential signs of network loops or abnormal traffic.

By positioning managed switches at key points and reviewing their MAC address tables over time, we could begin mapping which parts of the network were connected through each port — a much faster way of building a picture of the existing network without physically tracing every cable.

Looking Beyond the Obvious

Packet loss does not always mean a switch has failed. Several other possibilities needed investigating.

Cabling and Electrical Interference

Some network cables had potentially been installed close to electrical cabling. A cable can successfully negotiate a 1 Gbps Ethernet connection while still experiencing enough interference or errors to cause unreliable performance. Checking interface error counters helps identify this type of problem.

Broadcast and Multicast Traffic

Without visibility or control, excessive broadcast or multicast traffic generated in one part of the network could potentially affect devices elsewhere. By introducing managed switches further up the network and monitoring traffic statistics, we could identify which direction abnormal traffic was coming from and progressively work backwards towards its source.

Diagnostic Insight: Make the Network Visible Before Replacing It

Managed switching allows us to see MAC address locations, port errors, broadcast and multicast traffic, utilisation and other information that is simply not available from unmanaged switches. In a network that has grown organically over decades, this data is often the only practical way to begin understanding what is connected — and what might be causing problems.

A Practical Diagnostic Approach

Rather than immediately replacing dozens of switches, we used a small number of managed switches as diagnostic tools, temporarily installed in different sections of the network to collect MAC address information, TX/RX error counts, broadcast levels, multicast levels and port statistics. Moving these monitoring points around the network gradually created a much clearer picture of what was happening.

Why Multicast Matters

On another large fibre network, a very small number of endpoints began generating extremely high levels of multicast traffic. The issue originated several layers below the network core, yet the effect was packet loss and reduced performance across core switches running on a 10 Gbps fibre backbone. Because the infrastructure was managed, we could monitor multicast packets per port and work backwards through the topology until the problematic segment was located.

A performance problem affecting hundreds of devices can sometimes originate from just one device or one small section of the network.

The Approach

The recommended approach: introduce managed switches at strategic points; build an understanding of the network using MAC address tables; monitor port error counters; look for excessive broadcast and multicast traffic; check for possible network loops; identify questionable cabling runs; use managed switches temporarily in different segments; narrow the problem down progressively until the troublesome section is isolated.

Why This Matters

Legacy networks are common. Businesses grow, offices expand, new equipment gets added and temporary fixes gradually become permanent infrastructure. Thirty years later, nobody necessarily knows exactly how everything connects.

The answer is not always to rip everything out immediately. Sometimes the first job is simply to make an invisible network visible.

At Altitude IT Support, we approach complex network problems methodically — gathering evidence, questioning assumptions and narrowing the problem down before recommending expensive changes. Because with a network containing hundreds of devices, the fault affecting everyone could ultimately be coming from one cable, one switch port or one badly behaved device.