Status:

May 2025 - Testing completed, result & advisory agent outline published

Australia's nbn co announced in September 2024 that they would be accelerating the provision of multi-gigabit speed tiers and uplifting existing offerings, while maintaining current pricing. So I did some performance testing using current nbn based services to see where bottlenecks sit with my network and if I can utilise the improved performance.

Testing Setup and Results

Testing Scenario Setup

To conduct testing I used the following setup:

‌ This allowed for:

• Multiple User Agents (devices) - including iPhone, iPad, Notebook (with WiFi & 1GbE Adaptor) and Desktop (with 10GbE Connection)
• Multiple WiFi LANs - including older Apple Airport Express (100MbE) and Airport Extreme (1GbE) with a/b/g/n capabilities, Netgear AX3000 (1Gbps) with WiFi 6 and 2 x Ubiquity Uni Pro 7 (2.5GbE) with WiFi 7 capabilities connected to different VLANs
• Multiple Wired LAN - including 100MbE, 1GbE, 2.5GbE and 10GbE connected devices
• Router variation - Cisco 4331 Branch Router (1GbE, 2.5GbE & 10GbE) with license and ability to control total LAN to WAN routed throughput (upload + download) levels of: 100 Mbps, 300 Mbps throttled or up to 2.5 Gbps unthrottled and Netgear AX3000 (with WiFi 6) and 1GbE LAN/WAN ports
• 2 x nbn FTTP Services - via Aussie Broadband (500/200) as primary service and Telstra (50/17) as seperate VRF (VPN Routing and Forwarding) based independent service connected to alternate VLANs each with their own WiFi 7 LANs

The testing resulted in shifting the bottleneck from Device to WiFi to Router to nbn service.

Testing was done using Ookla "SpeedTest" on Notebook & Desktop and "SpeedSmart" on Phone & iPad and included:

• Running single device tests three times to get average result
• Doing simultaneous runs of all 4 devices at once to try to see impact of all devices on individual device throughput

To test above 500/200 throughput of the Aussie Broadband service I simulated LAN/WAN topology and used iperf3 for throughput testing.

The testing setup for this was:

‌In my scenario both iPerf3 Server and Mac Pro iPerf3 Client were running in Ubuntu Virtual Machines, so total throughput result will include slight virtualisation overhead.

The result reported from each of the apps/iperf3 were much the same and doing simultaneous runs does not really add any value as apps/iperf3 do good job of establishing multiple connections and ensuring network bandwidth is fully consumed.

Testing Results

For the testing results I was interested to see when customer network or nbn service was bottleneck point.

The results show the shift of the throughput bootleneck from:

• nbn - lowest cost Telstra nbn (50/17) service to
• WiFi Device - old Apple Airport Express a/b/g/n device with 100Mbps Ethernet to
• Cisco Router - throttled to 100 Mbps and then 300 Mbps to
• WiFi/nbn - WiFi (a/b/g/n) download, nbn 200 Mbps upload
• User Agent - with iPad showing lower upload speeds than iPhone and
• nbn - with Aussie Broadband (500/200) service and WiFi 7 and 10 GbE LAN

Also in the result set is "Multiple Device" test (average across devices), 4G & 5G tests.

The result show Upload vs Download with bubble size being Value (Bandwidth (Upload + Download) / Cost ($) ratio), so bigger is more bang for buck.

The result show some potential points where device and/or nbn service asymmetry can result in different bottlenecks and it is easy to hit Upload limit with both Telstra & Aussie services.

Rule of thumb observation:

• lowest speed nbn connections (i.e. 100Mbps) not likely to be limited by router / WiFi and at this price point maybe it is cheaper to use Mobile service and forget about nbn Wired Services
• WiFi LAN with a/b/g/n capabilities should be able to service nbn connection with higher speed tiers up to 300Mbps, but you need to also have router that can sustain this (in case of  Cisco 4331 this requires at least a "throughput" license)
• For nbn service of 400 Mbps and greater, you will need to have WiFi 6/7 and a higher performing router (or in case of  Cisco 4331 a "booster" license for unthrottled throughput).

Home Connectivity Use Case

For home use case scenario I purchased a mid-range Netgear AX3000, as current generation networking device with WiFi 6 and 1GbE LAN & WAN ports.

• JB HiFi & Officeworks sell the Netgear AX3000 for ~ $ 200 - Netgear claims this can sustain 1GbE ISP connecction, which is higher throughput than my Aussie 500/200 service. My initial thoughts were that this was optimistic, so lets test and see.
• Telstra (Smart Modem Gen 3 with 1GbE + WiFi 6) &  Optus (Ultra WiFi Model Gen 2 with 1GbE & WiFi 6) look to provide similar specification devices, so I would expect them to equivalent performance.

I tested the Netgear AX3000 via Cisco 4331, as current limit on performance was nbn bottleneck rather than home device one. Testing covered both wired (1GbE) and with WiFi 6 and the home device performed much better than expected hitting nbn bottleneck. Interestingly WiFi 6 test got better throughput than the direct 1GbE one, for this round of testing.

As I was constrained by current Aussie Broadband service I deviced to do simulated LAN/WAN throughput testing with Netgear AX3000 and found that it hit the throughput limit of the 1GbE interface (see result in "Business Connectivity Use Case" in next section).

This is good news for nbn home users as even modestly priced current generation Router/WiFi device should be able to sustain 500 to 1000 Mbps based service.

Business Connectivity Use Case

For SMB the Cisco based setup reflects SMB or Enterprise Office use case with Branch Routers & seperate Wireless Access Points.

The potential throughput this could achieve was simulated via iPerf testing:

This illustrates shift from:

• nbn bottleneck at 500/200 service to
• Netgear Home Device Router (AX3000) to
• potential nbn bootleneck for top speed tier 2 Gbps capable PON (Passive Optical Network) service

Bubble sizes based on estimated cost for 1G & higher nbn services.

From a device perspective the unthrottled 4331 (with SM-X-16G4M2X module installed for 16 x 1GbE, 4 x 2.5GbE & 2 x 10GbE) with WiFi 7 access points should be able to sustain up to 2 Gbps service.

For this test 4331 was configured as simple NAT router and was able to sustain 2Gbps of throughput. This is good result and consistent with documentation on SM-X-16G4M2X module. To get this throughput requires that all traffic is via SM-X based interfaces, to get advantage of 10GbE interface up/down to routing cores and avoids traffic going across NIM/SM-X modules resulting in backplane bottleneck (see NOTE #1), through the "ISR 4K Multigigabit Fabric (MGF)".

NOTE #1: From is ISR 4000 datasheet: "Each service-module slot offers high data-throughput capability of up to 10 Gbps toward the system and up to 1 Gbps to other module slots".

NOTE #2:  The 4331 is now end of life and the Cisco recommended replacement model is C8200-1N-4T, which is documented as having 1Gbps sustained throughput. So to get equivalent performance  as unthrottled 4331 (with SM-X module) would mean going up a step to a C8300-1N1S-6T with 2.5 GbE capable module installed.

NOTE #3: Market observation on Cisco branch routers, is that they do not seem to have had jump in performance that would be expected given around nearly seven years between ISR4000 (2014) & C8200/C8300 (2021) product introductions. Given general obaqueness of Cisco pricing, it is hard to comment on C8200/C8300 vs. ISR4000 value.

An Advisory Agent ...

My initial motivation for testing was to see if I could/would get benefits from new nbn Speed Tiers (the offer to get more for the same price sounded tempting), but might mean I have to either upgrade my service or equipment.

My "Use Case" is more business focused, as this blog and other services are self hosted and I needed to update from ISR2901 to ISR4331 for various reasons (see: "Cisco IOS 15.5 (ISR 2901) to IOS XE 16.6 / 16.9 (ISR 4331) Migration"), not least of which was wanting to get benefit of higher bandwidth nbn service.

The testing shows that the basis for identifing thoughtput bottlenecks is an analysis of connections serial topology with bottleneck up or down being node with minimum bandwidth. A more sophisicated analysis would identify "active" parallel connection paths and add the bandwidth of these for resulting serial equivalent.

Currently nbn runs a regime of throughput tests and reports these to Australian Competition and Consumer Commission (ACCC), all to ensure consumer gets fair service. This does very little to help the consumer know if:

• They are getting the experience they are paying for, as current nbn pespective is only for its part of end to end chain,
• They are on the "best" connection plan for their needs and
• Will they realise benefit of an upgrade to higher tier service or will they also be required to upgrade their other network equipment ?

To provide customer with bettter experience, I believe it would be useful for nbn to provide an "agent" to help address these questions. This would be more informative and immediate than an ACCC policing regimen.

The "Advisory Agent" should be able to  answer the following questions:

• Question #1: Is my service delivering against purchased service throughput ?
• Diagnosis Case - if not then where is the bottleneck that is preventing this?
• Question #2: Given my usage scenario, what is recommended service throughput need ?
• Usage Case - would require then to select against stereotypical usage cases
• Question #3: Will I need to upgrade my network equipment if I elect to move to a higher bandwidth service ?
• Predictive Case - detect current topology and equipment and determine if this can sustain desire throughput
• Question #4: What is likely highest throughput I can achieve with my current setup
• Predictive Case - detect current topology and equipment and determine theoretical maximum throughput

To provide a good experience the "Advisory Agent" needs to:

• require zero or trivial installation  (i.e. likely can be invoked from browser)
• run on any device (deskopt / laptop computer, phone, tablet ...)
• require minimal input from the user, so it must discover the current situation and use knowledge to answer the questions.

So what is current "state of the art" ?

Here is ChatGPT response to:

• "What is maximum throughput I can achieve with my nbn connection ?"

To determine the maximum throughput you can achieve with your NBN connection, there are several factors to consider:

1. NBN Technology Type
The type of NBN technology your connection uses will directly affect the maximum speed you can achieve. NBN offers several different technologies:

FTTP (Fibre to the Premises): Can theoretically support speeds of up to 1 Gbps (1000 Mbps) or higher with the right plan.
FTTC (Fibre to the Curb): Typically offers speeds up to 100 Mbps, but can sometimes reach 200 Mbps with the right conditions.
FTTN (Fibre to the Node): This depends on how far you are from the node; speeds can range from 25 Mbps to 100 Mbps or more, with longer distances resulting in lower speeds.
HFC (Hybrid Fibre Coaxial): Can support speeds up to 1000 Mbps (1 Gbps), depending on the plan and infrastructure.
Fixed Wireless: Speeds vary based on location and congestion, but typically range from 25 Mbps to 100 Mbps.
Sky Muster (Satellite): Generally provides speeds of around 25/5 Mbps, with some areas seeing up to 100/40 Mbps in optimal conditions.
2. NBN Plan
The maximum speed you can achieve is also dependent on the NBN plan you're on. Typical NBN plans are:

NBN 12: Max download speed of 12 Mbps
NBN 25: Max download speed of 25 Mbps
NBN 50: Max download speed of 50 Mbps
NBN 100: Max download speed of 100 Mbps
NBN 250: Max download speed of 250 Mbps
NBN 1000: Max download speed of 1000 Mbps (1 Gbps)
The speed you can get is also influenced by your router and the equipment you're using.

3. Network Congestion
During peak usage times, NBN speeds can be affected by congestion. If you're sharing the connection with multiple users or devices, this will also affect your achievable throughput.

4. Wi-Fi vs Wired Connection
If you're connecting via Wi-Fi, the maximum speed achievable might be lower due to the limitations of your router and interference in the environment. A wired Ethernet connection typically provides a more stable and faster connection.

5. Router Capabilities
The model of router you're using may limit your speed. Older routers or those not supporting higher standards (like Wi-Fi 5 or 6) can bottleneck your speed.

How to Find Your Maximum Throughput:

To get an exact figure for your connection, you can:

Check your NBN connection type (FTTP, FTTN, etc.) using your address on the NBN website or your provider’s website.
Verify the NBN speed plan you are subscribed to (12, 25, 50, 100, etc.).
Perform a speed test to see the current download and upload speeds, which will give you an indication of your actual throughput.
If you know your NBN type and plan, I can help you estimate the maximum throughput more accurately. Would you like to share that info?

So here is the information requested:

• "My nbn plan is 500/200, connection type is: FTTP, I am using WiFi and I test during office hours."

Thanks for the details! Here’s an estimate based on your specific situation:

1. NBN Plan (500/200)
Since you’re on an NBN 500 plan, your maximum download speed can reach 500 Mbps, and your upload speed can go up to 200 Mbps. These are the maximum speeds you can theoretically achieve on your current plan, assuming ideal conditions.

2. Connection Type (FTTP)
FTTP is the best connection type in terms of potential speed, so you should be able to get close to these maximum speeds (500 Mbps download / 200 Mbps upload) under optimal conditions.

3. Wi-Fi Connection
Wi-Fi speeds can be significantly lower than wired speeds, especially if:

Your router is older or doesn't support Wi-Fi 5 (802.11ac) or Wi-Fi 6 (802.11ax).
You're using the 2.4 GHz band, which is more congested and has slower speeds compared to the 5 GHz band.
There is interference from other devices, walls, or furniture between your router and your device.
If you’re connecting via Wi-Fi and you're not on a modern Wi-Fi 6 router, your actual speeds could be lower than the plan’s maximum. In practice, a Wi-Fi 5 router might give you speeds in the range of 100-300 Mbps, depending on factors like your distance from the router, signal strength, and interference.

4. Testing During Office Hours
If you're testing during office hours, it's worth considering network congestion. Since many people in your area are using the internet at the same time, this could cause a slight dip in speeds. However, since you're on a high-speed 500 Mbps plan with FTTP, any congestion should ideally have a minimal effect unless it's particularly busy in your area.

Expected Throughput:
Best case scenario (wired connection): You could easily hit near 500 Mbps download and 200 Mbps upload speeds on your FTTP connection, given the plan and connection type.
Wi-Fi scenario: On Wi-Fi, depending on the router and distance, you could expect somewhere between 100-300 Mbps for download speeds. The upload speeds on Wi-Fi are often less affected, so you might still see closer to 150-200 Mbps for upload.
Tips to Maximize Throughput:
Use 5 GHz Wi-Fi band if your router supports it, as it offers faster speeds and is less prone to interference than the 2.4 GHz band.
Reduce the distance between your device and router, or ensure there are fewer obstructions between them.
Test with a wired connection if possible to eliminate any Wi-Fi-related bottlenecks and get closer to your plan's maximum potential.
Check for congestion: If you're experiencing slow speeds during office hours, it could be due to local congestion in your area. You can try testing at different times (like late at night or early morning) to see if it improves.
Have you already run a speed test during those office hours? If so, how do your speeds compare to the expected ones?

Looking at this you can see that ChatGPT has done a pretty good job and provided sensible advise. The underlying knowledge model being used to provide this is apparent.

In prior ChatGPT it even advises that my WiFi 7 setup would likely top out at around ~700 Gbps. I found that asking the same question, does not elicit same response, so maybe it has learnt something in between requests.

This chat session is sufficient for predicative case, but would not help with the diagnostic case.

Here is diagnostic ChatGPT session:

• "I have an 500/200 nbn connection but am only getting 300 Mbps download, can you advise on what the problem is?"

If you're on a 500/200 NBN plan and only getting 300 Mbps download speeds, it’s not ideal, but there are a few possible reasons for the discrepancy. Let's go through some common issues and how to troubleshoot them:

1. Wi-Fi Limitations
Since you're using Wi-Fi, this is likely the most common cause of slower speeds. Wi-Fi is affected by several factors:

Router capabilities: If your router is older or doesn’t support high speeds (e.g., not Wi-Fi 5 or 6), it might not be able to handle the full 500 Mbps.
Frequency band: If you're using the 2.4 GHz band, it can be slower and more congested. Try switching to 5 GHz, which should give faster speeds and less interference.
Distance and obstacles: The further you are from your router, the weaker the signal, leading to slower speeds. Walls and furniture can also block the signal.
Device limitations: Older devices may not support speeds higher than 300 Mbps on Wi-Fi.
What to do:

Make sure you’re connected to the 5 GHz band.
Test the speed close to the router to see if the issue persists (this will rule out distance as a factor).
If possible, upgrade to a Wi-Fi 6 router that supports higher speeds and better range.
2. Network Congestion or Throttling
Local congestion: During peak hours (e.g., office hours), other people in your area may be using the network, which can cause slowdowns. This can be especially true if you’re on a shared FTTP network with other customers in the same area.
ISP throttling: Some ISPs might implement traffic management or throttling during peak times to manage congestion, even though you're on a 500/200 plan.
What to do:

Run speed tests at different times of the day to see if the speed fluctuates. If it's consistently slower during peak hours, it could be due to network congestion.
Contact your ISP to confirm whether there are any issues in your area or any traffic management practices that could be affecting your speeds.
3. Router or Hardware Limitations
Your router might not be able to handle the full speed of your plan, especially if it’s older or not designed for high-speed plans.

What to do:

Check the specifications of your router. Make sure it supports the speed you're paying for (500 Mbps).
If your router is quite old, consider upgrading to a more modern one that supports higher speeds (Wi-Fi 5 or Wi-Fi 6).
Ensure that you’re using Ethernet cables that are capable of handling high speeds. For example, Cat 5e or Cat 6 cables are required for higher speeds.
4. Connection Issues or Faults
There could be an issue with your NBN connection, such as signal degradation or issues at the exchange.

What to do:

Test your connection wired (via Ethernet) directly to the router to rule out Wi-Fi as the cause. This will give you an idea of whether the issue is with the connection itself or just the wireless performance.
If your wired connection is also slower than expected, there could be an issue with your NBN equipment (like the NBN box or the modem), or a problem with the line that your NBN service is using.
Reset your NBN equipment (modem and router) to ensure it's not a temporary glitch.
If the problem persists, contact your ISP and ask them to check the connection quality, as they may need to escalate it to NBN for investigation.
5. Speed Test Methodology
Speed tests can sometimes be misleading if certain factors are overlooked:

Server location: Make sure you're testing with a server that's geographically close to you. Servers that are far away can result in slower speeds.
Device performance: If your device is old or running too many background processes, it might not be able to handle the full speed.
Background usage: If other devices or users are using the internet heavily (e.g., streaming, gaming), it can affect the speed test results.
What to do:

Run the speed test on a wired connection directly to the router to get an accurate reading of your true throughput.
Use a reliable speed test tool (like Speedtest.net) and choose a server that's nearby to get the most accurate results.
6. Overloaded Router or Interference
If there are too many devices connected to the router or if the router is experiencing interference (e.g., from microwave ovens, Bluetooth devices, etc.), this can slow down speeds.

What to do:

Try disconnecting unnecessary devices from your network and running the speed test again.
Change the Wi-Fi channel if there’s interference (especially on the 2.4 GHz band).
Next Steps:
Test with Ethernet: Try connecting directly via Ethernet to rule out Wi-Fi limitations.
Test at different times of the day to identify any patterns of slow speeds (e.g., during peak hours).
Contact your ISP to confirm if there are any known issues in your area or throttling being applied.
If the problem persists with wired connections, consider having your ISP run diagnostics on your line or possibly send a technician to investigate further.
Let me know how these tests go! We can dig deeper into any specific area if necessary.

In my first session with ChatGPT, it asked if I wanted to proceed with testing and I answered "yes" about live trouble shooting , but ChatGPT disconnected me.

In the next session it sent me to the service provider, which is not what I want "my Agent" to do, excepts as last resort.

Summary

What this illustrates, given that I have asked a fairly specific set of questions to a general purpose "large language model" agent,  is that creating an "Advsory Agent" is possible and likely quite simple with available tools.

This seems to be a good direction for nbn customer experience team to head in, to:

• Entertain and inform their ultimate end customers,
• Increase the number of customer shifting to higher speed tiers and
• Ensure that their customers are getting the most from their nbn service.

I close by saying, not once did ChatGPT ask me to "reset the modem" which we know is a favourite thing for the human agents for some telco's to say when we call then for help ;-) .

Acknowledgement: Thanks to Toby Zines from nbn co for interesting discussion on nbn "Speed".

NOTE: See technical appendix for some tips on setting up up Ubiquity Access Points with Raspbery Pi Unifi controller.

Links & References:

NBN Co Higher Speed Tiers - nbn announement (September 2024) on Multi-Gigabit (up to 2 Gbps from maximum of 1 Gbps), with Home Fast (100/20) moving to 500/50 (Mbps down/up), Home Superfast (250/25) moving 750/50 and Home Ultrafast (1000/50) to 1000/100. For business services nbn's 500/200 Mbps plus Pro and 1000/400 Mbps plus Pro services will have reduced price and business wanting 2 Gbps services will get new 4  x 2.5 Gbps port NTD. New higher speed services and rates are targeted for September 2025 delivery.

iperf3 - Open Source network performance testing utility that is built available in Linux & BSD based operating systems

Install Ubiquity Unfi on Ubuntu (Raspberry Pi) - there are lots of links on installing unifi on Ubuntu and Pi, but this Reddit discussion thread is what I followed as it provides sound rational and wors with Ubuntu 24.04 as well: https://www.reddit.com/r/Ubiquiti/comments/1bvf1t8/unifi_controller_v81x_on_mongodb_v70_ubuntu_lts/

Appendix #1 - Ubiquity Unifi Controller on Raspberry Pi

While setting up this test scenario I added two Ubiquity Uni Pro 7 Access Points to my network.

The Ubuiquity devices requires that you either connect to a Ubiquity Cloud Controller service or run your own Ubiquity Unifi Controller to configure your devices.

As I had two Uni Pro 7 devices running on different VLANs I setup two local controllers, one on Ubuntu VM and one on Raspberry PI (as this was cheaper then buying and new NIC card for server machine with VMs.

This tool quite a bit of fiddling as I found lot of different "tips" and also as I tried initially to get Raspberry Pi controller going on Pi 3 (failed) and ended up using Pi 5 as contoller needed more "grunt".

For my Pi setup I used:

• Raspberry Pi 5 with Ubuntu Server 24.04 LTS

The Unifi controller uses MongoDB database and is an Java application build with JDK 17.

The Ubquity Linux install page says that you should use MongoDB 3.6, but this is a very old version and result in issue with ssl library version.

Instead I followed these "Unifi Controller v8.1.X+ on MongoDB v7.0 Ubuntu LTS 22.04" instructions from "btbutts" on reddit.

I used the same instructions for both Pi ARM64 Ubuntu and AMD64 VM installations. The result was working installs and more current environment.

I also successfully installed and got running using same instructions with Pi 3 with Ubuntu 24.04, but on trying to "adapt" a new access point as part of new device setup, the Pi 3 keep hanging the access got left in non-working state and needed a hardware reset (paper clip job).

I have recently got back a Pi 4 so might see if this works ....