Bitrates for eth0 do not reflect Internet traffic on Broadcom routers

Hello @kennedyshead ,

TL;DR: Do not expect to measure the Internet traffic when looking at eth0 of a Broadcom based Asuswrt device.

This is maybe more a clarification than a bug. I am using the ASUSWRT integration for Home Assistant and the bandwidth monitoring shows a strange behaviour when checking my gateway: Upload and download rates are all the time almost identical. Which is kind of weird, when you are downloading a file at say 200MBit/s and upload also show a rate of about 200MBit/s - which can't be, as my upload is limited to 30MBit/s.

So I started digging and checked the plugin, as well as aioasuswrt:

Readings are taken from the interface eth0 of the router, which is the interface connected to the Internet (WAN).
I am using an ASUS RT-AC66U (rev A) with the recent release of John's fork of Asuswrt-Merlin (374.43_44EAj9527).
OT: WiFi is disabled due to a bug in the Broadcom WiFi driver which leads to the router rebooting randomly every now and then. If this happens during a Skype meeting, it takes quite some time until the router is up again - which is very annoying. With WiFi disabled, the device is running rock-solid.

The strange behaviour is reproducable, also based on the low-level kernel statistics of the interface. But the traffic monitor feature of Asuswrt displays a reasonable graph for Internet traffic when doing a download (download rate >> upload rate). So I checked how the traffic monitor gets its numbers.

Basically it is similar to the implementation in aioasuswrt: read the total of bytes every 'x' seconds and divide the difference by 'x', but with a twist for Broadcom based routers: when doing the delta, first a difference is computed like eth0 minus vlan1 (for both TX and RX bytes). See the function netdev_calc() in release/src/router/shared/misc.c - search for // special handle for non-tag wan of broadcom solution.

The implementation looks like this: If in NVRAM switch_wantag is set to none (and there is an interface vlan1 - see bcmvlan_models(model, modelvlan)), the special handling is activated. eth0 will become the WIRED connection, while the difference will become the INTERNET connection.

You can see the network traffic information the Traffic Monitor page gets by e.g. using the developer tools of Firefox. Look for calls to update.cgi and check the returned JSON data:

netdev = {
 'WIRED':{rx:0x694fd5a0,tx:0x38930af4}
,'INTERNET':{rx:0x5f4b4b20,tx:0x156603d5}
,'BRIDGE':{rx:0x5712764,tx:0x6545be3c}
,'WIRELESS0':{rx:0x0,tx:0x0}
,'WIRELESS1':{rx:0x0,tx:0x0}
}

So the bottom line is: if you want to measure your Internet throughput, you need to use figures from INTERNET, not WIRED (eth0). //

Unfortunately this metric is not available via aioasuswrt, as you can only give an existing interface for monitoring. A new approach would be needed. I quickly whipped up a short script to illustrate the solution (which produces sensible values for me):

#!/usr/bin/env python3

import asyncio
import logging
import re

import sys

from aioasuswrt.asuswrt import AsusWrt
from aioasuswrt.helpers import convert_size

component = AsusWrt("gateway", 22, username='admin', ssh_key='~/.ssh/gateway_rsa')
logging.basicConfig(stream=sys.stdout, level=logging.DEBUG)
logger = logging.getLogger(__name__)

async def print_data():
    # sampling interval
    dtime = 5

    prevrx = None
    prevtx = None
    while True:
        eth0rx = eth0tx = 0
        vlanrx = vlantx = 0

        # based on the built-in Traffic Monitor code of AsusWrt
        # see - ej_netdev() in release/src/router/httpd/web.c
        #     - netdev_calc() in release/src/router/shared/misc.c
        #       search for '// special handle for non-tag wan of broadcom solution'

        # BONUS: gets a *consistent* snapshot of the transferred bytes across all interfaces
        netdev = await component.connection.async_run_command("cat /proc/net/dev")

        # skip the first two header lines, process each interface
        for line in netdev[2:]:
            parts = re.split('[\s:]+', line.strip())
            # NOTES:
            #  * assuming eth0 always comes before vlan1 in dev file
            #  * counted bytes wrap around at 0xFFFFFFFF
            if (parts[0] == "eth0"):
                eth0rx = int(parts[1]) # received bytes
                eth0tx = int(parts[9]) # transmitted bytes
            elif (parts[0] == "vlan1"):
                vlanrx = int(parts[1]) # received bytes
                vlantx = int(parts[9]) # transmitted bytes

        def handle32bitwrap(v):
            return v if v > 0 else v + 0xFFFFFFFF

        # the true amount of Internet related data equals eth0 - vlan1
        inetrx = handle32bitwrap(eth0rx - vlanrx)
        inettx = handle32bitwrap(eth0tx - vlantx)

        if prevrx is None or prevtx is None:
            rx = tx = 0
        else:
            rx = int(handle32bitwrap(inetrx - prevrx)/dtime)
            tx = int(handle32bitwrap(inettx - prevtx)/dtime)

        prevrx = inetrx
        prevtx = inettx

        logger.debug("DL: {}/s UL: {}/s".format(convert_size(rx), convert_size(tx)))

        await asyncio.sleep(dtime)

loop = asyncio.get_event_loop()

loop.run_until_complete(print_data())
loop.close()

Hope this helps if anybody comes across this phenomenon. Perhaps it would be great to have some kind of virtual interface like inet which could be passed to aioasuswrt which would trigger the special handling on readout?

Anyway, thanks for your work & Cheers, tempura

kennedyshead / aioasuswrt

Bitrates for eth0 do not reflect Internet traffic on Broadcom routers #44