prometheus-node-exporter-lua: WiFi traffic stats broken

[jpds]

Maintainer: @champtar Environment: Ubiquiti UniFi nanoHD, MediaTek MT7621 ver:1 eco:3, OpenWrt 21.02.0 r16279-5cc0535800 / LuCI openwrt-21.02 branch git-21.231.26241-422c175

Description:

The metric keys wifi_station_receive_kilobits_per_second and wifi_station_transmit_kilobits_per_second return garbage data. I'm seeing consist values of 650000 and 702000 from my devices.

Here is a screenshot of what I'm seeing in prometheus for wifi_station_receive_kilobits_per_second:

The red line here is my laptop which downloaded an Arch Linux ISO as a test, the purple line is my phone - and the exporter seems to be saying that they're doing an equal amount of data transfer.

On the wifi_station_receive_packets_total graph however - I can see the ISO download:

I think these two keys should be changed to wifi_station_receive_bytes_total and wifi_station_transmit_bytes_total (to make it consistent with node_network_receive_bytes_total/node_network_transmit_bytes_total).

[champtar]

wifi_station_receive_kilobits_per_second and wifi_station_transmit_kilobits_per_second give you the rate at which the station receive/send, but that doesn't mean it's transmitting constantly. https://github.com/openwrt/packages/blob/master/utils/prometheus-node-exporter-lua/files/usr/lib/lua/prometheus-collectors/wifi_stations.lua What you want to use is wifi_station_transmit_bytes_total and wifi_station_receive_bytes_total

[jpds]

What you want to use is wifi_station_transmit_bytes_total and wifi_station_receive_bytes_total

Hmm, I do not see these in my Prometheus metrics explorer, nor when I curl the node_exporter endpoint, are they in the 2021.02.15-1 version of the package?

Edit: I've just checked the filesystem and indeed the code is there - it's just not going into the HTTP endpoint.

[champtar]

You can try the hostapd exporter to see if you get better info

[jpds]

I installed that - did a reboot, and all it reports on this device is:

# TYPE hostapd_station_vht gauge
# TYPE hostapd_station_ht gauge
# TYPE hostapd_station_wmm gauge
# TYPE hostapd_station_mfp gauge
# TYPE hostapd_station_rx_packets counter
# TYPE hostapd_station_rx_bytes counter
# TYPE hostapd_station_tx_packets counter
# TYPE hostapd_station_tx_bytes counter
# TYPE hostapd_station_inactive_msec counter
# TYPE hostapd_station_signal gauge
# TYPE hostapd_station_connected_time counter
# TYPE hostapd_station_wpa gauge
# TYPE hostapd_station_sae_group gauge
# TYPE hostapd_station_vht_capb_su_beamformee gauge
# TYPE hostapd_station_vht_capb_mu_beamformee gauge
node_scrape_collector_duration_seconds{collector="hostapd_stations"} 0.018948793411255
node_scrape_collector_success{collector="hostapd_stations"} 1

[champtar]

Just to be sure you do have something connected when trying (as you just rebooted device might have moved on another ap) Else look at the code it's just running hostapd_cli, try to run it manually

[jpds]

Yes, I can see clients in the LuCI web UI - but indeed, only those two wifi_station_(receive|transmit)_bytes_total are missing (I can see the others) and the hostapd metrics are still empty, but there is:

wifi_stations{ifname="wlan0"} 4

[jpds]

This looks odd:

root@nanohd:~# hostapd_cli -i wlan1 all_sta
root@nanohd:~# hostapd_cli -i wlan0 all_sta

root@nanohd:~# hostapd_cli -i wlan0 status
state=ENABLED
phy=wlan0
freq=2472
num_sta_non_erp=0
num_sta_no_short_slot_time=0
num_sta_no_short_preamble=0
olbc=0
num_sta_ht_no_gf=3
num_sta_no_ht=0
num_sta_ht_20_mhz=2
num_sta_ht40_intolerant=0
olbc_ht=0
ht_op_mode=0x4
cac_time_seconds=0
cac_time_left_seconds=N/A
channel=13
edmg_enable=0
edmg_channel=0
secondary_channel=0
ieee80211n=1
ieee80211ac=0
ieee80211ax=0
beacon_int=100
dtim_period=2
ht_caps_info=01ec
supported_rates=0c 12 18 24 30 48 60 6c
max_txpower=20
bss[0]=wlan0
bssid[0]=...
ssid[0]=,,,
num_sta[0]=3
chan_util_avg=8

[champtar]

can you try master branch by any chance ?

[champtar]

Actually no, you need a full version of hostapd / wpad, see https://github.com/openwrt/packages/commit/af8f8f80469fe595171d9dd4895a7dc0fbe0e29b

[jpds]

can you try master branch by any chance ?

I do not know how to try this - I'm new to OpenWRT.

Actually no, you need a full version of hostapd / wpad

OK, so I had wpad-basic-wolfssl installed (I think - by the system image) - I replaced this doing:

# opkg remove wpad-basic-wolfssl
# opkg install wpad-wolfssl

Then I did a reboot, and now I can see in all_sta:

# hostapd_cli -i wlan1 all_sta
84:cf:bf:...
flags=[AUTH][ASSOC][AUTHORIZED][WMM][MFP][HT][VHT]
aid=2
capability=0x111
listen_interval=1
supported_rates=8c 12 98 24 b0 48 60 6c
timeout_next=NULLFUNC POLL
dot11RSNAStatsSTAAddress=84:cf:bf:...
dot11RSNAStatsVersion=1
dot11RSNAStatsSelectedPairwiseCipher=00-0f-ac-4
dot11RSNAStatsTKIPLocalMICFailures=0
dot11RSNAStatsTKIPRemoteMICFailures=0
wpa=2
AKMSuiteSelector=00-0f-ac-6
hostapdWPAPTKState=11
hostapdWPAPTKGroupState=0
rx_packets=10095
tx_packets=9928
rx_bytes=2239142
tx_bytes=16666955
inactive_msec=6312
signal=-75
rx_rate_info=60
tx_rate_info=3900 vhtmcs 8 vhtnss 1 shortGI
rx_vht_mcs_map=fffe
tx_vht_mcs_map=fffe
ht_mcs_bitmask=ff000000000000000000
connected_time=1438
min_txpower=8
max_txpower=20
vht_caps_info=0x33907120
ht_caps_info=0x016e
ext_capab=0000000200000040

I can see the rx_bytes and tx_bytes in the above - however, I still cannot see the bytes_total counters in node_exporter:

$ curl localhost:9100/metrics | grep wifi | grep 84
wifi_station_signal_dbm{mac="84:CF:BF:...",ifname="wlan1"} -79
wifi_station_inactive_milliseconds{mac="84:CF:BF:...",ifname="wlan1"} 13984
wifi_station_expected_throughput_kilobits_per_second{mac="84:CF:BF:...",ifname="wlan1"} 212718
wifi_station_transmit_kilobits_per_second{mac="84:CF:BF:...",ifname="wlan1"} 325000
wifi_station_receive_kilobits_per_second{mac="84:CF:BF:...",ifname="wlan1"} 6000
wifi_station_transmit_packets_total{mac="84:CF:BF:...",ifname="wlan1"} 10228
wifi_station_receive_packets_total{mac="84:CF:BF:...",ifname="wlan1"} 10483

I have also tried restarting node_exporter as well as rebooting.

[alandebortolo]

I've accidentally installed this version on my Kimax device, and now, can't access because I don't have the password. Can you provide me?

[champtar]

Merged #23683 in master