search

bairhys / prometheus-cake-autorate-exporter

A Prometheus exporter for CAKE Autorate
GNU General Public License v3.0
3 stars 1 forks source link

Manually explorer start - ValueError: could not convert string to float: ' [1723422345.39835]' #6

Open MisiekDP opened 2 months ago

MisiekDP commented 2 months ago

Unfortunately when running explorer manually I get an error with incorrect value conversion:

root@Xiaomi_AX3600:~# python prometheus_cake_autorate_exporter.py
2024-08-12 02:25:45,417 - Log file /var/log/cake-autorate.primary.log, metrics at: http://localhost:9101/metrics
Traceback (most recent call last):
  File "/root/prometheus_cake_autorate_exporter.py", line 155, in <module>
    readLineData(data)
  File "/root/prometheus_cake_autorate_exporter.py", line 64, in readLineData
    ICMP_TIMESTAMP.set(data[8])
  File "/usr/lib/python3.11/site-packages/prometheus_client/metrics.py", line 438, in set
    self._value.set(float(value))
                    ^^^^^^^^^^^^
ValueError: could not convert string to float: ' [1723422345.39835]'

and:

root@Xiaomi_AX3600:~# python prometheus_cake_autorate_exporter.py
2024-08-12 02:16:19,687 - Log file /var/log/cake-autorate.primary.log, metrics at: http://localhost:9101/metrics
Traceback (most recent call last):
  File "/root/prometheus_cake_autorate_exporter.py", line 155, in <module>
    readLineData(data)
  File "/root/prometheus_cake_autorate_exporter.py", line 56, in readLineData
    reflector = data[9].replace(' ', '')
                ~~~~^^^
IndexError: list index out of range

My logs:

root@Xiaomi_AX3600:~# tail -f /var/log/cake-autorate.primary.log | grep -e SUMMARY
SUMMARY; 2024-08-12-02:16:41; 1723421801.652599; 23; 46; 0; 0; 973; 973; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:41; 1723421801.701377; 23; 46; 0; 0; 718; 718; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:41; 1723421801.754100; 23; 46; 0; 0; 755; 755; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:41; 1723421801.803609; 20; 23; 0; 0; 866; 866; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:41; 1723421801.852652; 20; 23; 0; 0; 974; 974; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:41; 1723421801.901629; 20; 23; 0; 0; 772; 772; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:41; 1723421801.954732; 20; 23; 0; 0; 888; 888; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.003498; 24; 30; 0; 0; 1010; 1010; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.052425; 24; 30; 0; 0; 853; 853; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.104903; 24; 30; 0; 0; 930; 930; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.153806; 24; 30; 0; 0; 986; 986; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.202891; 31; 34; 0; 0; 1052; 1052; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.252170; 31; 34; 0; 0; 812; 812; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.304786; 31; 34; 0; 0; 884; 884; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.354812; 31; 34; 0; 0; 1053; 1053; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.402976; 21; 23; 0; 0; 840; 840; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.452368; 21; 23; 0; 0; 685; 685; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.505762; 21; 23; 0; 0; 874; 874; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.554775; 21; 23; 0; 0; 1047; 1047; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.603495; 25; 30; 0; 0; 900; 900; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.652759; 25; 30; 0; 0; 683; 683; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.705243; 25; 30; 0; 0; 851; 851; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.755141; 25; 30; 0; 0; 1016; 1016; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.804367; 18; 20; 0; 0; 865; 865; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.853307; 18; 20; 0; 0; 701; 701; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.903616; 18; 20; 0; 0; 658; 658; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:42; 1723421802.955803; 18; 20; 0; 0; 850; 850; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.005005; 61; 45; 0; 0; 779; 779; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.056068; 61; 45; 0; 0; 827; 827; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.105316; 61; 45; 0; 0; 845; 845; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.155187; 61; 45; 0; 0; 961; 961; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.204276; 34; 34; 0; 0; 981; 981; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.256757; 34; 34; 0; 0; 1041; 1041; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.305344; 34; 34; 0; 0; 1025; 1025; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.354502; 34; 34; 0; 0; 866; 866; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.404254; 36; 155; 0; 0; 755; 755; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.456852; 36; 155; 0; 0; 850; 850; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.506065; 36; 155; 0; 0; 962; 962; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.555030; 36; 155; 0; 0; 769; 769; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.604504; 18; 25; 0; 0; 647; 647; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.656972; 18; 25; 0; 0; 808; 808; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.705950; 18; 25; 0; 0; 909; 909; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.755455; 18; 25; 0; 0; 730; 730; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.804571; 22; 24; 0; 0; 564; 564; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.857357; 22; 24; 0; 0; 717; 717; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.905219; 22; 24; 0; 0; 724; 724; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:43; 1723421803.957309; 22; 24; 0; 0; 711; 711; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.007108; 311; 254; 0; 0; 752; 752; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.055954; 311; 254; 0; 0; 764; 764; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.108623; 311; 254; 0; 0; 1050; 1050; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.157496; 311; 254; 0; 0; 1005; 1005; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.206076; 134; 245; 0; 0; 1046; 1046; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.258711; 134; 245; 0; 0; 1117; 1117; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.307667; 134; 245; 0; 0; 1129; 1129; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.357163; 134; 245; 0; 0; 1170; 1170; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.405683; 34; 43; 0; 0; 884; 884; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.458309; 34; 43; 0; 0; 925; 925; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.506969; 34; 43; 0; 0; 975; 975; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.556162; 34; 43; 0; 0; 723; 723; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.609148; 39; 49; 0; 0; 814; 814; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.658215; 39; 49; 0; 0; 889; 889; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.706976; 39; 49; 0; 0; 963; 963; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.756734; 39; 49; 0; 0; 773; 773; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.808525; 20; 24; 0; 0; 866; 866; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.857439; 20; 24; 0; 0; 940; 940; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.906951; 20; 24; 0; 0; 702; 702; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:44; 1723421804.959454; 20; 24; 0; 0; 763; 763; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.008538; 20; 19; 0; 0; 859; 859; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.057464; 20; 19; 0; 0; 888; 888; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.109646; 20; 19; 0; 0; 936; 936; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.158689; 20; 19; 0; 0; 990; 990; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.207881; 26; 27; 0; 0; 1036; 1036; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.260833; 26; 27; 0; 0; 1111; 1111; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.309862; 26; 27; 0; 0; 1147; 1147; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.358386; 26; 27; 0; 0; 1189; 1189; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.406909; 23; 26; 0; 0; 921; 921; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.459954; 23; 26; 0; 0; 979; 979; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.509189; 23; 26; 0; 0; 1025; 1025; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.557946; 23; 26; 0; 0; 741; 741; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.610976; 18; 24; 0; 0; 795; 795; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.659286; 18; 24; 0; 0; 832; 832; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.708242; 18; 24; 0; 0; 884; 884; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:16:45; 1723421805.761717; 18; 24; 0; 0; 940; 940; dl_idle; ul_idle; 500000; 60000

As far as the settings for reflector are the default (nothing is changed).

MisiekDP commented 2 months ago

Last few lines from the log file /var/log/cake-autorate.primary.log:


DATA; 2024-08-12-02:44:25; 1723423465.003639; 1723423465.002786; 15; 16; 0; 0; [1723423464.99987]; 9.9.9.11; 5370; 3409; 4380; 932; 971; 30024; 3409; 4380; 932; 971; 30205; 0; 967; 10024; 60024; 0; 967; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.004893; 15; 16; 0; 0; 967; 967; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.052712; 1723423465.051865; 15; 16; 0; 0; [1723423465.04910]; 8.8.4.4; 5370; 3518; 3930; 840; 412; 30024; 3518; 3930; 840; 412; 30205; 0; 998; 10024; 60024; 0; 998; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.053802; 15; 16; 0; 0; 998; 998; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.105647; 1723423465.104878; 15; 16; 0; 0; [1723423465.10218]; 185.228.168.9; 5371; 3550; 5400; 1040; 1850; 30024; 3550; 5400; 1040; 1850; 30205; 0; 1078; 10024; 60024; 0; 1078; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.106699; 15; 16; 0; 0; 1078; 1078; dl_idle; ul_idle; 500000; 60000
REFLECTOR; 2024-08-12-02:44:25; 1723423465.111492; 1723423465.110997; 185.228.168.9; 6676; 7100; 424; 20000; 840; 1040; 200; 10000; 840; 1040; 200; 10000
REFLECTOR; 2024-08-12-02:44:25; 1723423465.113069; 1723423465.112618; 156.154.70.5; 6676; 7030; 354; 20000; 840; 895; 55; 10000; 840; 895; 55; 10000
REFLECTOR; 2024-08-12-02:44:25; 1723423465.114488; 1723423465.114043; 8.8.8.8; 6676; 6864; 188; 20000; 840; 1025; 185; 10000; 840; 1025; 185; 10000
REFLECTOR; 2024-08-12-02:44:25; 1723423465.115895; 1723423465.115444; 208.67.222.123; 6676; 6676; 0; 20000; 840; 1009; 169; 10000; 840; 1009; 169; 10000
REFLECTOR; 2024-08-12-02:44:25; 1723423465.117436; 1723423465.116945; 9.9.9.11; 6676; 6818; 142; 20000; 840; 932; 92; 10000; 840; 932; 92; 10000
REFLECTOR; 2024-08-12-02:44:25; 1723423465.118934; 1723423465.118480; 8.8.4.4; 6676; 7036; 360; 20000; 840; 840; 0; 10000; 840; 840; 0; 10000
DATA; 2024-08-12-02:44:25; 1723423465.154101; 1723423465.153327; 15; 16; 0; 0; [1723423465.15053]; 156.154.70.5; 5371; 3516; 4540; 907; 1024; 30024; 3516; 4540; 907; 1024; 30205; 0; 1072; 10024; 60024; 0; 1072; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.155189; 15; 16; 0; 0; 1072; 1072; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:25; 1723423465.175558; 1723423465.175312; 18; 47; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.203849; 1723423465.202854; 18; 47; 0; 0; [1723423465.19936]; 8.8.8.8; 5371; 3432; 3915; 973; 483; 30024; 3432; 3915; 973; 483; 30205; 0; 1083; 10024; 60024; 0; 1083; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.205031; 18; 47; 0; 0; 1083; 1083; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.256113; 1723423465.255135; 18; 47; 0; 0; [1723423465.25186]; 208.67.222.123; 5371; 3339; 5100; 1080; 1761; 30024; 3339; 5100; 1080; 1761; 30205; 0; 1083; 10024; 60024; 0; 1083; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.257294; 18; 47; 0; 0; 1083; 1083; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.304937; 1723423465.303966; 18; 47; 0; 0; [1723423465.30073]; 9.9.9.11; 5371; 3410; 4515; 948; 1105; 30024; 3410; 4515; 948; 1105; 30205; 0; 1105; 10024; 60024; 0; 1105; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.306079; 18; 47; 0; 0; 1105; 1105; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.354034; 1723423465.353057; 18; 47; 0; 0; [1723423465.34980]; 8.8.4.4; 5371; 3518; 4005; 806; 487; 30024; 3518; 4005; 806; 487; 30205; 0; 1118; 10024; 60024; 0; 1118; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.355164; 18; 47; 0; 0; 1118; 1118; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:25; 1723423465.377268; 1723423465.376969; 40; 51; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.403470; 1723423465.402490; 40; 51; 0; 0; [1723423465.39909]; 185.228.168.9; 5372; 3550; 3605; 946; 55; 30024; 3550; 3605; 946; 55; 30205; 0; 819; 10024; 60024; 0; 819; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.404607; 40; 51; 0; 0; 819; 819; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.455946; 1723423465.454918; 40; 51; 0; 0; [1723423465.45155]; 156.154.70.5; 5372; 3517; 4790; 941; 1273; 30024; 3517; 4790; 941; 1273; 30205; 0; 860; 10024; 60024; 0; 860; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.457105; 40; 51; 0; 0; 860; 860; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.504647; 1723423465.503671; 40; 51; 0; 0; [1723423465.50039]; 8.8.8.8; 5372; 3432; 4165; 950; 733; 30024; 3432; 4165; 950; 733; 30205; 0; 902; 10024; 60024; 0; 902; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.505783; 40; 51; 0; 0; 902; 902; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.553513; 1723423465.552469; 40; 51; 0; 0; [1723423465.54910]; 208.67.222.123; 5372; 3339; 3470; 989; 131; 30024; 3339; 3470; 989; 131; 30205; 0; 630; 10024; 60024; 0; 630; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.554674; 40; 51; 0; 0; 630; 630; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:25; 1723423465.579000; 1723423465.578697; 31; 34; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.606026; 1723423465.605048; 31; 34; 0; 0; [1723423465.60185]; 9.9.9.11; 5372; 3411; 4805; 990; 1394; 30024; 3411; 4805; 990; 1394; 30205; 0; 678; 10024; 60024; 0; 678; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.607193; 31; 34; 0; 0; 678; 678; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.655265; 1723423465.654291; 31; 34; 0; 0; [1723423465.65108]; 8.8.4.4; 5372; 3518; 4375; 810; 857; 30024; 3518; 4375; 810; 857; 30205; 0; 740; 10024; 60024; 0; 740; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.656394; 31; 34; 0; 0; 740; 740; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.704191; 1723423465.703217; 31; 34; 0; 0; [1723423465.70007]; 185.228.168.9; 5373; 3550; 3830; 882; 280; 30024; 3550; 3830; 882; 280; 30205; 0; 778; 10024; 60024; 0; 778; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.705309; 31; 34; 0; 0; 778; 778; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.757334; 1723423465.755961; 31; 34; 0; 0; [1723423465.75277]; 156.154.70.5; 5373; 3518; 5150; 1006; 1632; 30024; 3518; 5150; 1006; 1632; 30205; 0; 837; 10024; 60024; 0; 837; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.758529; 31; 34; 0; 0; 837; 837; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:25; 1723423465.780702; 1723423465.780395; 2513; 20; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.806359; 1723423465.805366; 2513; 20; 0; 0; [1723423465.80165]; 8.8.8.8; 5373; 3433; 4535; 964; 1102; 30024; 3433; 4535; 964; 1102; 30205; 0; 899; 10024; 60024; 0; 899; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.807993; 2513; 20; 0; 0; 899; 899; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.854668; 1723423465.853677; 2513; 20; 0; 0; [1723423465.85019]; 208.67.222.123; 5373; 3339; 3755; 934; 416; 30024; 3339; 3755; 934; 416; 30205; 0; 946; 10024; 60024; 0; 946; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.855857; 2513; 20; 0; 0; 946; 946; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.904996; 1723423465.904006; 2513; 20; 0; 0; [1723423465.89955]; 9.9.9.11; 5373; 3387; 3385; 895; -2; 30024; 3387; 3385; 895; -2; 30205; 0; 714; 10024; 60024; 0; 714; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.906181; 2513; 20; 0; 0; 714; 714; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:25; 1723423465.956460; 1723423465.955486; 2513; 20; 0; 0; [1723423465.95222]; 8.8.4.4; 5373; 3519; 4675; 842; 1156; 30024; 3519; 4675; 842; 1156; 30205; 0; 764; 10024; 60024; 0; 764; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:25; 1723423465.957595; 2513; 20; 0; 0; 764; 764; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:25; 1723423465.982448; 1723423465.982145; 12358; 295; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.006311; 1723423466.005335; 12358; 295; 2; 0; [1723423466.00185]; 185.228.168.9; 5374; 3550; 4440; 882; 890; 30024; 3550; 4440; 882; 890; 30205; 0; 865; 10024; 60024; 0; 865; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.007444; 12358; 295; 0; 0; 865; 865; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.054701; 1723423466.053720; 12358; 295; 2; 0; [1723423466.05048]; 156.154.70.5; 5374; 3518; 3720; 929; 202; 30024; 3518; 3720; 929; 202; 30205; 0; 627; 10024; 60024; 0; 627; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.055827; 12358; 295; 0; 0; 627; 627; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.107428; 1723423466.106453; 12358; 295; 2; 0; [1723423466.10334]; 8.8.8.8; 5374; 3434; 5100; 1030; 1666; 30024; 3434; 5100; 1030; 1666; 30205; 0; 721; 10024; 60024; 0; 721; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.108540; 12358; 295; 0; 0; 721; 721; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.155911; 1723423466.154932; 12358; 295; 2; 0; [1723423466.15173]; 208.67.222.123; 5374; 3339; 4265; 933; 926; 30024; 3339; 4265; 933; 926; 30205; 0; 806; 10024; 60024; 0; 806; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.157038; 12358; 295; 0; 0; 806; 806; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:26; 1723423466.184090; 1723423466.183793; 26; 27; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.204867; 1723423466.203895; 26; 27; 0; 0; [1723423466.20078]; 9.9.9.11; 5374; 3387; 3745; 843; 358; 30024; 3387; 3745; 843; 358; 30205; 0; 866; 10024; 60024; 0; 866; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.206010; 26; 27; 0; 0; 866; 866; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.258626; 1723423466.257453; 26; 27; 0; 0; [1723423466.25419]; 8.8.4.4; 5374; 3520; 5400; 940; 1880; 30024; 3520; 5400; 940; 1880; 30205; 0; 987; 10024; 60024; 0; 987; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.259823; 26; 27; 0; 0; 987; 987; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.306873; 1723423466.305897; 26; 27; 0; 0; [1723423466.30272]; 185.228.168.9; 5375; 3551; 4630; 900; 1079; 30024; 3551; 4630; 900; 1079; 30205; 0; 1018; 10024; 60024; 0; 1018; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.307987; 26; 27; 0; 0; 1018; 1018; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.355718; 1723423466.354745; 26; 27; 0; 0; [1723423466.35155]; 156.154.70.5; 5375; 3518; 4005; 887; 487; 30024; 3518; 4005; 887; 487; 30205; 0; 1066; 10024; 60024; 0; 1066; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.356845; 26; 27; 0; 0; 1066; 1066; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:26; 1723423466.385767; 1723423466.385468; 23; 34; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.408254; 1723423466.407278; 23; 34; 0; 0; [1723423466.40405]; 8.8.8.8; 5375; 3435; 5200; 1099; 1765; 30024; 3435; 5200; 1099; 1765; 30205; 0; 1082; 10024; 60024; 0; 1082; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.409384; 23; 34; 0; 0; 1082; 1082; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.457175; 1723423466.456186; 23; 34; 0; 0; [1723423466.45291]; 208.67.222.123; 5375; 3340; 4600; 964; 1260; 30024; 3340; 4600; 964; 1260; 30205; 0; 1138; 10024; 60024; 0; 1138; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.458341; 23; 34; 0; 0; 1138; 1138; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.506188; 1723423466.505216; 23; 34; 0; 0; [1723423466.50200]; 9.9.9.11; 5375; 3387; 4095; 830; 708; 30024; 3387; 4095; 830; 708; 30205; 0; 1196; 10024; 60024; 0; 1196; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.507338; 23; 34; 0; 0; 1196; 1196; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.555967; 1723423466.554994; 23; 34; 0; 0; [1723423466.55173]; 8.8.4.4; 5375; 3520; 3925; 889; 405; 30024; 3520; 3925; 889; 405; 30205; 0; 950; 10024; 60024; 0; 950; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.557096; 23; 34; 0; 0; 950; 950; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:26; 1723423466.587494; 1723423466.587193; 26; 31; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.608230; 1723423466.607250; 26; 31; 0; 0; [1723423466.60402]; 185.228.168.9; 5376; 3552; 5050; 956; 1498; 30024; 3552; 5050; 956; 1498; 30205; 0; 1020; 10024; 60024; 0; 1020; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.609358; 26; 31; 0; 0; 1020; 1020; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.657173; 1723423466.656198; 26; 31; 0; 0; [1723423466.65296]; 156.154.70.5; 5376; 3518; 4450; 891; 932; 30024; 3518; 4450; 891; 932; 30205; 0; 1094; 10024; 60024; 0; 1094; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.658309; 26; 31; 0; 0; 1094; 1094; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.705911; 1723423466.704935; 26; 31; 0; 0; [1723423466.70173]; 8.8.8.8; 5376; 3435; 3790; 1028; 355; 30024; 3435; 3790; 1028; 355; 30205; 0; 859; 10024; 60024; 0; 859; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.707043; 26; 31; 0; 0; 859; 859; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.755452; 1723423466.754476; 26; 31; 0; 0; [1723423466.75124]; 208.67.222.123; 5376; 3340; 3500; 887; 160; 30024; 3340; 3500; 887; 160; 30205; 0; 676; 10024; 60024; 0; 676; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.756599; 26; 31; 0; 0; 676; 676; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:26; 1723423466.789194; 1723423466.788892; 22; 24; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.807848; 1723423466.806870; 22; 24; 0; 0; [1723423466.80374]; 9.9.9.11; 5376; 3388; 4710; 876; 1322; 30024; 3388; 4710; 876; 1322; 30205; 0; 778; 10024; 60024; 0; 778; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.809041; 22; 24; 0; 0; 778; 778; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.857137; 1723423466.856159; 22; 24; 0; 0; [1723423466.85290]; 8.8.4.4; 5376; 3520; 4245; 873; 725; 30024; 3520; 4245; 873; 725; 30205; 0; 832; 10024; 60024; 0; 832; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.858277; 22; 24; 0; 0; 832; 832; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.906213; 1723423466.905228; 22; 24; 0; 0; [1723423466.90188]; 185.228.168.9; 5377; 3552; 3685; 877; 133; 30024; 3552; 3685; 877; 133; 30205; 0; 604; 10024; 60024; 0; 604; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.907361; 22; 24; 0; 0; 604; 604; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:26; 1723423466.958660; 1723423466.957681; 22; 24; 0; 0; [1723423466.95450]; 156.154.70.5; 5377; 3519; 4955; 942; 1436; 30024; 3519; 4955; 942; 1436; 30205; 0; 688; 10024; 60024; 0; 688; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:26; 1723423466.959791; 22; 24; 0; 0; 688; 688; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:26; 1723423466.990968; 1723423466.990605; 20; 24; 500000; 60000
DATA; 2024-08-12-02:44:27; 1723423467.007859; 1723423467.006887; 20; 24; 0; 0; [1723423467.00346]; 8.8.8.8; 5377; 3435; 4390; 1021; 955; 30024; 3435; 4390; 1021; 955; 30205; 0; 788; 10024; 60024; 0; 788; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:27; 1723423467.008982; 20; 24; 0; 0; 788; 788; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:27; 1723423467.056000; 1723423467.055015; 20; 24; 0; 0; [1723423467.05186]; 208.67.222.123; 5377; 3340; 3550; 822; 210; 30024; 3340; 3550; 822; 210; 30205; 0; 796; 10024; 60024; 0; 796; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:27; 1723423467.057193; 20; 24; 0; 0; 796; 796; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:27; 1723423467.108856; 1723423467.107881; 20; 24; 0; 0; [1723423467.10471]; 9.9.9.11; 5377; 3389; 4925; 938; 1536; 30024; 3389; 4925; 938; 1536; 30205; 0; 832; 10024; 60024; 0; 832; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:27; 1723423467.110005; 20; 24; 0; 0; 832; 832; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:27; 1723423467.157996; 1723423467.157017; 20; 24; 0; 0; [1723423467.15390]; 8.8.4.4; 5377; 3520; 4480; 881; 960; 30024; 3520; 4480; 881; 960; 30205; 0; 871; 10024; 60024; 0; 871; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:27; 1723423467.159157; 20; 24; 0; 0; 871; 871; dl_idle; ul_idle; 500000; 60000
LOAD; 2024-08-12-02:44:27; 1723423467.192575; 1723423467.192276; 23; 24; 500000; 60000
DATA; 2024-08-12-02:44:27; 1723423467.206818; 1723423467.205839; 23; 24; 0; 0; [1723423467.20277]; 185.228.168.9; 5378; 3552; 3875; 824; 323; 30024; 3552; 3875; 824; 323; 30205; 0; 903; 10024; 60024; 0; 903; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:27; 1723423467.207921; 23; 24; 0; 0; 903; 903; dl_idle; ul_idle; 500000; 60000
DATA; 2024-08-12-02:44:27; 1723423467.259555; 1723423467.258583; 23; 24; 0; 0; [1723423467.25532]; 156.154.70.5; 5378; 3520; 5100; 1002; 1580; 30024; 3520; 5100; 1002; 1580; 30205; 0; 927; 10024; 60024; 0; 927; 10205; 60205; dl_idle; ul_idle; 500000; 60000
SUMMARY; 2024-08-12-02:44:27; 1723423467.260697; 23; 2
lynxthecat commented 2 months ago

@bairhys it's just because the log line formats have changed again. And now there are CPU usage log lines that show CPU usage on each detected CPU core. 

DEBUG; 2024-08-12-12:11:08; 1723461068.464175; Detected 2 CPU cores.
SUMMARY_HEADER; LOG_DATETIME; LOG_TIMESTAMP; DL_ACHIEVED_RATE_KBPS; UL_ACHIEVED_RATE_KBPS; DL_SUM_DELAYS; UL_SUM_DELAYS; DL_AVG_OWD_DELTA_US; UL_AVG_OWD_DELTA_US; DL_LOAD_CONDITION; UL_LOAD_CONDITION; CAKE_DL_RATE_KBPS; CAKE_UL_RATE_KBPS
CPU_HEADER; LOG_DATETIME; LOG_TIMESTAMP; STATS_READ_TIME; CPU_USAGE; CPU0_USAGE; CPU1_USAGE
CPU_RAW_HEADER; LOG_DATETIME; LOG_TIMESTAMP; STATS_READ_TIME; CPU_ID; USER; NICE; SYSTEM; IDLE; IOWAIT; IRQ; SIRQ; STEAL; GUEST; GUEST_NICE
...
CPU_RAW; 2024-08-12-12:11:10; 1723461070.565500; 1723461070.564537; cpu; 34529430; 0; 9549478; 529241700; 97; 1144417; 2789852; 0; 0; 0
CPU_RAW; 2024-08-12-12:11:10; 1723461070.567101; 1723461070.564537; cpu0; 17017115; 0; 4800492; 264553169; 52; 611703; 1581935; 0; 0; 0
CPU_RAW; 2024-08-12-12:11:10; 1723461070.568870; 1723461070.564537; cpu1; 17512314; 0; 4748985; 264688530; 45; 532713; 1207916; 0; 0; 0
CPU; 2024-08-12-12:11:10; 1723461070.570006; 1723461070.564537; 8; 8; 8

Any chance of updating the exporter to reflect the latest code on the master branch of cake-autorate?

bairhys commented 2 months ago

Hi @MisiekDP and @lynxthecat, I haven't updated my cake autorate for a long time since it has been working well but today I have updated my cake autorate to the latest prerelease. I also get the same issue. I have updated the code that reads data lines but not finished yet. The code I am working on is in this branch.

MisiekDP commented 2 months ago

Okey @bairhys thanks for reply. Could you maybe write here to inform me when you finish the code? I would like to test it too! :)

lynxthecat commented 2 months ago

Thanks @bairhys. I am tempted to test this out too since the visualization it provides is phenomenal. Too bad I don't have a spare RPi4 on my network (just routers and computers that get switched off and on).

bairhys commented 2 months ago

Okey @bairhys thanks for reply. Could you maybe write here to inform me when you finish the code? I would like to test it too! :)

You can use it now, I am not happy with it but it has been running the last 24hours. In my install instructions, use this link instead to download the exporter:

wget https://raw.githubusercontent.com/bairhys/prometheus-cake-autorate-exporter/6-manually-explorer-start-valueerror-could-not-convert-string-to-float-172342234539835/prometheus_cake_autorate_exporter.py
bairhys commented 2 months ago

Thanks @bairhys. I am tempted to test this out too since the visualization it provides is phenomenal. Too bad I don't have a spare RPi4 on my network (just routers and computers that get switched off and on).

If you want, can install docker on a computer like Windows and get this working pretty easy. Then it will collect data whenever your computer is on which is probably fine 😉

lynxthecat commented 2 months ago

Can it fill in the gap since it was last run by grabbing and plotting the intervening historical data? I’ve been thinking about running the log data through gzip albeit getting the implementation of that right could be tricky.

MisiekDP commented 2 months ago

Okey @lynxthecat @bairhys I did a small tests on Windows and it looks really nice. Now I see that I can try to start configure cake-autorate :)

Btw. when I manually run prometheus script I got logs with "Data too short" like this:

2024-08-14 17:49:25,311 - DATA too short 29, ['DATA', ' 2024-08-14-17:49:25', ' 1723650565.248910', ' 1723650565.247930', ' 15', ' 16', ' 0', ' 0', ' 1723650565.24476', ' 9.9.9.11', ' 1693', ' 3298', ' 4335', ' 1728', ' 1037', ' 30024', ' 3298', ' 4335', ' 1728', ' 1037', ' 30205', ' 0', ' 869', ' 10024', ' 60024', ' 0', ' 869', ' 10205', '']
2024-08-14 17:49:29,360 - DATA too short 2, ['DATA', ' 2024-08']
2024-08-14 17:49:30,373 - DATA too short 29, ['DATA', ' 2024-08-14-17:49:30', ' 1723650570.358049', ' 1723650570.357064', ' 22', ' 20', ' 0', ' 0', ' 1723650570.35378', ' 9.9.9.11', ' 1710', ' 3308', ' 4505', ' 1130', ' 1197', ' 30024', ' 3308', ' 4505', ' 1130', ' 1197', ' 30205', ' 0', ' 952', ' 10024', ' 60024', ' 0', ' 952', ' 10205', ' 6']
2024-08-14 17:49:48,604 - DATA too short 18, ['DATA', ' 2024-08-14-17:49:48', ' 1723650588.587640', ' 1723650588.586664', ' 21', ' 26', ' 0', ' 0', ' 1723650588.58351', ' 156.154.70.5', ' 2', ' 3505', ' 4405', ' 920', ' 900', ' 30024', ' 3505', '']
2024-08-14 17:49:51,643 - DATA too short 15, ['DATA', ' 2024-08-14-17:49:51', ' 1723650591.593295', ' 1723650591.592315', ' 30785', ' 326', ' 6', ' 0', ' 1723650591.58909', ' 156.154.70.5', ' 12', ' 3499', ' 4020', ' 742', ' 5']
2024-08-14 17:49:52,656 - DATA too short 9, ['DATA', ' 2024-08-14-17:49:52', ' 1723650592.615141', ' 1723650592.614169', ' 17', ' 23', ' 0', ' 0', ' 172365059']
2024-08-14 17:49:53,670 - DATA too short 31, ['DATA', ' 2024-08-14-17:49:53', ' 1723650593.596659', ' 1723650593.595650', ' 38', ' 33', ' 0', ' 0', ' 1723650593.59239', ' 9.9.9.11', ' 18', ' 3324', ' 3870', ' 970', ' 546', ' 30024', ' 3324', ' 3870', ' 970', ' 546', ' 30205', ' 0', ' 1114', ' 10024', ' 60024', ' 0', ' 1114', ' 10205', ' 60205', ' dl_idle', '']
2024-08-14 17:49:56,712 - DATA too short 29, ['DATA', ' 2024-08-14-17:49:56', ' 1723650596.703800', ' 1723650596.702322', ' 21', ' 23', ' 0', ' 0', ' 1723650596.69723', ' 156.154.70.5', ' 29', ' 3508', ' 3575', ' 869', ' 67', ' 30024', ' 3508', ' 3575', ' 869', ' 67', ' 30205', ' 0', ' 668', ' 10024', ' 60024', ' 0', ' 668', ' 10205', ' 60']
2024-08-14 17:49:58,740 - DATA too short 28, ['DATA', ' 2024-08-14-17:49:58', ' 1723650598.675536', ' 1723650598.674561', ' 21', ' 63', ' 0', ' 0', ' 1723650598.67126', ' 156.154.71.2', ' 35', ' 13147', ' 14000', ' 766', ' 853', ' 30024', ' 13147', ' 14000', ' 766', ' 853', ' 30205', ' 0', ' 1034', ' 10024', ' 60024', ' 0', ' 1034', ' 10205']
2024-08-14 17:50:01,783 - DATA too short 29, ['DATA', ' 2024-08-14-17:50:01', ' 1723650601.779725', ' 1723650601.778737', ' 18', ' 19', ' 0', ' 0', ' 1723650601.77537', ' 156.154.71.3', ' 45', ' 12503', ' 13600', ' 1072', ' 1097', ' 30024', ' 12503', ' 13600', ' 1072', ' 1097', ' 30205', ' 0', ' 1172', ' 10024', ' 60024', ' 0', ' 1172', ' 10205', ' 602']
2024-08-14 17:50:04,828 - DATA too short 9, ['DATA', ' 2024-08-14-17:50:04', ' 1723650604.815591', ' 1723650604.814454', ' 25', ' 61', ' 0', ' 0', ' 1723650604.']
2024-08-14 17:50:05,844 - DATA too short 17, ['DATA', ' 2024-08-14-17:50:05', ' 1723650605.836552', ' 1723650605.835482', ' 28', ' 37', ' 0', ' 0', ' 1723650605.83169', ' 156.154.71.5', ' 59', ' 12453', ' 13400', ' 977', ' 947', ' 30024', ' 12']
2024-08-14 17:50:06,859 - DATA too short 30, ['DATA', ' 2024-08-14-17:50:06', ' 1723650606.819490', ' 1723650606.818510', ' 18', ' 19', ' 0', ' 0', ' 1723650606.81521', ' 9.9.9.11', ' 62', ' 3347', ' 4360', ' 971', ' 1013', ' 30024', ' 3347', ' 4360', ' 971', ' 1013', ' 30205', ' 0', ' 820', ' 10024', ' 60024', ' 0', ' 820', ' 10205', ' 60205', ' dl_']
2024-08-14 17:50:09,904 - DATA too short 12, ['DATA', ' 2024-08-14-17:50:09', ' 1723650609.894164', ' 1723650609.893144', ' 273', ' 31', ' 0', ' 0', ' 1723650609.88960', ' 156.154.71.3', ' 72', '']
2024-08-14 17:50:12,950 - DATA too short 10, ['DATA', ' 2024-08-14-17:50:12', ' 1723650612.929822', ' 1723650612.928846', ' 38', ' 38', ' 0', ' 0', ' 1723650612.92560', ' 156.154.70']
2024-08-14 17:50:13,966 - DATA too short 30, ['DATA', ' 2024-08-14-17:50:13', ' 1723650613.948530', ' 1723650613.947544', ' 24', ' 31', ' 0', ' 0', ' 1723650613.94398', ' 156.154.71.5', ' 86', ' 12462', ' 12600', ' 677', ' 138', ' 30024', ' 12462', ' 12600', ' 677', ' 138', ' 30205', ' 0', ' 826', ' 10024', ' 60024', ' 0', ' 826', ' 10205', ' 60205', ' dl_i']
2024-08-14 17:50:15,997 - DATA too short 2, ['DATA', ' 2024-08-14-17:5']
2024-08-14 17:50:17,014 - DATA too short 2, ['DATA', ' 2024-08']
2024-08-14 17:50:18,030 - DATA too short 19, ['DATA', ' 2024-08-14-17:50:18', ' 1723650618.008650', ' 1723650618.007662', ' 22', ' 19', ' 0', ' 0', ' 1723650618.00441', ' 156.154.71.3', ' 99', ' 12529', ' 14250', ' 1111', ' 1721', ' 30024', ' 12529', ' 14250', ' 1111']
2024-08-14 17:50:22,106 - DATA too short 4, ['DATA', ' 2024-08-14-17:50:22', ' 1723650622.063106', ' 17']
2024-08-14 17:50:25,155 - DATA too short 6, ['DATA', ' 2024-08-14-17:50:25', ' 1723650625.149473', ' 1723650625.148484', ' 635', ' ']
2024-08-14 17:50:26,171 - DATA too short 10, ['DATA', ' 2024-08-14-17:50:26', ' 1723650626.152930', ' 1723650626.151950', ' 23', ' 24', ' 0', ' 0', ' 1723650626.14873', ' 156.154.70.']
2024-08-14 17:50:27,188 - DATA too short 22, ['DATA', ' 2024-08-14-17:50:27', ' 1723650627.173220', ' 1723650627.172211', ' 23', ' 24', ' 0', ' 0', ' 1723650627.16894', ' 156.154.71.5', ' 130', ' 12480', ' 13750', ' 925', ' 1270', ' 30024', ' 12480', ' 13750', ' 925', ' 1270', ' 30205', ' 0']
2024-08-14 17:50:30,238 - DATA too short 4, ['DATA', ' 2024-08-14-17:50:30', ' 1723650630.230085', ' 1723']
2024-08-14 17:50:31,255 - DATA too short 10, ['DATA', ' 2024-08-14-17:50:31', ' 1723650631.233907', ' 1723650631.232924', ' 20', ' 20', ' 0', ' 0', ' 1723650631.22963', ' 156.']
2024-08-14 17:50:32,271 - DATA too short 21, ['DATA', ' 2024-08-14-17:50:32', ' 1723650632.216136', ' 1723650632.215156', ' 24', ' 230', ' 0', ' 0', ' 1723650632.21185', ' 1.1.1.1', ' 147', ' 3422', ' 3930', ' 767', ' 508', ' 30024', ' 3422', ' 3930', ' 767', ' 508', ' 302']
2024-08-14 17:50:35,321 - DATA too short 3, ['DATA', ' 2024-08-14-17:50:35', ' 17236']
2024-08-14 17:50:36,338 - DATA too short 6, ['DATA', ' 2024-08-14-17:50:36', ' 1723650636.272769', ' 1723650636.271568', ' 25', ' 31']
2024-08-14 17:50:38,371 - DATA too short 31, ['DATA', ' 2024-08-14-17:50:38', ' 1723650638.296131', ' 1723650638.295149', ' 179', ' 75', ' 0', ' 0', ' 1723650638.29178', ' 156.154.71.5', ' 167', ' 12465', ' 13600', ' 934', ' 1135', ' 30024', ' 12465', ' 13600', ' 934', ' 1135', ' 30205', ' 0', ' 1096', ' 10024', ' 60024', ' 0', ' 1096', ' 10205', ' 60205', ' dl_idle', '']
2024-08-14 17:50:39,388 - DATA too short 2, ['DATA', ' 2024-08-14-17:50:39']
2024-08-14 17:50:41,422 - DATA too short 10, ['DATA', ' 2024-08-14-17:50:41', ' 1723650641.383791', ' 1723650641.382816', ' 22', ' 20', ' 0', ' 0', ' 1723650641.37959', ' 9.9.9.1']
2024-08-14 17:50:46,505 - DATA too short 4, ['DATA', ' 2024-08-14-17:50:46', ' 1723650646.490800', ' 1723650646.4897']
2024-08-14 17:50:47,522 - DATA too short 14, ['DATA', ' 2024-08-14-17:50:47', ' 1723650647.493911', ' 1723650647.492932', ' 14', ' 19', ' 0', ' 0', ' 1723650647.48969', ' 156.154.70.5', ' 198', ' 3560', ' 4590', '']
2024-08-14 17:50:48,538 - DATA too short 28, ['DATA', ' 2024-08-14-17:50:48', ' 1723650648.514758', ' 1723650648.513783', ' 28', ' 30', ' 0', ' 0', ' 1723650648.51053', ' 156.154.71.5', ' 201', ' 12478', ' 14200', ' 883', ' 1722', ' 30024', ' 12478', ' 14200', ' 883', ' 1722', ' 30205', ' 0', ' 1033', ' 10024', ' 60024', ' 0', ' 1033', ' ']
2024-08-14 17:50:49,555 - DATA too short 30, ['DATA', ' 2024-08-14-17:50:49', ' 1723650649.495394', ' 1723650649.494419', ' 18', ' 19', ' 0', ' 0', ' 1723650649.49125', ' 9.9.9.11', ' 204', ' 3353', ' 3760', ' 1036', ' 407', ' 30024', ' 3353', ' 3760', ' 1036', ' 407', ' 30205', ' 0', ' 860', ' 10024', ' 60024', ' 0', ' 860', ' 10205', ' 60205', ' d']
2024-08-14 17:50:50,571 - DATA too short 4, ['DATA', ' 2024-08-14-17:50:50', ' 1723650650.546728', ' 17']
2024-08-14 17:50:51,588 - DATA too short 9, ['DATA', ' 2024-08-14-17:50:51', ' 1723650651.568034', ' 1723650651.567038', ' 21', ' 19', ' 0', ' 0', ' 1723650651.563']
2024-08-14 17:50:52,605 - DATA too short 14, ['DATA', ' 2024-08-14-17:50:52', ' 1723650652.571505', ' 1723650652.570487', ' 28', ' 58', ' 0', ' 0', ' 1723650652.56716', ' 156.154.71.3', ' 214', ' 12428', ' 14200', '']
2024-08-14 17:50:53,622 - DATA too short 18, ['DATA', ' 2024-08-14-17:50:53', ' 1723650653.552425', ' 1723650653.551389', ' 28', ' 32', ' 0', ' 0', ' 1723650653.54786', ' 1.1.1.1', ' 218', ' 3451', ' 3760', ' 649', ' 309', ' 30024', ' 3451', '']
2024-08-14 17:50:55,655 - DATA too short 29, ['DATA', ' 2024-08-14-17:50:55', ' 1723650655.573215', ' 1723650655.572224', ' 25558', ' 256', ' 5', ' 0', ' 1723650655.56881', ' 156.154.71.3', ' 224', ' 12434', ' 12600', ' 1079', ' 166', ' 30024', ' 12434', ' 12600', ' 1079', ' 166', ' 30205', ' 0', ' 698', ' 10024', ' 60024', ' 0', ' 698', ' 10205', ' 60']

And Grafana looks like this: image

image

image

image

image

image

image

Now I will fight with transporting data from OpenWRT router to RPi3 via Telegraf and traansfer data to InfluxDB database in cloud with grafana :P

lynxthecat commented 2 months ago

Looks freaking amazing. Are the CPU stats also available to verify no core is ever saturated?

MisiekDP commented 2 months ago

@lynxthecat I am not sure if @bairhys added CPU data to prometeus but on /metrics I see only this for cpu: image

MisiekDP commented 2 months ago

@bairhys Do you perhaps have a problem with some of the ‘truncated’ values for ‘Cake limit’ (blue line) in the file what you ceated yesterday? After entering the Inspect section, I see that some of the values are ‘undercut’. Look it here (Grafana snapshot): https://snapshots.raintank.io/dashboard/snapshot/PAeDAcJoOpmknDHNUzRYnGRFTLRTHFLu?orgId=0&inspect=4

I mean this:

image image image image image image

image image

MisiekDP commented 2 months ago

Or maybe there should be a CPU rule too? image

bairhys commented 2 months ago

Can it fill in the gap since it was last run by grabbing and plotting the intervening historical data? I’ve been thinking about running the log data through gzip albeit getting the implementation of that right could be tricky.

Not as far as I know. Prometheus exporter data is considered the instantaneous or current live values, there are no past values. So when Prometheus is off, the data this exporter prepares is lost. There might be a way to backfill data but I have not investigated this.

bairhys commented 2 months ago

Okey @lynxthecat @bairhys I did a small tests on Windows and it looks really nice. Now I see that I can try to start configure cake-autorate :)

Btw. when I manually run prometheus script I got logs with "Data too short" like this:

Yeah I get the same, this is what I am not happy with, this data too short error. For some reason, it doesn't read all the data so results in the zero measurements sometimes 🤔

lynxthecat commented 2 months ago

Are the CPU stats still to be read in?

bairhys commented 2 months ago

Yep CPU stats are todo

MisiekDP commented 2 months ago

Okey @lynxthecat @bairhys I did a small tests on Windows and it looks really nice. Now I see that I can try to start configure cake-autorate :) Btw. when I manually run prometheus script I got logs with "Data too short" like this:

Yeah I get the same, this is what I am not happy with, this data too short error. For some reason, it doesn't read all the data so results in the zero measurements sometimes 🤔

But why isn't all the data being read? Do you know, perhaps? And what does it depend on? Is it from the lack of reading of some of this data that the effect is some of the truncated values for ‘Cake limit’ (blue line) that I showed a few posts above? If this is the case then perhaps the data entry should be discarded altogether so that no cull values get into Grafana, or perhaps re-read all the data again when it is detected that the data is too short. Also, the error with too short data reads does not occur that often again, you could ignore this error a few times (perhaps until a better solution is found) 😉

MisiekDP commented 2 months ago

@bairhys @lynxthecat that is okey? Are the reflectors too much by any chance???? I have them set to the default value.

image

image

bairhys commented 2 months ago

Okey @lynxthecat @bairhys I did a small tests on Windows and it looks really nice. Now I see that I can try to start configure cake-autorate :) Btw. when I manually run prometheus script I got logs with "Data too short" like this:

Yeah I get the same, this is what I am not happy with, this data too short error. For some reason, it doesn't read all the data so results in the zero measurements sometimes 🤔

But why isn't all the data being read? Do you know, perhaps? And what does it depend on? Is it from the lack of reading of some of this data that the effect is some of the truncated values for ‘Cake limit’ (blue line) that I showed a few posts above? If this is the case then perhaps the data entry should be discarded altogether so that no cull values get into Grafana, or perhaps re-read all the data again when it is detected that the data is too short. Also, the error with too short data reads does not occur that often again, you could ignore this error a few times (perhaps until a better solution is found) 😉

Not sure why all the data is not being read as it used to read all the data fine previously. I haven't had time to work on it since my last commit but when we get some time, will work something out and add the other new interesting stats 👍

bairhys commented 2 months ago

@bairhys @lynxthecat that is okey? Are the reflectors too much by any chance???? I have them set to the default value.

From memory, the reflectors are rotated to reduce load on any single reflector?

lynxthecat commented 2 months ago

From memory, the reflectors are rotated to reduce load on any single reflector?

There is a pretty sophisticated reflector rotation algorithm that aims to converge onto a good set of reflectors within a list of reflectors no mater what starting set is given. There are heuristics for weeding out bad or temporarily faulty reflectors.

MisiekDP commented 2 months ago

@bairhys @lynxthecat that is okey? Are the reflectors too much by any chance???? I have them set to the default value.

From memory, the reflectors are rotated to reduce load on any single reflector?

@lynxthecat @bairhys You mean this option? image

Btw how to make it not too much pingers? I mean the good number for me is just like 6. Should I delete more that 6 adresses or disable any option in settings?