search

bb-Ricardo / fritzinfluxdb

Writes data from fritzbox to influxdb
MIT License
144 stars 37 forks source link

No CPU, RAM, Temps and Hosts #129

Closed Bond246 closed 1 month ago

Bond246 commented 1 month ago

Hello,

i'm using fritz2influx now for a while. Migrating the container to different hosts, changing the complete hardware/docker-host and so on, never a problem.

But since some days i have no data for:

Before i upgraded to FritzOS 8.0 on my Fritzbox 7590 AX. But after that upgrade it worked for around 10 hours. But i'm thinkig about if a docker-version upgrade to 27.3.1 could be the problem.

The fritz2influx-Log looks ok to me

INFO: Starting main loop
INFO: Successfully connected to FritzBox '192.168.xxxx' (fritz.box) Model: FRITZ!Box 7590 AX (DSL) - FW: 8.0
INFO: Starting fritzinfluxdb v1.2.2 (2024-07-13)
INFO: Done reading config files
INFO: Successfully parsed config
INFO: Connection to InfluxDB v2.7.8 established and bucket is present
INFO: Successfully established FritzBox TR-069 session
INFO: Disabling queries via Lua. Fritz!OS version must be at least 7.XX

I'm using fritz2influx in a bridged docker-network. I've defined a custome bridge where also InfluxDBv2 is hosted. Host is a ubuntu 24.04 on proxmox.

Any ideas why this metrics are not present anymore and how to fix it?

Thanks

ronaldn1969 commented 1 month ago

The reason is, that your OS is 8.xx and not 7.xx

INFO: Disabling queries via Lua. Fritz!OS version must be at least 7.XX

Tom already made a modification... take a look her: https://github.com/bb-Ricardo/fritzinfluxdb/pull/128

Bond246 commented 1 month ago

but it worked for about 10 hours after updating...? ahhhhhhhhhh because of restarting the container it made a new version-test but with the running container it didn't noticed the change. i will try this later...

Thanks

Bond246 commented 1 month ago

works Thanks again 👍

bb-Ricardo commented 1 month ago

Sorry, still being busy the last few days. Hope to add support asap.

bb-Ricardo commented 1 month ago

fixed in version https://github.com/bb-Ricardo/fritzinfluxdb/releases/tag/v1.2.3

xyrolaith commented 1 month ago

Sorry for reopening this issue but after upgrading to FritzOS 8.00 and v1.2.3 my energy readouts disappeared as well.

After some investigation I found a knowledge base article (#3808) on the AVM web site. They completely removed the energy monitor for Cable boxes in FritzOS 8.00 and will reimplement it in an upcoming firmware release.

bb-Ricardo commented 1 month ago

Oh wow, but not sure what I can do about it, if the FritzBox does not provide the values. Any ideas?

xyrolaith commented 1 month ago

Using os_max_versions does not look feasible as only Cable boxes are affected and the function should be back again in the future.

Maybe by implementing some kind of blacklist?

    {
        "os_min_versions": "7.29",
        "method": "POST",
        "name": "System Stats",
        "blacklisted_versions": { "FritzBoxLinkTypes.Cable": "8.00" },
        [...]
    }
bb-Ricardo commented 1 month ago

So what is the current status? An error message appears on every start of the tool or on every request interval?

xyrolaith commented 1 month ago

Unfortunately on every interval.

Oct 07 21:38:19 fritzinfluxdb[1238]: ERROR: Unable to extract 'cpu_temp' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': True, 'li...
Oct 07 21:38:19 fritzinfluxdb[1238]: ERROR: Unable to extract 'cpu_utilization' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tr...
Oct 07 21:38:19 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_fixed' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tr...
Oct 07 21:38:19 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_dynamic' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': ...
Oct 07 21:38:19 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_free' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tru...
Oct 07 21:38:21 fritzinfluxdb[1238]: ERROR: Unable to extract 'energy_consumption' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd':...
Oct 07 21:40:52 fritzinfluxdb[1238]: ERROR: Unable to extract 'cpu_temp' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': True, 'li...
Oct 07 21:40:52 fritzinfluxdb[1238]: ERROR: Unable to extract 'cpu_utilization' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tr...
Oct 07 21:40:52 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_fixed' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tr...
Oct 07 21:40:52 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_dynamic' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': ...
Oct 07 21:40:52 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_free' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tru...
Oct 07 21:40:55 fritzinfluxdb[1238]: ERROR: Unable to extract 'energy_consumption' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd':...
Oct 07 21:43:26 fritzinfluxdb[1238]: ERROR: Unable to extract 'cpu_temp' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': True, 'li...
Oct 07 21:43:26 fritzinfluxdb[1238]: ERROR: Unable to extract 'cpu_utilization' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tr...
Oct 07 21:43:26 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_fixed' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tr...
Oct 07 21:43:26 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_dynamic' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': ...
Oct 07 21:43:26 fritzinfluxdb[1238]: ERROR: Unable to extract 'ram_usage_free' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd': Tru...
Oct 07 21:43:28 fritzinfluxdb[1238]: ERROR: Unable to extract 'energy_consumption' form '{'pid': 'overview', 'hide': {'shareUsb': True, 'tvhd':...
bb-Ricardo commented 1 month ago

Ah, damn it. Can you please post a json out for this one as well? Seems like a little f**k up by AVM.

xyrolaith commented 1 month ago

As I said, they completely removed the functionality in all FritzOS 8.00 Cable boxes. Even the mail digest does not include them.

Anyway... This is the current JSON response. I had to remove all personal data first.

send: b'POST /data.lua HTTP/1.1\r\nHost: fritz.box\r\nUser-Agent: python-requests/2.32.3\r\nAccept-Encoding: gzip, deflate\r\nAccept: /\r\nConnection: keep-alive\r\nContent-Length: 41\r\nContent-Type: application/x-www-form-urlencoded\r\n\r\n' send: b'sid=&page=ecoStat&lang=de' reply: 'HTTP/1.1 200 OK\r\n' header: Cache-Control: no-cache header: Connection: close header: Content-Type: application/json; charset=utf-8 header: Date: Mon, 07 Oct 2024 20:23:22 GMT header: Expires: -1 header: Pragma: no-cache header: X-Frame-Options: SAMEORIGIN header: X-Content-Type-Options: nosniff header: Content-Security-Policy: default-src 'none'; connect-src 'self'; font-src 'self'; frame-src https://service.avm.de https://fritzhelp.avm.de/help/ https://www.avm.de https://avm.de https://assets.avm.de https://clickonce.avm.de http://clickonce.avm.de http://download.avm.de https://download.avm.de 'self'; img-src 'self' https://tv.avm.de https://help.avm.de/images/ http://help.avm.de/images/ data:; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'; frame-ancestors 'self'; media-src 'self' header: Referrer-Policy: no-referrer, same-origin 2024-10-07 22:23:22,095 - DEBUG: FritzBox Lua request successful 2024-10-07 22:23:22,095 - DEBUG: Request FritzBox Lua service 'System Stats (Fritz!OS 7.29 - latest)' returned successfully 2024-10-07 22:23:22,096 - ERROR: Unable to extract 'cpu_temp' form '{}', got '<class 'NoneType'>'

{
    'pid': 'overview',
    'hide': {
        'shareUsb': True,
        'tvhd': True,
        'liveTv': True,
        'dvbradio': True,
        'dectRdio': True,
        'tvsd': True,
        'rrd': True,
        'rss': True,
        'mobile': True,
        'dvbSet': True,
        'ssoSet': True,
        'dectMail': True,
        'tfa': True,
        'dvbSig': True,
        'liveImg': True
    },
    'timeTillLogout': '1200',
    'time': [],
    'data': {
        'fritzos': {
            'isLabor': False,
            'NoPwd': False,
            'connectionStatus': True,
            'nspver': '8.00',
            'twofactor_disabled': False,
            'fb_name': '',
            'showUpdate': True,
            'Productname': 'FRITZ!Box 6591 Cable',
            'ShowDefaults': False,
            'boxDate': '22:23:19 07.10.2024',
            'noUpdateSearch': False,
            'hasEnergy': False,
            'noAutoupdate': True,
            'FirmwareSigned': True,
            'isUpdateAvail': False,
            'energy': '0',
            'iframeUrl': 'https://fritzhelp.avm.de/help/?hardware=233&oem=avm&language=de&country=049&version=161.08.00&subversion=&set=024p1&action=feature'
        },
        'docsis': {
            'txt': 'verbunden',
            'led': 'led green',
            'title': 'Kabel',
            'up': 52480,
            'down': 1126400,
            'link': 'docOv'
        },
        'wlan_guest': {
            'txt': 'an, Funknetz (2,4/5 GHz): Gastnetz',
            'led': 'led green',
            'title': 'WLAN',
            'link': 'wGuest'
        },
        'lan': {
            'txt': 'verbunden (LAN 1, LAN 2)',
            'led': 'led green',
            'title': 'LAN',
            'link': 'netDev'
        },
        'foncalls': {
            'callsToday': '0',
            'calls': [{
                    'number': '12345',
                    'type': 'fail',
                    'name': '',
                    'display': '12345',
                    'date': '1. Mai',
                    'time': '10:00',
                    'addible': True
                }
            ],
            'activecalls': []
        },
        'comfort': {
            'func': [{
                    'linktxt': 'Anrufbeantworter',
                    'details': 'a
                    ktiv',
                    'link': 'tam'
                }, {
                    'linktxt': 'Telefonbuch',
                    'details': '1 Telefonbuch vorhanden',
                    'link': 'bookLi'
                }, {
                    'linktxt': 'Smart Home',
                    'details': '5 verbunden ',
                    'link': 'sh_dev'
                }, {
                    'linktxt': 'WLAN-Gastzugang',
                    'details': 'aktiv (2,4/5 GHz), gesichert, 0 Geräte, Funknetz: Gastnetz',
                    'link': 'wGuest'
                }, {
                    'linktxt': 'Anrufe sperren',
                    'details': 'aktiv',
                    'link': 'callLock'
                }, {
                    'linktxt': 'Portfreigabe',
                    'details': 'aktiv, 6 Portfreigaben eingerichtet',
                    '
                    link': 'portoverview'
                }, {
                    'linktxt': 'Speicher (NAS)',
                    'details': 'deaktiviert',
                    'link': 'usbOv'
                }, {
                    'linktxt': 'Kindersicherung',
                    'details': 'bereit',
                    'link': 'kidLis'
                }, {
                    'linktxt': 'DynDNS',
                    'details': 'aktiviert, host.example.com, IPv4-Status: erfolgreich angemeldet, IPv6-Status: erfolgreich angemeldet',
                    'link': 'dyndns'
                }
            ],
            'anyComfort': True
        },
        'net': {
            'anyUnmeshedDevices': False,
            'count': 16,
            'active_count': 16,
            'devices': [{
                    'own_client_device': False,
                    'type': 'lan',
                    'name': 'FRITZ!Repeater, Repeater',
                    'stateinfo': {
                        'nexustrust': True,
                        'active': True
                    },
                    'url': 'http://1.2.3.4'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'ED',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'Device #1',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'Device #2',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '2,4 GHz'
                }, {
                    'own_client_device': False,
                    'type':
                    'wlan',
                    'name': 'Device #3',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'Device #4',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'Device #5',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'Device #6',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': False,
                    'type': 'wlan',
                    'name': 'Device #6',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': '',
                    'desc': '5 GHz'
                }, {
                    'own_client_device': True,
                    'type': 'lan',
                    'name': 'server',
                    'stateinfo': {
                        'online': True,
                        'active': True
                    },
                    'url': ''
                }, {
                    'type': 'handset',
                    'name': 'Phone',
                    'stateinfo': {
                        'active': True
                    },
                    'url': ''
                }, {
                    'type': 'smarthome',
                    'name': 'FRITZ!DECT 200, #1',
                    'stateinfo': {
                        'active': True
                    },
                    'url': ''
                }, {
                    'type': 'smarthome',
                    'name': 'FRITZ!DECT 200, #2',
                    'stateinfo': {
                        'active': True
                    },
                    'url': ''
                }, {
                    'type': 'smarthome',
                    'name': 'FRITZ!DECT 200, #3',
                    'stateinfo': {
                        'active': True
                    },
                    'url': ''
                }, {
                    'type': 'smarthome',
                    'name': 'FRITZ!DECT 200, #4',
                    'stateinfo': {
                        'active': True
                    },
                    'url': ''
                }, {
                    'type': 'smarthome',
                    'name': 'FRITZ!DECT 200 #4',
                    'stateinfo': {
                        'active': True
                    },
                    'url': ''
                }
            ],
            'otherAVMUpdates': 0
        },
        'vpn': {
            'title': 'Fernzugang',
            'elements': []
        },
        'fonnum': {
            'txt': '2 Rufnummern aktiv: 0123456789, 0987654321',
            'led': 'led green',
            'title': 'Telefonie',
            'link': 'numLi'
        },
        'dect': {
            'txt': 'an, ein Schnurlostelefon angemeldet',
            'led': 'led green',
            'title': 'DECT',
            'link': 'dectSet'
        },
        'usb': {
            'count': 0
        },
        'internet': {
            'box_type': 'cable',
            'connections': [{
                    'active': True,
                    'medium_upstream': 52480,
                    'downstream': 1126400,
                    'role': 'single',
                    'provider': '',
                    'ipv4': {
                        'connected': True,
                        'dns': [{
                                'edns0_mode': 'none',
                                'type': 'normal',
                                'ip': '8.8.8.8'
                            }, {
                                'edns0_mode': 'none',
                                'type': 'normal',
                                'ip': '9.9.9.9'
                            }
                        ],
                        'dslite': False,
                        'ip': '1.1.1.1',
                        'since': 1727899075
                    },
                    'ipv6': {
                        'ip_lifetime': {
                            'valid': 84863,
                            'preferred': 84863
                        },
                        'connected': True,
                        'dns': [{
                                'edns0_mode': 'none',
                                'type': 'normal',
                                'purpose': 'standard',
                                'ip': '2000:123:4:b::1'
                            }, {
                                'edns0_mode': 'none',
                                'type': 'normal',
                                'ip': '2000:123:4:a::1'
                            }
                        ],
                        'ip': '2000:123:ab00:5::20db/64',
                        'prefix': '2000:123:ab54:6700::/59',
                        'prefix_lifetime': {
                            'valid': 84863,
                            'preferred': 84863
                        },
                        'since': 1727899075
                    },
                    'shapedrate': False,
                    'medium_downstream': 1126400,
                    'state': 'connected',
                    'upstream': 52480,
                    'name': 'internet',
                    'type': 'cable',
                    'provider_id': 'other',
                    'ready_for_fallback': False,
                    'connected': True,
                    'medium': 'cable'
                }
            ]
        },
        'wlan': [{
                'txt': 'an, Funknetz 2,4 GHz: WLAN',
                'led': 'led green',
                'title': 'WLAN',
                'link': 'wSet'
            }, {
                'txt': 'an, Funknetz 5 GHz: WLAN',
                'led': 'led green',
                'title': 'WLAN',
                'link': 'wSet'
            }
        ]
    },
    'sid': 'abcdef12345678'
}
bb-Ricardo commented 1 month ago

great, thank you for the quick response, I'll have a look.

bb-Ricardo commented 1 month ago

Hey @xyrolaith,

I just pushed a new commit to the next-release branch. can you please have a look?

thank you

xyrolaith commented 1 month ago

@bb-Ricardo Looks good. I am not seeing any errors in the log.