search

bb-Ricardo / fritzinfluxdb

Writes data from fritzbox to influxdb
MIT License
140 stars 35 forks source link

Calllist since FRITZ!OS v7.90 (Beta) not correct [FRITZ!Box 7590] #122

Closed falke9176 closed 1 day ago

falke9176 commented 1 month ago

Hi Ricardo,

I have the latest Beta on my F!B7590 and since that point, die Calllist Information may be different ?! grafik

Do you have the same Problem? I can show you the debug output if you want to.....

Grüße Sven

bb-Ricardo commented 1 month ago

Hi @falke9176,

Indeed that seems different. Will need to check out the beta.

If it's possible: try to open the page with the calls in a browser and see if you can get the json response from the dev tools. This would really help if you don't mind.

falke9176 commented 1 month ago

I will, but can you short explain how?

bb-Ricardo commented 1 month ago

Open the dev tools and switch to network. Navigate to the call log in the FritzBox UI. Then check for the API call (usually xhr) and get the response.

falke9176 commented 1 month ago

mmmh.... I tried.... But I can only see a "data.lua" file.... ?! I cleared the DevTool Window and clicked on "Anruferliste aktuallisieren" grafik

falke9176 commented 1 month ago

the debug output by fritzinfluxdb.py for a incoming testcall

2024-06-12 12:09:46,736 - DEBUG: Request FritzBox Lua service 'Phone call list (Fritz!OS 7.29 - latest)' returned successfully
2024-06-12 12:09:47,423 - DEBUG: FritzBox Lua request successful
2024-06-12 12:09:47,423 - DEBUG: Request FritzBox Lua service 'VPN Users - IPSec (Fritz!OS 7.39 - latest)' returned successfully
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 10:09:46+00:00: call_list_type=undefined ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 12:09:00+02:00: call_list_type=incoming ({'box': 'hausanschluss', 'uid': '20b79c0fbbe21172df1f9ed973288afb'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 10:09:46+00:00: call_list_caller_name=Name ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 12:09:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': '20b79c0fbbe21172df1f9ed973288afb'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 10:09:46+00:00: call_list_caller_number=Rufnummer ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 12:09:00+02:00: call_list_caller_number= ({'box': 'hausanschluss', 'uid': '20b79c0fbbe21172df1f9ed973288afb'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 10:09:46+00:00: call_list_extension=Landes-/Ortsnetzbereich ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 12:09:00+02:00: call_list_extension=0177XXXXXX6 ({'box': 'hausanschluss', 'uid': '20b79c0fbbe21172df1f9ed973288afb'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 10:09:46+00:00: call_list_number_called=Nebenstelle ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:09:47,423 - DEBUG: 2024-06-12 12:09:00+02:00: call_list_number_called=Spitzboden Sven ({'box': 'hausanschluss', 'uid': '20b79c0fbbe21172df1f9ed973288afb'})
2024-06-12 12:09:47,424 - DEBUG: 2024-06-12 10:09:46+00:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:09:47,424 - DEBUG: 2024-06-12 12:09:00+02:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '20b79c0fbbe21172df1f9ed973288afb'})
2024-06-12 12:09:47,424 - DEBUG: 2024-06-12 10:09:47+00:00: myfritz_host_name=6kwbifyb24gkrsqw.myfritz.net ({'box': 'hausanschluss'})
2024-06-12 12:09:47,424 - DEBUG: 2024-06-12 10:09:47+00:00: vpn_type=IPSec Xauth PSK ({'box': 'hausanschluss'})
2024-06-12 12:09:47,424 - DEBUG: Trying to write a maximum of '1000' measurements to InfluxDB
2024-06-12 12:09:47,450 - DEBUG: Successfully wrote 14 measurements to InfluxDB
2024-06-12 12:09:47,450 - DEBUG: Current InfluxDB measurement buffer length: 0
2024-06-12 12:09:49,030 - DEBUG: FritzBox Lua request successful
2024-06-12 12:09:49,031 - DEBUG: Request FritzBox Lua service 'VPN Users - WireGuard (Fritz!OS 7.39 - latest)' returned successfully
falke9176 commented 1 month ago

each time of read out the calllist, the first line will be write into the variables - and than to influxdb. But the uid is still the same.

2024-06-12 12:21:33,579 - DEBUG: 2024-06-12 10:21:32+00:00: call_list_type=undefined ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:21:33,589 - DEBUG: 2024-06-12 10:21:32+00:00: call_list_caller_name=Name ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:21:33,599 - DEBUG: 2024-06-12 10:21:32+00:00: call_list_caller_number=Rufnummer ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:21:33,608 - DEBUG: 2024-06-12 10:21:32+00:00: call_list_extension=Landes-/Ortsnetzbereich ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:21:33,617 - DEBUG: 2024-06-12 10:21:32+00:00: call_list_number_called=Nebenstelle ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:21:33,626 - DEBUG: 2024-06-12 10:21:32+00:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
falke9176 commented 1 month ago

Ah, now I see the difference. A column has been added to the call list: "Landes-/Ortsnetzbereich". This did not exist FRITZ!OS <7.90. Now the reading for FRITZ!OS >7.90 and also the Grafana page would have to be adjusted.

2024-06-12 12:43:46,623 - DEBUG: 2024-06-12 10:43:46+00:00: call_list_type=undefined ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:43:46,623 - DEBUG: 2024-06-12 10:43:46+00:00: call_list_caller_name=Name ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:43:46,623 - DEBUG: 2024-06-12 10:43:46+00:00: call_list_caller_number=Rufnummer ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:43:46,623 - DEBUG: 2024-06-12 10:43:46+00:00: call_list_extension=Landes-/Ortsnetzbereich ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:43:46,623 - DEBUG: 2024-06-12 10:43:46+00:00: call_list_number_called=Nebenstelle ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:43:46,623 - DEBUG: 2024-06-12 10:43:46+00:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})

2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 10:44:47+00:00: call_list_type=undefined ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_type=outgoing ({'box': 'hausanschluss', 'uid': '9b50b0ec347e3ae8e6eadbfe55463e1d'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_type=incoming ({'box': 'hausanschluss', 'uid': '866704490fd3c6a4cb77086a28ef7c82'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 10:44:47+00:00: call_list_caller_name=Name ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_caller_name= ({'box': 'hausanschluss', 'uid': '9b50b0ec347e3ae8e6eadbfe55463e1d'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': '866704490fd3c6a4cb77086a28ef7c82'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 10:44:47+00:00: call_list_caller_number=Rufnummer ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_caller_number=Berlin ({'box': 'hausanschluss', 'uid': '9b50b0ec347e3ae8e6eadbfe55463e1d'})
2024-06-12 12:44:48,018 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_caller_number=Mahlow ({'box': 'hausanschluss', 'uid': '866704490fd3c6a4cb77086a28ef7c82'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 10:44:47+00:00: call_list_extension=Landes-/Ortsnetzbereich ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_extension=030xxxxxx33 ({'box': 'hausanschluss', 'uid': '9b50b0ec347e3ae8e6eadbfe55463e1d'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_extension=03379xxxxxx ({'box': 'hausanschluss', 'uid': '866704490fd3c6a4cb77086a28ef7c82'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 10:44:47+00:00: call_list_number_called=Nebenstelle ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_number_called=Spitzboden Sven ({'box': 'hausanschluss', 'uid': '9b50b0ec347e3ae8e6eadbfe55463e1d'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_number_called=FRITZ!Box-Schuppen ({'box': 'hausanschluss', 'uid': '866704490fd3c6a4cb77086a28ef7c82'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 10:44:48+00:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '9b50b0ec347e3ae8e6eadbfe55463e1d'})
2024-06-12 12:44:48,019 - DEBUG: 2024-06-12 12:44:00+02:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '866704490fd3c6a4cb77086a28ef7c82'})
2024-06-12 12:45:48,694 - DEBUG: 2024-06-12 10:45:48+00:00: call_list_type=undefined ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:45:48,694 - DEBUG: 2024-06-12 12:45:00+02:00: call_list_type=outgoing ({'box': 'hausanschluss', 'uid': '664dbdcfceb7bafddb085571a4fd21d0'})
2024-06-12 12:45:48,694 - DEBUG: 2024-06-12 10:45:48+00:00: call_list_caller_name=Name ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 12:45:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': '664dbdcfceb7bafddb085571a4fd21d0'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 10:45:48+00:00: call_list_caller_number=Rufnummer ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 12:45:00+02:00: call_list_caller_number=Berlin ({'box': 'hausanschluss', 'uid': '664dbdcfceb7bafddb085571a4fd21d0'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 10:45:48+00:00: call_list_extension=Landes-/Ortsnetzbereich ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 12:45:00+02:00: call_list_extension=030xxxxxx333 ({'box': 'hausanschluss', 'uid': '664dbdcfceb7bafddb085571a4fd21d0'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 10:45:48+00:00: call_list_number_called=Nebenstelle ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 12:45:00+02:00: call_list_number_called=Spitzboden Sven ({'box': 'hausanschluss', 'uid': '664dbdcfceb7bafddb085571a4fd21d0'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 10:45:48+00:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '35fbc0d3192345af308ba53a30f6e412'})
2024-06-12 12:45:48,695 - DEBUG: 2024-06-12 12:45:00+02:00: call_list_duration=0 ({'box': 'hausanschluss', 'uid': '664dbdcfceb7bafddb085571a4fd21d0'})
bb-Ricardo commented 1 month ago

The content of the data.lua is exactly what I would need

falke9176 commented 1 month ago

I wrote an email. I hope you got it.

bb-Ricardo commented 1 month ago

I checked, but I didn't get an email

falke9176 commented 1 month ago

I wrote to ricardo@bitchbrothers.com

bb-Ricardo commented 1 month ago

That is the correct address. Could you send it again please?

falke9176 commented 1 month ago

got it now?

bb-Ricardo commented 1 month ago

Jup, I got it. Also found the mail from this afternoon. Mails with frotzbox in it get sorted automatically in a different folder and I forgot about that. Sorry.

bb-Ricardo commented 1 month ago

Hey @falke9176,

I just pushed a new commit to next-release branch. would you be able to test it?

bb-Ricardo commented 1 month ago

you can also use the https://hub.docker.com/r/bbricardo/fritzinfluxdb/tags docker image next-release

falke9176 commented 1 month ago

I use a LXC Debian ....no docker ;-)

But I updated my installation and it would be better, but a bit is still wrong....

Instead of the "caller_number", the value of the city is put in. Don't you put the city data into any variable?! f.e. call_list_caller_location

2024-06-13 07:18:07,302 - DEBUG: Request FritzBox Lua service 'Phone call list (Fritz!OS 7.29 - latest)' returned successfully
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_type=incoming ({'box': 'hausanschluss', 'uid': 'f56a21cab91fe408f1f8562bcd6d2769'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_type=outgoing ({'box': 'hausanschluss', 'uid': 'c6d8664ea042a5d9e9b73e8d4423a7bc'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_type=incoming ({'box': 'hausanschluss', 'uid': '14abc517a759b61d9dd2cd61a3abaf69'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_type=incoming ({'box': 'hausanschluss', 'uid': 'e4c355e65c1baba84c9fa6c20c1ff0bf'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': 'f56a21cab91fe408f1f8562bcd6d2769'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': 'c6d8664ea042a5d9e9b73e8d4423a7bc'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': '14abc517a759b61d9dd2cd61a3abaf69'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_name=Sven Neuendorf ({'box': 'hausanschluss', 'uid': 'e4c355e65c1baba84c9fa6c20c1ff0bf'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_number= ({'box': 'hausanschluss', 'uid': 'f56a21cab91fe408f1f8562bcd6d2769'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_number=Berlin ({'box': 'hausanschluss', 'uid': 'c6d8664ea042a5d9e9b73e8d4423a7bc'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_number=Mahlow ({'box': 'hausanschluss', 'uid': '14abc517a759b61d9dd2cd61a3abaf69'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_caller_number= ({'box': 'hausanschluss', 'uid': 'e4c355e65c1baba84c9fa6c20c1ff0bf'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_extension=FRITZ!Box-Schuppen ({'box': 'hausanschluss', 'uid': 'f56a21cab91fe408f1f8562bcd6d2769'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_extension=FRITZ!Fon C6 [613] ({'box': 'hausanschluss', 'uid': 'c6d8664ea042a5d9e9b73e8d4423a7bc'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_extension=FRITZ!Box-Schuppen ({'box': 'hausanschluss', 'uid': '14abc517a759b61d9dd2cd61a3abaf69'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_extension=FRITZ!Box-Schuppen ({'box': 'hausanschluss', 'uid': 'e4c355e65c1baba84c9fa6c20c1ff0bf'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_number_called=Internet: 03379xxxxxx ({'box': 'hausanschluss', 'uid': 'f56a21cab91fe408f1f8562bcd6d2769'})
2024-06-13 07:18:08,526 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_number_called=Internet: 3xxxxxx ({'box': 'hausanschluss', 'uid': 'c6d8664ea042a5d9e9b73e8d4423a7bc'})
2024-06-13 07:18:08,527 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_number_called=Internet: 03379xxxxxx ({'box': 'hausanschluss', 'uid': '14abc517a759b61d9dd2cd61a3abaf69'})
2024-06-13 07:18:08,527 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_number_called=Internet: 03379xxxxxx ({'box': 'hausanschluss', 'uid': 'e4c355e65c1baba84c9fa6c20c1ff0bf'})
2024-06-13 07:18:08,527 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_duration=1 ({'box': 'hausanschluss', 'uid': 'f56a21cab91fe408f1f8562bcd6d2769'})
2024-06-13 07:18:08,527 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_duration=1 ({'box': 'hausanschluss', 'uid': 'c6d8664ea042a5d9e9b73e8d4423a7bc'})
2024-06-13 07:18:08,527 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_duration=1 ({'box': 'hausanschluss', 'uid': '14abc517a759b61d9dd2cd61a3abaf69'})
2024-06-13 07:18:08,527 - DEBUG: 2024-06-13 07:17:00+02:00: call_list_duration=1 ({'box': 'hausanschluss', 'uid': 'e4c355e65c1baba84c9fa6c20c1ff0bf'})

grafik

bb-Ricardo commented 1 month ago

Thank you for testing, then I need to fix this again

bb-Ricardo commented 4 weeks ago

Hi,

I made the call log parsing no more reliable. But turns out. The call log CSV returned from the FritzBox is wrong. The columns Rufnummer and Landes-/Ortsnetzbereich are switched in the Beta Firmware 7.90-113722.

So we need to wait until AVM fixed this issue and then it will work properly.

bb-Ricardo commented 4 weeks ago

Just sent a Feedback to their beta program. Let's see when this is fixed.

falke9176 commented 1 week ago

I'm now on FRITZ!OS 7.90-114109 and it seems to work again. My call log looks great again......

bb-Ricardo commented 1 week ago

Thank you for the feedback, will check if it works for me as well.

bb-Ricardo commented 1 day ago

Works for me as well