search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.56k stars 30.74k forks source link

AVM FRITZ!SmartHome no longer possible to login to FritzBox #83987

Closed HHoppenstock closed 1 year ago

HHoppenstock commented 1 year ago

The problem

Since the newest version of HomeAssistant I always get the error message "Invalid authentication" when I try to set up the "AVM FRITZ!SmartHome" integration. The other two FritzBox integrations "FRITZ!Box Tools" and "AVM FRITZ!Box Call Monitor" are working without any issue and can be setup with the same credentials "AVM FRITZ!SmartHome" won't accept. My FB is a 7490 with OS version 7.29.

What version of Home Assistant Core has the issue?

Home Assistant 2022.12.5

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

FRITZ!SmartHome

Link to integration documentation on our website

https://www.home-assistant.io/integrations/fritzbox

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @mib1185, @flabbamann, mind taking a look at this issue as it has been labeled with an integration (fritzbox) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `fritzbox` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign fritzbox` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)

fritzbox documentation fritzbox source (message by IssueLinks)

mib1185 commented 1 year ago

Hi @HHoppenstock

please enable debug logging, restart HA and provide the home-assistant.log. To do so add the following to your configuration.yaml:

logger:
  default: info
  logs:
    homeassistant.components.fritzbox: debug
    pyfritzhome: debug

Further please also check the logs if the Fritzbox itself, if there are authentication (or any other) issues logged.

Note: it is better to drag the log into the comment (which will add it as an attachment) and not copy paste as it is hard to read logs in GitHub.

HHoppenstock commented 1 year ago

home-assistant_2022-12-14T19-24-18.071Z.log configuration

I added the lines to my configuration.yaml and added the log. After changing the configuration.yaml and a restart of the docker container I tried to add the Smarthome integration.

No logs in my FritzBox

mib1185 commented 1 year ago

the log file seems to be truncated, there is nothing in it about your try to add the AVM!SmartHome integration 🤔

whiletruefranz commented 1 year ago

Just want to add that I am also not able to log in via the SmartHome Integration. The Tools Integration is working fine.

mib1185 commented 1 year ago

@whiletruefranz please provide debug logs as described in https://github.com/home-assistant/core/issues/83987#issuecomment-1351679013

whiletruefranz commented 1 year ago

@mib1185 Only pyfritzhome log is:

2022-12-19 13:30:04.685 WARNING (SyncWorker_9) [pyfritzhome.fritzhome] login failed 0000000000000000

I'm using the exact same credentials that work for the Fritz Box Tools Integration. Permissions to access Smart Home are set in fritzOS. EDIT: Event-Logs within fritzOS suggest a wrong password. I even changed it and copy-pasted it from a notepad. There is no way that the password is wrong.

flabbamann commented 1 year ago

Maybe a problem with some special character in the password?

whiletruefranz commented 1 year ago

@flabbamann I don't think so as it is alphanumeric.

mib1185 commented 1 year ago

which model and firmware is it? do have any kind of multi-factor-auth enabled?

whiletruefranz commented 1 year ago

@mib1185 Fritz!Box 7590 AX, 07.31 and no multi-factor-auth

mib1185 commented 1 year ago

please share the full message from FritzBox log (it's ok when in German) which occur during trying to setup the Fritz!Smarthome integration

whiletruefranz commented 1 year ago

@mib1185 I just tried again to get the error message and it worked first try. Same credentials as yesterday (still copy pasted from the notepad).

whiletruefranz commented 1 year ago

I don't actually know where to ask so I'll do it here: Is it possible to customize the implementation of the Fritz Dect 200/210 so that the consumption can be tracked as generation? I have the 210 attached to a micro inverter with a small PV setup on my garage and I'd like to use the data it provides correctly.

mib1185 commented 1 year ago

I would suggest to place this question on our forum, since we use GitHub for tracking issues with Home Assistant Core itself, not for providing support.

flabbamann commented 1 year ago

Hi, I think I found something.

Today in my HA dev environment the FRITZ!SmartHome integration could not log in to the FritzBox, while at the same time my productive HA instance can access and control my FB. As above no errors in HA or FB logs 🤨.

On login errors the original exception is lost. I opened a PR to change that in the library: https://github.com/hthiery/python-fritzhome/pull/73

With this change I found that HA gets an error 503 from the Fritzbox:

2022-12-21 00:09:36.345 ERROR (SyncWorker_0) [pyfritzhome.fritzhome] 503 Server Error: Service Unavailable for url: http://fritz.box/login_sid.lua

I don't really understand what's happening here. I have no problems logging in on the FB via browser or my productive HA instance. A bug in the Fritzbox? Hardware Limits? Solar storm? 🤔

I have a FB 7590 running latest FritzOS 7.50

HHoppenstock commented 1 year ago

I tried to add the FritzSmartHome integration with an older Docker Desktop version 4.12 with the latest stable homeassistant 12.7 and it worked without any issues. Using the latest Docker Desktop version 4.15 I run in the authentification error.

mib1185 commented 1 year ago

@flabbamann were there any entries in the Fritz!Box log itself at the time of the issue?

flabbamann commented 1 year ago

@mib1185 unfortunately not ☹️

mib1185 commented 1 year ago

maybe some new security feature in FritzOS 7.50 (somethin like "login count limit per source ip") 🤔 unfortunately, my 7530ax does not get the 7.50 yet, so i cannot test/check anything by my own 😕

MatthiasFraass commented 1 year ago

I just switched from a 7490 (where the Smarthome did work, but not stable - I had authentication and timeout problems all through the day) to a 7530AX (7.31) and login is not possible. I will try to activate debug logging.

But the problem must be the Smarthome Integration exclusively, as the "Tools" integration did work flawlessly with the 7530AX.

MatthiasFraass commented 1 year ago

Update: 1 days later I switched on debugging and... suddenly everything works (7530 AX)! Login was successfull immediately and it didn't lose authentiation since then (1 week ago). Strange. Maybe the fritzbox needs some more time after bootup to run all processes?

maddinhome commented 1 year ago

Since the newest version of HomeAssistant I always get the error message "Invalid authentication" when I try to set up the "AVM FRITZ!SmartHome" integration. The other two FritzBox integrations "FRITZ!Box Tools" and "AVM FRITZ!Box Call Monitor" are working without any issue and can be setup with the same credentials "AVM FRITZ!SmartHome" won't accept.

same issue here since ha: 2023.2.0 (OVA Synology VM) FB: 7590 FBOS: 7.50

HHoppenstock commented 1 year ago

It seems to be an issue with docker desktop for windows. I installed the latest version of HA on a raspberry and the Smarthome Integration works fine.

flabbamann commented 1 year ago

It seems to be an issue with docker desktop for windows. I installed the latest version of HA on a raspberry and the Smarthome Integration works fine.

Interesting. That matches my observations. I had problems in my dev environment (docker desktop on windows), while at the same time my productive instance (docker on Linux) was working fine.

maddinhome commented 1 year ago

same issue here since ha: 2023.2.0 (OVA Synology VM) FB: 7590 FBOS: 7.50

It now seems to work again, for a few days no more crashes in the connection to the FritzBox, the authentication is possible again without problems and is stable again. I can't understand why it's working again now. The logs of the Fritzbox are not helpful there either.

I apologize for the unhelpful messages from me.

MatthiasFraass commented 1 year ago

You are not alone @maddinhome. I can report the same. It's stable for weeks now. I once got prompted for re-signing the integration again but that's all. I am still on 7.31. Very strange.

phaelyx commented 1 year ago

Same issued with newest HA and FB 7530 (FW 7.29, no newer available)

No Error Message in FB, HA same as above login failed 0000000000000000

Yesterday I installed a new HA with HA OS and could connect in the beginning, but a few minutes later, same Error again. FB Router and Phonebook working correctly... Checked User and PW several times, tried different FB Users: Always the same error. Reboot FB and HA without effect.

Any suggestions what to do ?

mib1185 commented 1 year ago

For those who still have issues, please provide debug logs as described in https://github.com/home-assistant/core/issues/83987#issuecomment-1351679013

phaelyx commented 1 year ago

Seems to be a miracle when writing a comment here ... Updated my Fritzbox 7530 to 7.50, updated HA to Home Assistant 2023.3.0 Supervisor 2023.01.1 Operating System 9.5 Frontend 20230301.0 - latest

activated debug logs as described above restarted everything and wanted to upload the log, but

-> everything works now :-)

jayme-github commented 1 year ago

For those who still have issues, please provide debug logs as described in #83987 (comment)

Unfortunately there is not much that is logged. Even with fritzbox component and pyfritzhome in debug, only the following is logged:

2023-03-06 04:17:41.309 INFO (SyncWorker_3) [pyfritzhome.fritzhome] Updating Devices ...
2023-03-06 04:17:42.920 WARNING (SyncWorker_3) [pyfritzhome.fritzhome] login failed 0000000000000000
2023-03-06 04:17:42.940 ERROR (MainThread) [homeassistant.components.fritzbox] Authentication failed while fetching b71d5c3adcfec1233b9723b04e770dd6 data: login for user="homeassistant-smarthome" failed

I'm on Home Assistant 2023.2.5 but this happened for multiple/all older versions I was running as well sporadically.

This happens to me from time to time (with the FRITZ!SmartHome component only, FRITZ!Box Call Monitor and AVM FRITZ!Box Tools don't seem to suffer from this. But AIUI they don't use the HTTP API, so it's a completely different thing there really). When this happens I'm asked to reconfigure the integration and I can do so with the same password as before so I would assume this is a temporary issue.

Correlating logs from the fritzbox itself (in German only. Unfortunately the fritzbox does reboot and flush logs when changing the language):

06.03.23 04:17:43 Anmeldung einer App des Benutzers homeassistant von IP-Adresse 192.168.2.240.
06.03.23 04:17:42 Anmeldung an der FRITZ!Box-Benutzeroberfläche von IP-Adresse 192.168.2.240 gescheitert (ungültige Sitzungskennung). Zur Sicherheit werden alle noch gültigen Sitzungen zur IP-Adresse 192.168.2.240 beendet.
06.03.23 04:17:42 Anmeldung des Benutzers homeassistant-smarthome an der FRITZ!Box-Benutzeroberfläche von IP-Adresse .
06.03.23 04:17:12 Anmeldung einer App des Benutzers homeassistant von IP-Adresse 192.168.2.240.
06.03.23 04:17:12 Anmeldung an der FRITZ!Box-Benutzeroberfläche von IP-Adresse 192.168.2.240 gescheitert (ungültige Sitzungskennung). Zur Sicherheit werden alle noch gültigen Sitzungen zur IP-Adresse 192.168.2.240 beendet.
06.03.23 04:17:12 Anmeldung des Benutzers smarthome an der FRITZ!Box-Benutzeroberfläche von IP-Adresse .

The user smarthome is used by a different tool gathering metrics and for me this looks a bit like a race condition between these two. The login code of both (fritzhome and fritzconnection) for the HTTP API suggest that it's not unusual that the login fails (with valid credentials) returning the session ID 0000000000000000:

https://github.com/hthiery/python-fritzhome/blob/faf67f486db582b5d6ebcc502b92a0096857d75f/pyfritzhome/fritzhome.py#L93 https://github.com/kbr/fritzconnection/blob/183b1044219a9ec69cf01e1bc6e8aac9fabf1b75/fritzconnection/core/fritzhttp.py#L99

Also fritzhome uses the deprecated md5 challenge with HTTP GET instead of pbkdf2 and HTTP POST (as does the other metrics scraper connecting as user smarthome in my case). Maybe that's the reason why the fritzbox invalidated all sessions from the host.

The lazy way out is probably to retry the call (with exponential backoff) - if that would be possible.

mib1185 commented 1 year ago

they don't use the HTTP API

that's correct - FritzSmarthome uses the http, but FritzTools and FritzCallmonitor the tr064 api (see here for details)

Correlating logs from the fritzbox itself (in German only. Unfortunately the fritzbox does reboot and flush logs when changing the language):

06.03.23 04:17:43 Anmeldung einer App des Benutzers homeassistant von IP-Adresse 192.168.2.240.
06.03.23 04:17:42 Anmeldung an der FRITZ!Box-Benutzeroberfläche von IP-Adresse 192.168.2.240 gescheitert (ungültige Sitzungskennung). Zur Sicherheit werden alle noch gültigen Sitzungen zur IP-Adresse 192.168.2.240 beendet.
06.03.23 04:17:42 Anmeldung des Benutzers homeassistant-smarthome an der FRITZ!Box-Benutzeroberfläche von IP-Adresse .
06.03.23 04:17:12 Anmeldung einer App des Benutzers homeassistant von IP-Adresse 192.168.2.240.
06.03.23 04:17:12 Anmeldung an der FRITZ!Box-Benutzeroberfläche von IP-Adresse 192.168.2.240 gescheitert (ungültige Sitzungskennung). Zur Sicherheit werden alle noch gültigen Sitzungen zur IP-Adresse 192.168.2.240 beendet.
06.03.23 04:17:12 Anmeldung des Benutzers smarthome an der FRITZ!Box-Benutzeroberfläche von IP-Adresse .

there are three different users used from same? IP ... i think the fritzbox does not like that 🤔 try to use same user for FritzTools, FritzSmarthome and FritzCallmanager integration

Also fritzhome uses the deprecated md5 challenge with HTTP GET instead of pbkdf2 and HTTP POST (as does the other metrics scraper connecting as user smarthome in my case). Maybe that's the reason why the fritzbox invalidated all sessions from the host.

this should be reported to the python-fritzhome repository.

jayme-github commented 1 year ago

there are three different users used from same? IP ... i think the fritzbox does not like that thinking try to use same user for FritzTools, FritzSmarthome and FritzCallmanager integration The same happens with only one user for everything. I've only split them up to have a better understanding of which one triggers the actual issue. My understanding is that if one of the logins fail for whatever reason (the race condition part) the FritzBox invalidates all sessions from the source IP of the failed login.

Also fritzhome uses the deprecated md5 challenge with HTTP GET instead of pbkdf2 and HTTP POST (as does the other metrics scraper connecting as user smarthome in my case). Maybe that's the reason why the fritzbox invalidated all sessions from the host.

this should be reported to the python-fritzhome repository.

Will do. Do you happen do have an understanding on why python-fritzhome is used for that part whereas the other FritzBox integrations use fritzconnection (which has pkbdf2 login implemented)? It's probably for historical reasons but it seems a bit counter intuitive to depend on two different libraries here.

mib1185 commented 1 year ago

The Fritz!Smarthome and Fritz!Tools integrations were developed independently, further the rely on different APIs of the FritzBox. The python-fritzhome lib was the first which implements the http api (because it is the intention of this lib), but the fritzconnection got the http api only now in the last few weeks/months, so it is quite new and maybe not fully functional, yet.

issue-triage-workflows[bot] commented 1 year ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.