search

Somfy-Developer / Somfy-TaHoma-Developer-Mode

A collection of requests to use a local API with Somfy TaHoma gateways
147 stars 12 forks source link

local-API does not work #130

Closed elschnet closed 6 months ago

elschnet commented 6 months ago

I have a Tahoma and it has been in developer mode for several days. I can't get Overkiz to work with the local api. It doesn't matter whether I use the gateway-nnnn-nnnn-4442.local or the local IP address of the Tahome box. I have checked the Somfy credentials several times, that's not the problem - it works with the online API.

Here is a log of my last three of countless tries: -with gateway-nnnn-nnnn-4442.local = "Connection failed" (of course I have replaced nnnn-nnnn with my complete somfy-pin-code) -with http://IP-ADRESSE-TAHOMA = "Connection failed" -only with IP-ADRESS-TAHOMA = "Unknown error"

2024-04-30 13:27:18.751 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.072 seconds (success: True)
2024-04-30 13:27:35.097 DEBUG (MainThread) [homeassistant.components.overkiz] Cannot connect to host gateway-xxxx-xxxx-4442.local:8443 ssl:default [Name does not resolve]
2024-04-30 13:27:48.742 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.GATEWAY_SYNCHRONIZATION_STARTED: 'GatewaySynchronizationStartedEvent'>, timestamp=1714476455085, setupoid=_CountingAttr(...)
2024-04-30 13:27:48.742 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.TOKEN_CREATED: 'TokenCreatedEvent'>, timestamp=1714476455731, setupoid=_CountingAttr(...)
2024-04-30 13:27:48.742 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.GATEWAY_SYNCHRONIZATION_ENDED: 'GatewaySynchronizationEndedEvent'>, timestamp=1714476455731, setupoid=_CountingAttr(...)
2024-04-30 13:27:48.742 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.064 seconds (success: True)
2024-04-30 13:28:18.756 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.GATEWAY_SYNCHRONIZATION_STARTED: 'GatewaySynchronizationStartedEvent'>, timestamp=1714476476334, setupoid=_CountingAttr(...)
2024-04-30 13:28:18.757 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.TOKEN_CREATED: 'TokenCreatedEvent'>, timestamp=1714476476981, setupoid=_CountingAttr(...)
2024-04-30 13:28:18.757 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.GATEWAY_SYNCHRONIZATION_ENDED: 'GatewaySynchronizationEndedEvent'>, timestamp=1714476476982, setupoid=_CountingAttr(...)
2024-04-30 13:28:18.757 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.078 seconds (success: True)
2024-04-30 13:28:48.746 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.068 seconds (success: True)
2024-04-30 13:29:18.754 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.075 seconds (success: True)
2024-04-30 13:29:48.762 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.083 seconds (success: True)
2024-04-30 13:30:09.640 DEBUG (MainThread) [homeassistant.components.overkiz] Cannot connect to host http:443 ssl:default [Connect call failed ('45.76.93.104', 443)]
2024-04-30 13:30:18.739 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.061 seconds (success: True)
2024-04-30 13:30:45.273 ERROR (MainThread) [homeassistant.components.overkiz] Unknown error
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/overkiz/config_flow.py", line 233, in async_step_local
    user_input = await self.async_validate_input(user_input)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/overkiz/config_flow.py", line 70, in async_validate_input
    user_input[CONF_TOKEN] = await self._create_local_api_token(
                             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/overkiz/config_flow.py", line 412, in _create_local_api_token
    await local_client.login()
  File "/usr/local/lib/python3.12/site-packages/pyoverkiz/client.py", line 190, in login
    await self.register_event_listener()
  File "/usr/local/lib/python3.12/site-packages/backoff/_async.py", line 151, in retry
    ret = await target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyoverkiz/client.py", line 583, in register_event_listener
    response = await self.__post("events/register")
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/pyoverkiz/client.py", line 917, in __post
    await self.check_response(response)
  File "/usr/local/lib/python3.12/site-packages/pyoverkiz/client.py", line 984, in check_response
    raise MissingAuthorizationTokenException(message)
pyoverkiz.exceptions.MissingAuthorizationTokenException: Missing authorization token
2024-04-30 13:30:48.697 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.GATEWAY_SYNCHRONIZATION_STARTED: 'GatewaySynchronizationStartedEvent'>, timestamp=1714476645001, setupoid=_CountingAttr(...)
2024-04-30 13:30:48.698 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.TOKEN_CREATED: 'TokenCreatedEvent'>, timestamp=1714476645718, setupoid=_CountingAttr(...)
2024-04-30 13:30:48.698 DEBUG (MainThread) [homeassistant.components.overkiz] Event(name=<EventName.GATEWAY_SYNCHRONIZATION_ENDED: 'GatewaySynchronizationEndedEvent'>, timestamp=1714476645718, setupoid=_CountingAttr(...)
2024-04-30 13:30:48.698 DEBUG (MainThread) [homeassistant.components.overkiz] Finished fetching device events data in 0.020 seconds (success: True)

The online API works. How do I get the local api to work? What am I doing wrong? I used Home Assistant in the Docker version, if that's important.

teddywear21 commented 6 months ago

I face the same issue (Home-Assistant integration works with the cloud, but not on the local api). I also tried multiple ways to communicate with the local api, without any answer from it as expected. It looks like somfy says online that the developer-mode is enabled, while the box didn't get the command to activate it. Is there any option to start the activation of the local api again?

BernoldNL commented 6 months ago

Guys, shouldn't these issues be on HA's repositories instead?

There are some things that I notice:

-with gateway-nnnn-nnnn-4442.local = "Connection failed" (of course I have replaced nnnn-nnnn with my complete somfy-pin-code) -> Try with https://

teddywear21 commented 6 months ago

Hi @BernoldNL,

thank you for your answer. Of course it is related to HA but the local gateway is not answering as expected. There should be a way to figure out if the gateway is answering at all. And for my case, I tested a lot and can say relatively sure, it doesn't answer any call so far, though somfy tells me, that the local api is working. I found a page (https://somfy-developer.github.io/Somfy-TaHoma-Developer-Mode/#/Api/get_apiVersion) where there are multiple tests available, so far, none of them worked for my system. It's not related to any port-filtering or firewall-setting. So I'am afraid, that I will not ever be able to get this system running.

Best regards!

BernoldNL commented 6 months ago

@teddywear21 That Swagger page does not work for me either. There are a few things causing that:

  1. I'm on Windows and curl parameters are different.
  2. My gateway is on a different subnet for IoT devices. That breaks the .local mDNS name. But the gateway can be reached on its IP address.
  3. You need to tell curl to ignore the gateway's self-signed certificate with the --insecure parameter.

If you're on Windows too, you could give the following a try from a command line: curl https://192.168.20.123:8443/enduser-mobile-web/1/enduserAPI/apiVersion --insecure (Replace the IP address with yours)

Note that this will only work with apiVersion. For the other commands you will need a valid token first and then pass it along.

By the way, which gateway are you using and from which geographical region (EMEA, Northern America, Asia/Pacific)?

For HA specific, maybe @tetienne may be able to point you in the right direction.

teddywear21 commented 6 months ago

@BernoldNL The region is EMEA (Germany) and I'am using a Tahoma Switch. I'am working on ubuntu which is giving me the following response when using the IP:

  1. curl https://192.168.178.135:8443/enduser-mobile-web/1/enduserAPI/apiVersion --insecure
  2. (7) Failed to connect to 192.168.178.135 port 8443 after 201 ms: Could't connect to server

I enabled / disabled ufw, which is not making any difference. The gateway answers on ping flawless...

BernoldNL commented 6 months ago

@teddywear21 Just to be sure about the parameters I just checked on Ubuntu, but it works without a problem. (I think it should probably be something like: curl -X "GET" "https://192.168.178.135:8443/enduser-mobile-web/1/enduserAPI/apiVersion" --insecure)

I would advise you to contact Somfy support and ask them to check if Developer Mode is properly enabled. Mention that Developer Mode does not work.

teddywear21 commented 6 months ago

@BernoldNL Thanks again! Somfy was not helpfull so far, but I sent them an e-mail again today and keep you updated.

elschnet commented 6 months ago

Hi @BernoldNL, thanks for your answers.

My Region is EMEA (Germany) and I'm also using a Tahoma Switch.

I get these answer for the apiVersion:

I think my problem is maybe, that my local network is not secured?

When I try to add a new local hub with "https://10.70.80.43:8443" as Host and click the checkbox "SSL-Zertifikat überprüfen" ON, I get the result: "Es kann keine Verbindung zum Host hergestellt werden, die Zertifikatsüberprüfung ist fehlgeschlagen." (No connection to the host can be established, the certificate check has failed.) The message in the logfile is still for port 443 here: homeassistant.components.overkiz] Connection to host https:443 ssl:True not possible [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1000)')]

When I try the same and the checkbox "SSL-Zertifikat überprüfen" is OFF, I get the result: "Verbindung fehlgeschlagen" (Connection failed)

What do I have to enter at Host to instruct something like "--insecure"?

I have no idea whether this is an HA or an HA-Overkiz problem :o)

I'll very busy (traveling a lot) over the next few weeks. My response times could be long. Sorry for that. If Overkiz only works with a secure local network, that's okay too. I wanted to tackle that soon anyway, when I have more time again. Best regards.

BernoldNL commented 6 months ago

@elschnet This is clearly a different issue. The response you get from the curl command is correct. This means for your TaHoma Switch the enabling of Developer Mode has definitely been successful and it shows local API basically does work.

So it's clearly a Home Assistant issue. You need to figure out if HA can get the token that is required for the other API commands (only apiVersion does not require the token). I don't know where HA saves the token after it gets it from the cloud, otherwise you could check if it present there. I would advise you to ask the HA people for help. I'm not using HA myself (I developed an integration for a different home automation system), so unfortunately I can't help you with that.

teddywear21 commented 6 months ago

@elschnet: Nice to see how it looks when the gateway answers. This is very helpfull for me! I tried your exact command and again, got the answer, that curl "couldn't connect to server". In addition, I tried a portscan with "sudo nmap -sT -p- 192.168.178.135", telling me, that the gateway has only one port open: 32020/tcp. Maybe you can do me a favour and try this for your system too? If your gateway answers the nmap scan differently, I hope I can tell Somfy to fix this.

BernoldNL commented 6 months ago

@teddywear21 Oh wow. Very interesting! This is actually why I asked you about your region and gateway model. See, a user of my integration in the USA is trying with a TaHoma Switch NA (North America) version, which seems to be quite a new model over there (The common model was the TaHoma Beecon, which uses the different, closed Synergy API). His problem is exactly the same as yours.

I thought it was just a problem with the NA version, which is different anyway (no io-Homecontrol protocol). Just like you, he could enable Developer Mode. And he even got a token.

I had him do a port scan as well to compare with mine.

Here is his scan:

nw capture 1

And mine:

afbeelding

As you can see, the difference is port 8443.

I asked about this in https://github.com/Somfy-Developer/Somfy-TaHoma-Developer-Mode/issues/117 Unfortunately, no reply from Somfy yet. How about you add your comment there, to let Somfy know it's not just with the NA version? Or we could open a new issue. That way @elschnet can close this one.

teddywear21 commented 6 months ago

@BernoldNL It seems I have good news for you. After I wrote Somfy yesterday they remotely reseted my Tahoma Switch and after this, I was able to activate the developer mode again. By then port 8443 was open and answering as expected. It's a bit anoying, that I had to ask them three times about my issue, but now it's fine. Maybe this would help in your users case too. By the way: there is an option to reset the box also by the user on the Somfy website, which I didn't use as I did't want to pair the shutters again. But I'am not sure if this is what Somfy did in my case. I also didn't have to pair the shutters again. Now even HA works fine, even though I had to use the IP of the gateway instead of the name (gateway-xxxx-xxxx-xxxx.local:8443). From my side, the issue is solved.

Schwebebahn commented 6 months ago

Hi, I have the same problem. After some time, the switch is unavailable in my local network. Unavailable in my Home Assistant (Overkiz / Local API) and also in Apple Homekit. Can't ping and did not get an answer when asking for the version by curl, but still works with the cloud app. When I reboot the device, it works for some time. My WiFi is from UniFi, when I force the switch to connect to an other access point, it is also working for some time. So, could it be: a) That the switch is blocking access from local network? b) That UniFi is related to the problem?

teddywear21 commented 6 months ago

@Schwebebahn Hi, sorry to say this, but this sounds like a different issue. You were able to integrate the system in HA, so the local API is enabled. And that it at some point stops working, even can't answer a simple ping sounds like a hardware-defect or maybe IP misconfiguration. Could there be another device using the same IP in your network? Is the IP of the gateway configured statically?

Schwebebahn commented 6 months ago

No other device is using the IP. The IP is static. I don't think it is a hardware defect, only local access by local API and HomeKit stops working. The switch is still connected to the cloud and can be controlled with the Tahoma App.

teddywear21 commented 6 months ago

@Schwebebahn Hard to figure out what the issue is, when the device is not even answering a simple ping. Would a portscan with nmap work after the devices local api crashed? Maybe you face something like a DOS, which can happen due to multiple requests on the device. Can you capture the traffic in the network with wireshark, filtering for only the gateways ip as the target?

BernoldNL commented 6 months ago

@BernoldNL It seems I have good news for you. After I wrote Somfy yesterday they remotely reseted my Tahoma Switch and after this, I was able to activate the developer mode again. By then port 8443 was open and answering as expected. It's a bit anoying, that I had to ask them three times about my issue, but now it's fine. Maybe this would help in your users case too. By the way: there is an option to reset the box also by the user on the Somfy website, which I didn't use as I did't want to pair the shutters again. But I'am not sure if this is what Somfy did in my case. I also didn't have to pair the shutters again. Now even HA works fine, even though I had to use the IP of the gateway instead of the name (gateway-xxxx-xxxx-xxxx.local:8443). From my side, the issue is solved.

@teddywear21 Thank you very much. I will let him know and we will see if that works for him as well.

@Schwebebahn I agree that this is a different problem. Another person that I spoke, with an older TaHoma Switch has to reboot his gateway every couple of months to keep it working (But he now does that scheduled with my integration, which has a function to Update Firmware, which makes the gateway reboot afterwards - even when there is no update available). It's not a general Unifi issue, because I'm using Unifi (UDM Pro and U6-LR APs) and have never noticed this issue.