TaHoma API usage warning from Somfy

thijsdejong commented 4 years ago

[ moderator ] Here is a list of other threads related to this issue:

[ \moderator ]

I just recieved the following email from Somfy. This is the only way I use their api, so I thought it would be relevant to let you know.

Recieved 6-10-2020 18:24 from TaHoma By Somfy <no-reply@somfy.com>

Dear customer,

In an ongoing effort to provide the best services to our users, we monitor the traffic generated by our applications and Smart Home boxes (TaHoma and Connexoon). It appears that you are using a third-party solution, either developed by yourself or provided by an external entity (developer or community of developers not working for Somfy). Such developments are made without our willingness and, sometimes, do not respect our usage guidelines.

We are contacting you today to update the way your script or your application behaves with our API as it is overconsuming our backend resources for your own installation. To do so, please contact the provider of your solution or update your code by October 31st.

We are asking all third-party scripts or application to reduce their polling activity on our API. In particular, the GET /setup endpoint is meant to be used once and only once by end-user session and not to regularly refresh the devices data.

As we do not want every TaHoma and Connexoon users be impacted by the overuse of some individuals, we will react in the coming months and set up rate limiters and automated bans if such changes are not applied.

Thank you for your understanding.

ekivel commented 4 years ago

I have received it too...

iMicknl commented 4 years ago

Thanks @thijsdejong. We haven't received it ourselves yet, but I got another report as well.

Could you share your polling interval with me? (and which version of ha-tahoma) are you running. I would like to understand if there is possibly a bug that could cause bad behaviour.

In particular, the GET /setup endpoint is meant to be used once and only once by end-user session and not to regularly refresh the devices data.

This is actually a great point. The original integration was using this endpoint regularly to fetch new state data, where our new version is only using it on startup or when the session is disconnected. We try to be respectful to the Somfy servers and it is good to start the dialogue again.

thijsdejong commented 4 years ago

Thanks @thijsdejong. We haven't received it ourselves yet, but I got another report as well.

Could you share your polling interval with me? (and which version of ha-tahoma) are you running. I would like to understand if there is possibly a bug that could cause bad behaviour.

In particular, the GET /setup endpoint is meant to be used once and only once by end-user session and not to regularly refresh the devices data.

This is actually a great point. The original integration was using this endpoint regularly to fetch new state data, where our new version is only using it on startup or when the session is disconnected. We try to be respectful to the Somfy servers and it is good to start the dialogue again.

I'm currently on 5528c04, but have always been very quick to update to the latest version of the integration. I have never touched the polling interval, but it seems to be set to 30 seconds (i assume this is the default). Furthermore I have 24 devices (windows and covers), not sure if that makes a difference.

larsvinc commented 4 years ago

Hi. I have also gotten this report today.

I am on v 2.3.0.

pascalkrielen commented 4 years ago

I also got the email. Using the integration in HomeAssistant. Not sure where to find any version or polling interval though ? In homeassistant the integration refers to : https://github.com/home-assistant/core/tree/dev/homeassistant/components/somfy Not using HACS. Device to control the 4 roller blinds is a Connexoon in my case.. (But I guess that does not matter in the issue.)

URBANsUNITED commented 4 years ago

Hi! I got the email too. I see that I have a connection to tahomalink.com every 30 seconds. Never changed the polling In my setup I have 1 Tahoma v2 2 Sunis wirefree io 10 roller motors io Using V2.3.1 while I got the email. Updated to 2.3.3 now.

maybe it helps...

Thanks for your great work!!!

Cheers URBANsUNITED

arjen-w commented 4 years ago

Version info can be found in HACS. Current version is 2.3.3 (aka 5528c04, you can find that here in the repo)

I got the same message, so maybe the default polling of 30 seconds is too much..

mdeakin99 commented 4 years ago

Using 2.3.3 via HACS

iMicknl commented 4 years ago

Thanks all for reporting! We are investigating and @tetienne will reach out to his contacts at Somfy to get this clarified. By the way, none of the developers of this integration did receive the email, thus we really need your input and thoughts.

Apparently not only the tahoma integration (tahomalink.com API) and somfy integration (official API) are facing issues, but other platforms like Homey are facing the same issues.

What I don't understand yet is that the mail mentions the '/setup' endpoint especially, but this is not used by the Somfy integration at all... And on the Home Assistant Community forum we have seen quite some reports of this issue with the Somfy integration as well.

I got the same message, so maybe the default polling of 30 seconds is too much..

According to the API documentation of tahomalink.com, we are allowed to pull the event listener up to every second. The bitter thing about this situation is that we rewrote the tahoma integration from scratch to remove all the 'expensive' API calls and move to their light weight event listener...

It looks like the main issue is around the /setup endpoint which we only call on login (Home Assistant start / integration added) AND on a ServerDisconnectedError. The latter could be the issue, since some people could face this a lot and thus do many calls to '/setup'.

We will work on an updated version of this integration tomorrow and it would be great if some people could give it a try with debug logging turned on.

llevering commented 4 years ago

Thanks all for reporting! We are investigating and @tetienne will reach out to his contacts at Somfy to get this clarified. By the way, none of the developers of this integration did receive the email, thus we really need your input and thoughts.

It seems that Somfy mailed all user with their own API key of official supported API, while they have issues with people using /setup endpoint of their Tahomalink website... I received the mail as well, even though I didn't use the official api for at least two months anymore.

I think your rewrite of the integration with using /setup is great and probably what Somfy is after. Once Somfy is rate limiting /setup I guess this integration is still 'safe'. Though there might be some people using the old version causing issues. I wouldn't be surprised that when rate limiting is implemented the biggest issues will be found on their own website using /setup to much. I wouldn't but to much effort into it now, I think the integration is already on the right track.

iMicknl commented 4 years ago

We just released v2.3.4, with improved debug logging. Could you please update and run this integration with debug logging enabled. Run it for a while and please share your full log here (or via email, see profile).

llevering commented 4 years ago

debug log

``` 2020-10-07 10:35:19 DEBUG (MainThread) [custom_components.tahoma.coordinator] Initialized DataUpdateCoordinator with 0:00:30 interval. 2020-10-07 10:35:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.158 seconds 2020-10-07 10:35:19 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (io:AtlanticDomesticHotWaterProductionV3IOComponent - WaterHeatingSystem - DomesticHotWaterProduction) 2020-10-07 10:35:19 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (io:AtlanticDomesticHotWaterProductionV3IOComponent - WaterHeatingSystem - DomesticHotWaterProduction) 2020-10-07 10:35:19 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (io:AtlanticDomesticHotWaterProductionV3IOComponent - WaterHeatingSystem - DomesticHotWaterProduction) 2020-10-07 10:35:19 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (io:AtlanticDomesticHotWaterProductionV3IOComponent - WaterHeatingSystem - DomesticHotWaterProduction) 2020-10-07 10:35:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.098 seconds 2020-10-07 10:36:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.083 seconds 2020-10-07 10:36:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.116 seconds 2020-10-07 10:37:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.099 seconds 2020-10-07 10:37:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.120 seconds 2020-10-07 10:38:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.096 seconds 2020-10-07 10:38:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.556 seconds 2020-10-07 10:39:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.171 seconds 2020-10-07 10:39:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.108 seconds 2020-10-07 10:40:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.105 seconds 2020-10-07 10:40:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.122 seconds 2020-10-07 10:41:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.117 seconds 2020-10-07 10:41:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.104 seconds 2020-10-07 10:42:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.108 seconds 2020-10-07 10:42:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.110 seconds 2020-10-07 10:43:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.100 seconds 2020-10-07 10:43:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.099 seconds 2020-10-07 10:44:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.087 seconds 2020-10-07 10:44:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.098 seconds 2020-10-07 10:45:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.092 seconds 2020-10-07 10:45:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.105 seconds 2020-10-07 10:46:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.133 seconds 2020-10-07 10:46:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.120 seconds 2020-10-07 10:47:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.100 seconds 2020-10-07 10:47:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.098 seconds 2020-10-07 10:48:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.099 seconds 2020-10-07 10:48:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.097 seconds 2020-10-07 10:49:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.104 seconds 2020-10-07 10:49:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.097 seconds 2020-10-07 10:50:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.140 seconds 2020-10-07 10:50:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.115 seconds 2020-10-07 10:51:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.103 seconds 2020-10-07 10:51:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.113 seconds 2020-10-07 10:52:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.152 seconds 2020-10-07 10:52:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.106 seconds 2020-10-07 10:53:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.152 seconds 2020-10-07 10:53:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.106 seconds 2020-10-07 10:54:19 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.172 seconds 2020-10-07 10:54:49 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.105 seconds ```

iMicknl commented 4 years ago

Thanks @llevering, please keep it running for a while. The Finished fetching TaHoma Event Fetcher data in 0.098 seconds is a sign that the DataUpdateCoordinator is working correctly.

We are mainly looking for any exceptions or errors and for the message Fetching all devices and state via /setup/devices. If we don't face those errors, we are honoring their limits and in that case we should be ready for October 31.

mdeakin99 commented 4 years ago

We just released v2.3.4, with improved debug logging. Could you please update and run this integration with debug logging enabled. Run it for a while and please share your full log here (or via email, see profile).

Please excuse my ignorance, where should I be putting the following?

logger: default: critical logs: custom_components.tahoma: debug

thijsdejong commented 4 years ago

We just released v2.3.4, with improved debug logging. Could you please update and run this integration with debug logging enabled. Run it for a while and please share your full log here (or via email, see profile).

Please excuse my ignorance, where should I be putting the following?

logger: default: critical logs: custom_components.tahoma: debug

configuration.yaml as stated here https://www.home-assistant.io/integrations/logger/ . It's just like any other configuration.

Edit:

Here are some logs during a Tahoma box update. Did not execute any commands, verified in the log in the tahoma app that no devices moved.

``` 2020-10-07 11:37:00 DEBUG (MainThread) [custom_components.tahoma.coordinator] Initialized DataUpdateCoordinator with 0:00:30 interval. 2020-10-07 11:37:01 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.229 seconds 2020-10-07 11:37:31 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.065 seconds 2020-10-07 11:38:01 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.065 seconds 2020-10-07 11:38:31 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.066 seconds 2020-10-07 11:39:01 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.074 seconds 2020-10-07 11:39:31 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.074 seconds 2020-10-07 11:40:01 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.062 seconds 2020-10-07 11:40:31 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.069 seconds 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.END_USER_LOGIN/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_REGISTERED/026f44c8-ac10-3e01-1400-9e04290a3eb5 (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/026f44c8-ac10-3e01-1400-9e04290a3eb5 (device: None, state: ExecutionState.INITIALIZED -> ExecutionState.NOT_TRANSMITTED) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/026f44c8-ac10-3e01-1400-9e04290a3eb5 (device: None, state: ExecutionState.NOT_TRANSMITTED -> ExecutionState.TRANSMITTED) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.REFRESH_ALL_DEVICES_STATES_COMPLETED/None (device: None, state: None -> None) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/026f44c8-ac10-3e01-1400-9e04290a3eb5 (device: None, state: ExecutionState.TRANSMITTED -> ExecutionState.IN_PROGRESS) 2020-10-07 11:41:01 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.077 seconds 2020-10-07 11:41:02 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.020 seconds 2020-10-07 11:41:03 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.019 seconds 2020-10-07 11:41:04 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.020 seconds 2020-10-07 11:41:05 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.019 seconds 2020-10-07 11:41:06 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.019 seconds 2020-10-07 11:41:07 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.020 seconds 2020-10-07 11:41:08 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.019 seconds 2020-10-07 11:41:09 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.019 seconds 2020-10-07 11:41:10 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.020 seconds 2020-10-07 11:41:11 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.024 seconds 2020-10-07 11:41:12 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.017 seconds 2020-10-07 11:41:13 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.019 seconds 2020-10-07 11:41:14 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.016 seconds 2020-10-07 11:41:15 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.018 seconds 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/026f44c8-ac10-3e01-1400-9e04290a3eb5 (device: None, state: ExecutionState.IN_PROGRESS -> ExecutionState.COMPLETED) 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:41:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.020 seconds 2020-10-07 11:41:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:41:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:41:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_DOWN/None (device: None, state: None -> None) 2020-10-07 11:41:46 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.063 seconds 2020-10-07 11:42:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.071 seconds 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_ALIVE/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f16e-ac10-3e01-646e-9c3c9a3fd922 (device: None, state: ExecutionState.INITIALIZED -> ExecutionState.NOT_TRANSMITTED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] GatewayModeChangedEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: internal://1203-8631-0841/alarm/0, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: internal://1203-8631-0841/pod/0, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] GatewayBootEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: internal://1203-8631-0841/pod/0, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] GatewayModeChangedEvent/None (device: None, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: internal://1203-8631-0841/pod/0, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1a3-ac10-3e01-2c20-6d2d491d09cd (device: None, state: ExecutionState.INITIALIZED -> ExecutionState.QUEUED_SERVER_SIDE) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1ac-ac10-3e01-2c20-6d2d7b943746 (device: None, state: ExecutionState.INITIALIZED -> ExecutionState.QUEUED_SERVER_SIDE) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f16e-ac10-3e01-646e-9c3c9a3fd922 (device: None, state: ExecutionState.NOT_TRANSMITTED -> ExecutionState.TRANSMITTED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f16e-ac10-3e01-646e-9c3c9a3fd922 (device: None, state: ExecutionState.TRANSMITTED -> ExecutionState.IN_PROGRESS) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f16e-ac10-3e01-646e-9c3c9a3fd922 (device: None, state: ExecutionState.IN_PROGRESS -> ExecutionState.COMPLETED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1a3-ac10-3e01-2c20-6d2d491d09cd (device: None, state: ExecutionState.QUEUED_SERVER_SIDE -> ExecutionState.NOT_TRANSMITTED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1a3-ac10-3e01-2c20-6d2d491d09cd (device: None, state: ExecutionState.NOT_TRANSMITTED -> ExecutionState.TRANSMITTED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/02710bc7-ac10-3e01-646e-9c3ce79da590 (device: None, state: ExecutionState.INITIALIZED -> ExecutionState.NOT_TRANSMITTED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: internal://1203-8631-0841/alarm/0, state: None -> None) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/02710bc7-ac10-3e01-646e-9c3ce79da590 (device: None, state: ExecutionState.NOT_TRANSMITTED -> ExecutionState.TRANSMITTED) 2020-10-07 11:42:46 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.089 seconds 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1a3-ac10-3e01-2c20-6d2d491d09cd (device: None, state: ExecutionState.TRANSMITTED -> ExecutionState.IN_PROGRESS) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1a3-ac10-3e01-2c20-6d2d491d09cd (device: None, state: ExecutionState.IN_PROGRESS -> ExecutionState.COMPLETED) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1ac-ac10-3e01-2c20-6d2d7b943746 (device: None, state: ExecutionState.QUEUED_SERVER_SIDE -> ExecutionState.NOT_TRANSMITTED) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1ac-ac10-3e01-2c20-6d2d7b943746 (device: None, state: ExecutionState.NOT_TRANSMITTED -> ExecutionState.TRANSMITTED) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/02710bc7-ac10-3e01-646e-9c3ce79da590 (device: None, state: ExecutionState.TRANSMITTED -> ExecutionState.IN_PROGRESS) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_FUNCTION_CHANGED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] GatewayModeChangedEvent/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_FUNCTION_CHANGED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/02710bc7-ac10-3e01-646e-9c3ce79da590 (device: None, state: ExecutionState.IN_PROGRESS -> ExecutionState.COMPLETED) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1ac-ac10-3e01-2c20-6d2d7b943746 (device: None, state: ExecutionState.TRANSMITTED -> ExecutionState.IN_PROGRESS) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.EXECUTION_STATE_CHANGED/0270f1ac-ac10-3e01-2c20-6d2d7b943746 (device: None, state: ExecutionState.IN_PROGRESS -> ExecutionState.COMPLETED) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolUnavailableEvent/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] DeviceProtocolAvailableEvent/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/1546402, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/3153159, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/591787, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/3837249, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/6048714, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/5048043, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/6691736, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/6805465, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/6826978, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/6846553, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/7022875, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/11464190, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/11983678, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/12079682, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/12591733, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/13416522, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/12719514, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/13789384, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/14295127, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/14399858, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: io://1203-8631-0841/15834547, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:43:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.157 seconds 2020-10-07 11:43:46 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.099 seconds 2020-10-07 11:44:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.075 seconds 2020-10-07 11:44:46 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.071 seconds 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.END_USER_LOGIN/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_STATE_CHANGED/None (device: internal://1203-8631-0841/pod/0, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.REFRESH_ALL_DEVICES_STATES_COMPLETED/None (device: None, state: None -> None) 2020-10-07 11:45:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.072 seconds 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.END_USER_LOGIN/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] PushSubscriptionCreatedEvent/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_STARTED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.GATEWAY_SYNCHRONIZATION_ENDED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.REFRESH_ALL_DEVICES_STATES_COMPLETED/None (device: None, state: None -> None) 2020-10-07 11:45:46 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.069 seconds 2020-10-07 11:46:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.073 seconds ```

vlebourl commented 4 years ago

you can add it to your configuration.yaml file. Beware to respect the indentation:

logger:
  default: critical
  logs:
    custom_components.tahoma: debug

mdeakin99 commented 4 years ago

you can add it to your configuration.yaml file. Beware to respect the indentation:
logger:
  default: critical
  logs:
    custom_components.tahoma: debug

Thanks. I thought it might be but wanted to get the right answer so i can start logging now and get some logs for the team. Will post something here in an hour or so.

vlebourl commented 4 years ago

Thanks @llevering, please keep it running for a while. The Finished fetching TaHoma Event Fetcher data in 0.098 seconds is a sign that the DataUpdateCoordinator is working correctly.

We are mainly looking for any exceptions or errors and for the message Fetching all devices and state via /setup/devices. If we don't face those errors, we are honoring their limits and in that case we should be ready for October 31.

As said by Mick, please only post in case of error messages or Fetching all devices and state via /setup/devices to keep the thread clean. Thanks

URBANsUNITED commented 4 years ago

It seems that Somfy mailed all user with their own API key of official supported API, while they have issues with people using /setup endpoint of their Tahomalink website... I received the mail as well, even though I didn't use the official api for at least two months anymore.

Couldn't it be that Somfy reacts on old situations with a very very long delay? Means, there is no issues with this setup (ha-tahoma) , but the old official HA API way (https://www.home-assistant.io/integrations/somfy)

Just my thoughts as I know how slow Somfy usually is - I am in direct contact with a roller motor developer ;)

vlebourl commented 4 years ago

possible, except the official API way does not use the /setup endpoint... so their email is quite strange...

vlebourl commented 4 years ago

Please, keep the reported log messages to the minimum, ie. any exceptions or errors messages and the message Fetching all devices and state via /setup/devices. Avoid pasting a long list of other debugging message that are not relevant to this issue like Finished fetching TaHoma Event Fetcher messages or [custom_components.tahoma.coordinator] EventName. Also, please use code formating by pasting the logs in a code block : ``` your log here ```

mdeakin99 commented 4 years ago

Removed all of the 'finished fectching...' from the logs. My limited understanding of the log makes me thing nothing API related here.

2020-10-07 10:48:18 DEBUG (MainThread) [custom_components.tahoma.coordinator] Initialized DataUpdateCoordinator with 0:00:30 interval.
2020-10-07 10:48:18 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (upnpcontrol:SonosPlayOneComponent - MusicPlayer - MediaRenderer)
2020-10-07 10:48:18 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (upnpcontrol:SonosPlayOneComponent - MusicPlayer - MediaRenderer)
2020-10-07 10:48:18 DEBUG (MainThread) [custom_components.tahoma] Unsupported TaHoma device detected (upnpcontrol:SonosPlayOneComponent - MusicPlayer - MediaRenderer)
2020-10-07 10:53:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/40, state: None -> None)
2020-10-07 10:53:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/1, state: None -> None)
2020-10-07 10:53:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/21, state: None -> None)
2020-10-07 10:53:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/37, state: None -> None)
2020-10-07 11:03:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/40, state: None -> None)
2020-10-07 11:03:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/21, state: None -> None)
2020-10-07 11:03:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/37, state: None -> None)
2020-10-07 11:03:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/15, state: None -> None)
2020-10-07 11:13:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/8, state: None -> None)
2020-10-07 11:13:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/15, state: None -> None)
2020-10-07 11:23:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/18, state: None -> None)
2020-10-07 11:23:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/8, state: None -> None)
2020-10-07 11:33:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/18, state: None -> None)
2020-10-07 11:43:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/1, state: None -> None)
2020-10-07 11:43:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_UNAVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/8, state: None -> None)
2020-10-07 11:53:48 DEBUG (MainThread) [custom_components.tahoma.coordinator] EventName.DEVICE_AVAILABLE/None (device: hue://1211-4023-3539/001788292aad/lights/8, state: None -> None)

iMicknl commented 4 years ago

@mdeakin99 thanks for sharing! Indeed your log looks clean as well, just messages related to the event polling mechanism.

Please keep the integration running with debug logging for a few days to understand if we even call the setup method. The first signs look promising and show that Somfy's email could be outdated / not targeted at us. 👍🏻

jensihnow commented 4 years ago

I did recieve the same warning email. My take was that I got it because I'm actually running the old integration and this one in parrallel. Well I'm going to disable the old one today and go to latest version of this one.

Also, I was tempted to answer: "OK Somfy, how about a LOCAL API!"

jensihnow commented 4 years ago

Just wrote a summary for Somfy and was about to reply, too bad they are sending out from no-reply@somfy.com. I know this is doesn't really belong here, but in case someone from Somfy will look at this: You need to keep conversation with your customers going! What I have seen in the past 10 years since I started using TaHoma made me think to not buy anything for the next home from Somfy.

llevering commented 4 years ago

Please keep the integration running with debug logging for a few days to understand if we even call the setup method.

Still not a single mention of the /setup endpoint 👍

xnk commented 4 years ago

I did recieve the same warning email. My take was that I got it because I'm actually running the old integration and this one in parrallel. Well I'm going to disable the old one today and go to latest version of this one.

Also, I was tempted to answer: "OK Somfy, how about a LOCAL API!"

Local API is definitely the way to go. Unfortunately there's nothing open by default on the gateway (it's something that should be fairly trivial to add though, as long as you've successfully entered RomBOOT SAM-BA mode over USB). Forcing the NAND flash CE high during boot will automatically drop you in RomBOOT. It's been a while since I looked at it last but if Somfy starts messing with the API again I guess it's time to bring this to the table. There SEEMS to be a way to open local diagnostics if the gateway receives a command from their cloud instance, but we obviously want to enable local access without having the Somfy servers involved at all.

After all the gateway (at least the Kizbox Mini / Connexxoon) is a standard at91sam9g25 linux system plus the stm32f101rct6 controlling the radio transceiver. Funny enough the two processors speak UART and that UART is available on a non-populated pin header so if you want a brute-force local access that's the UART you would use. Unencrypted, unauthenticated communication.

vlebourl commented 4 years ago

Quite technical, gotta say, I didn't get much off of that message 😁 But if you ever find a way to force open the tahoma box, I'd be happy to try it!

xnk commented 4 years ago

Quite technical, gotta say, I didn't get much off of that message 😁 But if you ever find a way to force open the tahoma box, I'd be happy to try it!

Sorry for my technobabble :) I just wanted to share some of the (very) early findings. As of right now you have to open the enclosure and temporarily short a signal on the board in order to be able to inject custom code over USB, so it's not for everyone yet.

iMicknl commented 4 years ago

Please keep the integration running with debug logging for a few days to understand if we even call the setup method.

Still not a single mention of the /setup endpoint 👍

Great to hear! I have received some other logs and in none of the cases we are calling the /setup endpoint. Please keep an eye on your logs to see if Fetching all devices and state via /setup/devices is present in the coming days.

It looks like a false positive from Somfy or at least, they communicated the wrong message. Let's hope that they will clarify this situation soon. @tetienne has been in contact with Somfy and they promised to get back to us.

I am preparing a more complete answer with the team to clarify as soon as possible.

(and for the Dutch readers, the Somfy TaHoma situation was featured on Tweakers.net yesterday)

thijsdejong commented 4 years ago

Had one weird occurance today, this is the only time in 2 days it mentions setup:

2020-10-09 16:04:46 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.072 seconds
2020-10-09 16:05:16 DEBUG (MainThread) [custom_components.tahoma.coordinator] Not authenticated
2020-10-09 16:05:16 ERROR (MainThread) [custom_components.tahoma] Unexpected error fetching TaHoma Event Fetcher data: Bad credentials
Traceback (most recent call last):
  File "/config/custom_components/tahoma/coordinator.py", line 63, in _async_update_data
    events = await self.client.fetch_events()
  File "/usr/local/lib/python3.8/site-packages/pyhoma/client.py", line 163, in fetch_events
    response = await self.__post(f"events/{self.event_listener_id}/fetch")
  File "/usr/local/lib/python3.8/site-packages/pyhoma/client.py", line 263, in __post
    await self.check_response(response)
  File "/usr/local/lib/python3.8/site-packages/pyhoma/client.py", line 299, in check_response
    raise NotAuthenticatedException(message)
pyhoma.exceptions.NotAuthenticatedException: Not authenticated

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 143, in async_refresh
    self.data = await self._async_update_data()
  File "/config/custom_components/tahoma/coordinator.py", line 71, in _async_update_data
    await self.client.login()
  File "/usr/local/lib/python3.8/site-packages/pyhoma/client.py", line 83, in login
    response = await self.__post("login", data=payload)
  File "/usr/local/lib/python3.8/site-packages/pyhoma/client.py", line 263, in __post
    await self.check_response(response)
  File "/usr/local/lib/python3.8/site-packages/pyhoma/client.py", line 295, in check_response
    raise BadCredentialsException(message)
pyhoma.exceptions.BadCredentialsException: Bad credentials
2020-10-09 16:05:16 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 0.117 seconds
2020-10-09 16:05:46 DEBUG (MainThread) [custom_components.tahoma.coordinator] Not authenticated
2020-10-09 16:05:47 DEBUG (MainThread) [custom_components.tahoma.coordinator] Fetching all devices and state via /setup/devices
2020-10-09 16:05:48 INFO (MainThread) [custom_components.tahoma] Fetching TaHoma Event Fetcher data recovered
2020-10-09 16:05:48 DEBUG (MainThread) [custom_components.tahoma] Finished fetching TaHoma Event Fetcher data in 2.029 seconds

iMicknl commented 4 years ago

Thanks @thijsdejong. This is indeed a weird situation where we don't catch the error. This is due to the fact that the tahomalink API randomly throws a BadCredentialsException, even though your credentials are correct..

Great that the DataUpdateCoordinator is able to recover from this and fetches the latest state via /setup. As long as this happens once in a few days, we still call the /setup endpoint less than the original tahomalink.com app.

xekil commented 4 years ago

Hello, the custom component TAHOMA is calling / setup?

vlebourl commented 4 years ago

Yes.

xekil commented 4 years ago

ok, thank you, is it better to use the official integration then?

vlebourl commented 4 years ago

It depends on the devices you would like to control. The official API is quite limited but does work well with some of them, and is not gonna be targeted by Somfy. The unofficial one virtually supports every devices that can be linked to Tahoma, but as you well know, it's future isn't written yet...

xekil commented 4 years ago

I have IO roller shutters and IO thermostat. With the official integration, the thermostat is not supported and the shutter status feedback is longer than with the custom component

iMicknl commented 4 years ago

Hello, the custom component TAHOMA is calling / setup?

We only call /setup when you start Home Assistant or when we run into an error, thus this should be acceptable. Apparently the main issue with the TaHoma integration is that we keep the session 24/7 open, since we would like to receive state changes in Home Assistant.

TheNr1Guest commented 3 years ago

@iMicknl So when I restart normally and there are no errors, this /setup is not called?

Also, did someone check with Somfy if calling their API once every 30 seconds is OK or if that is still too much (since that would result in 43200 API calls a day)?

vlebourl commented 3 years ago

From what I understood, I don't think the polling interval is really the issue. It's maintaining open a connection to the client that they don't like, which is required to get regular updates of sensors and devices states.

iMicknl commented 3 years ago

@iMicknl So when I restart normally and there are no errors, this /setup is not called?

No. I just edited my previous post to clarify. When the TaHoma integration is loaded, we call /setup to retrieve the latest states and values, thus on every start of Home Assistant. Next to that, we call /setup when we have to renew the event listener, since we need to retrieve the latest state. This only happens on errors, and based on the extra debug logging added this is a rare event.

Also, did someone check with Somfy if calling their API once every 30 seconds is OK or if that is still too much (since that would result in 43200 API calls a day)?

As @vlebourl mentioned, their main issue is keeping the connection open. Increasing the polling interval should not matter, since the connection is still open in the background and Somfy needs to collect all events.

A future possibility could be to only register and open the event listener after an action has been executed, however this means that Home Assistant can not respond to state updates outside Home Assistant. (which sucks in my opinion).

TheNr1Guest commented 3 years ago

@iMicknl OK thanks and I take it there is no way to avoid this /setup call after a reboot? It can happen I reboot frequently in a small time window, especially when I am developing new stuff in HA.

But my underlying fear is that I get banned and won't be able to control my devices via HA. I don't like the fact they communicated it in this way, if they just add rate limiters to the /setup call that would be fine, but now I'm on edge...

iMicknl commented 3 years ago

@iMicknl OK thanks and I take it there is no way to avoid this /setup call after a reboot? It can happen I reboot frequently in a small time window, especially when I am developing new stuff in HA.

Currently this is not possible. You could create a new issue with a feature request, but I don't think this will be built soon, since there are some other pressing issues.

But my underlying fear is that I get banned and won't be able to control my devices via HA. I don't like the fact they communicated it in this way, if they just add rate limiters to the /setup call that would be fine, but now I'm on edge...

Every time you login on tahomalink.com, you will pull /setup, thus I don't think this will be a big issue. The issue around calling /setup is mainly for scripts (like Homey) where they pull /setup every programmatically to receive the latest states. We use the event mechanism to avoid that, but apparently that is a way which Somfy doesn't approve as well.

larsvinc commented 3 years ago

Any resolution in this topic? Did anyone contact Somfy and discuss? A local API would solve this elegantly and simply - like the Philips Hue. My tahoma-box goes from valuable to worthless.

Tahoma-integration stopped working today. Updated the gateway, and updated the Tahoma plugin to v2.3.4. But no luck, and no particular warning in the log file.

Like @TheNr1Guest I depend on controlling this from HA.

This is frustrating. I only have stateless devices, so I would be fine with not getting state changes outside of HA? Worst case, can anyone point me to how to modify the integration to open a session only when I send a command?

jensihnow commented 3 years ago

Tahoma-integration stopped working today.

For me it is still working, what errors are you seeing?

As mentioned earlier, I sent a few lines with my feelings about this notification to Somfy. I actually got a reply asking about which integrations I use and if they are developed by myself or a third-party plugin available on an open-source platform

I did replied (just now) and asked if they can provide me any data about the issue they see. I actually wonder if they get into trouble if people are updating the HA instances and e.g. doing multiple restarts (wich gives a /setup call each time). But the App and us logging into the web client will do exactly the same.

Please get in contact with them on developer at somfy.com and express your thoughts, need and feedback.

iMicknl commented 3 years ago

Any resolution in this topic? Did anyone contact Somfy and discuss? A local API would solve this elegantly and simply - like the Philips Hue. My tahoma-box goes from valuable to worthless.

We are waiting on a response from Somfy, but we have shared our concerns... At the moment there is not much we can do.

Tahoma-integration stopped working today. Updated the gateway, and updated the Tahoma plugin to v2.3.4. But no luck, and no particular warning in the log file.

@larsvinc that is weird. Can you login via tahomalink.com? Are you using the TaHoma box or Connexoon? I am using v2.3.4 and the latest version of the TaHoma firmware without issues at the moment.

This is frustrating. I only have stateless devices, so I would be fine with not getting state changes outside of HA? Worst case, can anyone point me to how to modify the integration to open a session only when I send a command?

Could you create a new issue for this? It looks like multiple users are requesting this functionality, thus would be good to track in a new GitHub issue.

larsvinc commented 3 years ago

Hi,

I am sorry about the late response here. I have been out of my home. And now it works again. I did not see any issue in the logs. If it misbehaves I will report back.

@iMicknl: Just submitted it as a feature request: https://github.com/iMicknl/ha-tahoma/issues/300

llevering commented 3 years ago

Since yesterday I get all kind of 'Bad credentials' warnings/exceptions. Decided to restart HA in the hope to fix it, however now I get 'Bad auth'. I am wondering if this is just some other random Somfy glitch, or if they have started to block people. I see I can still login in to Tahomalink.com so credentials are ok (I have a Connexxon, so I get the credentials error, however the /setup endpoint with all data is still loading the developer console). Are other people experiencing issues?

xekil commented 3 years ago

Same problem !

TheNr1Guest commented 3 years ago

Since yesterday I get all kind of 'Bad credentials' warnings/exceptions. Decided to restart HA in the hope to fix it, however now I get 'Bad auth'. I am wondering if this is just some other random Somfy glitch, or if they have started to block people. I see I can still login in to Tahomalink.com so credentials are ok (I have a Connexxon, so I get the credentials error, however the /setup endpoint with all data is still loading the developer console). Are other people experiencing issues?

Yes, I did as well. After the first reboot did not work (same bad auth error) I decided to wait until later that day. After I rebooted later, it started to work again.

iMicknl / ha-tahoma

TaHoma API usage warning from Somfy #289