Dyson integration stopped working with 2021.2

[afk11]

The problem

I have been using the dyson integration with my hot/cold fan successfully for a while. I upgraded HA last night to 2021.2 and saw alerts that the integration couldn't initialize

What is version of Home Assistant Core has the issue?

2021.2

What was the last working version of Home Assistant Core?

2021.1

What type of installation are you running?

Home Assistant Core

Integration causing the issue

Dyson

Link to integration documentation on our website

https://www.home-assistant.io/integrations/dyson/

Example YAML snippet

dyson:
  username: EMAIL
  password: !secret SECRET
  language: GB
  devices:
    - device_id: DEVICE_ID
      device_ip: LAN_IP

Anything in the logs that might be useful for us?

# First Warning in logs

2021-02-11 03:17:41 ERROR (SyncWorker_2) [homeassistant.components.dyson] Not connected to Dyson account. Unable to add devices
2021-02-11 03:17:41 INFO (MainThread) [homeassistant.setup] Setup of domain dyson took 0.6 seconds
2021-02-11 03:17:41 ERROR (MainThread) [homeassistant.setup] Setup failed for dyson: Integration failed to initialize.
2021-02-11 03:17:41 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=persistent_notification, service=create, service_data=title=Invalid config, message=The following integrations and platforms could not be set up:
- dyson

# Later warning in logs. I'm not sure if this is actually from dyson or not. The IP looked weird, but when I googled it, it's some AWS thing. 

2021-02-11 03:17:41 ERROR (SyncWorker_2) [homeassistant.components.dyson] Not connected to Dyson account. Unable to add devices
2021-02-11 03:17:41 INFO (MainThread) [homeassistant.setup] Setup of domain dyson took 0.6 seconds
2021-02-11 03:17:41 ERROR (MainThread) [homeassistant.setup] Setup failed for dyson: Integration failed to initialize.
2021-02-11 03:17:41 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=persistent_notification, service=create, service_data=title=Invalid config, message=The following integrations and platforms could not be set up:
- dyson
Please check your config and [logs](/config/logs)., notification_id=invalid_config>
2021-02-11 03:17:41 DEBUG (SyncWorker_0) [botocore.utils] Caught retryable HTTP exception while making metadata service request to http://169.254.169.254/latest/api/token: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 169, in _new_conn
conn = connection.create_connection(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 96, in create_connection
raise err
File "/usr/local/lib/python3.8/site-packages/urllib3/util/connection.py", line 86, in create_connection
sock.connect(sa)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line 311, in send
urllib_response = conn.urlopen(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 755, in urlopen
retries = retries.increment(
File "/usr/local/lib/python3.8/site-packages/urllib3/util/retry.py", line 506, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 735, in reraise
raise value
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 699, in urlopen
httplib_response = self._make_request(
File "/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 394, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 234, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/usr/local/lib/python3.8/http/client.py", line 1255, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/usr/local/lib/python3.8/site-packages/botocore/awsrequest.py", line 92, in _send_request
rval = super(AWSConnection, self)._send_request(
File "/usr/local/lib/python3.8/http/client.py", line 1301, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.8/http/client.py", line 1250, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/usr/local/lib/python3.8/site-packages/botocore/awsrequest.py", line 120, in _send_output
self.send(msg)
File "/usr/local/lib/python3.8/site-packages/botocore/awsrequest.py", line 204, in send
return super(AWSConnection, self).send(str)
File "/usr/local/lib/python3.8/http/client.py", line 950, in send
self.connect()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 200, in connect
conn = self._new_conn()
File "/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 174, in _new_conn
raise ConnectTimeoutError(
urllib3.exceptions.ConnectTimeoutError: (<botocore.awsrequest.AWSHTTPConnection object at 0x7f41143bd5e0>, 'Connection to 169.254.169.254 timed out. (connect timeout=1)')
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.8/site-packages/botocore/utils.py", line 377, in _fetch_metadata_token
response = self._session.send(request.prepare())
File "/usr/local/lib/python3.8/site-packages/botocore/httpsession.py", line 344, in send
raise ConnectTimeoutError(endpoint_url=request.url, error=e)
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL: "http://169.254.169.254/latest/api/token"
2021-02-11 03:17:41 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=persistent_notification.invalid_config, old_state=None, new_state=<state persistent_notification.invalid_config=notifying; title=Invalid config, friendly_name=Invalid config, message=The following integrations and platforms could not be set up:
- dyson

I'm running the qcow2 image in KVM. Happy to muck about with suggested tweaks if it helps narrow down the issue.

[QuaySo]

Same issue here, running Home Assistant OS.

My error log is not exactly the same however

2021-02-11 16:04:04 ERROR (SyncWorker_0) [homeassistant.components.dyson] Not connected to Dyson account. Unable to add devices
2021-02-11 16:04:06 ERROR (MainThread) [homeassistant.setup] Setup failed for dyson: Integration failed to initialize.
2021-02-11 16:04:06 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform dyson.fan: Unable to set up component.
2021-02-11 16:04:06 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform dyson.sensor: Unable to set up component.

[afk11]

I don't get the later two warnings from your log, just the first two.

For more detail, try adding this to your config:

logger:
  default: debug
  logs:
    homeassistant.components.sensor.dyson: debug
    homeassistant.components.fan.dyson: debug
    homeassistant.components.dyson: debug

[probot-home-assistant[bot]]

dyson documentation dyson source _{^{(message by IssueLinks)}}

[QuaySo]

With the detailed logs turned on I get these warnings.

2021-02-11 16:48:30 INFO (MainThread) [homeassistant.setup] Setup of domain dyson took 1.7 seconds
2021-02-11 16:48:30 ERROR (MainThread) [homeassistant.setup] Setup failed for dyson: Integration failed to initialize.

2021-02-11 16:48:30 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=persistent_notification, service=create, service_data=title=Invalid config, message=The following integrations and platforms could not be set up:
- dyson
Please check your config and [logs](/config/logs)., notification_id=invalid_config>
2021-02-11 16:48:30 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform dyson.fan: Unable to set up component.
2021-02-11 16:48:30 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform dyson.sensor: Unable to set up component.

2021-02-11 16:48:30 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event state_changed[L]: entity_id=persistent_notification.invalid_config, old_state=None, new_state=<state persistent_notification.invalid_config=notifying; title=Invalid config, friendly_name=Invalid config, message=The following integrations and platforms could not be set up:
- dyson
Please check your config and [logs](/config/logs). @ 2021-02-11T16:48:30.802287+01:00>>
2021-02-11 16:48:30 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event persistent_notifications_updated[L]>
2021-02-11 16:48:30 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=persistent_notification, service=create, service_data=title=Invalid config, message=The following integrations and platforms could not be set up:
- dyson
- dyson.fan
Please check your config and [logs](/config/logs)., notification_id=invalid_config>
2021-02-11 16:48:30 DEBUG (MainThread) [homeassistant.core] Bus:Handling <Event call_service[L]: domain=persistent_notification, service=create, service_data=title=Invalid config, message=The following integrations and platforms could not be set up:
- dyson
- dyson.fan
- dyson.sensor
Please check your config and [logs](/config/logs)., notification_id=invalid_config>

Unfortunately no good hints pointing to the cause of the warnings as far as I can see...

[curt7000]

Same here. Running 2021.2.2 in a VENV on Ubuntu 18.04LTS. Has been running for several yrs, no issue. Restarted yesterday while creating a new Binary Sensor. Dyson integration will no longer setup, listed as invalid_config.

I have power cycled network, power cycled Dyson fans (I have 5), without fix.

[geekofweek]

I have a feeling they are blocking the User Agent, or more specifically, looking for a very specific User Agent. This has happened previously.

I've tested a few changes and yet to find one that works.

[Stanwise]

For what it's worth, it doesn't work after downgrade to 2021.1.5 either, so it looks like a Dyson-side change

[jstott1070]

Ditto, Hot & Cool fan working fine pre-update.. Now not functioning

[afk11]

@Stanwise thanks for confirming, I was trying to roll back earlier but couldn't find how :)

[DarKOrange75]

Same problem here, HA can't log into my Dyson account. It doesn't seem to be a problem with HA but with Dyson's servers.

[googanhiem]

Yep @Stanwise is correct this is a Dyson api change.. probably a cloudflare block of this components dependency libpurecool and its auth method. They've blocked it in the past (probably not even intentionally).

This means its a fix needed for libpurecool not HA.

People are looking for the fix for the dependency here

[arifroni]

On dyson home assistant integration page it is written that the integration's IoT class is 'local push'. so why HA needs to communicate with the server?

[kurozero0]

I think Dyson is just having issues authenticating on their end. I couldn't even log into the website. When trying to log into the app, it says "Sorry, we can't connect to the Dyson network. We're working to fix this. Please try again later."

[dougmaitelli]

Adding to @arifroni comment. I understand the integration uses the email and password to log into Dyson service remotely to fetch the local machines information in order to connect to the machines. In my case my machines have static IPs and they are all configured manually in my configuration.yml, so I don't know why in this case this integration even needs the email and password.

[bfayers]

In my case my machines have static IPs and they are all configured manually in my configuration.yml, so I don't know why in this case this integration even needs the email and password.

While the machines themselves run local MQTT they need credentials that only the Dyson server can provide :/

[googanhiem]

Yes, this component is a little clunky and forgotten. @shenxn is looking to build a new component that may fix a lot of the current issues.

The current component relies on this initial auth to gain control, these credentials are not cached so they're updated on every reboot of HA.

@kurozero0 The Dyson website and this component appear to use different servers, so the issues are probably unrelated

[dougmaitelli]

@kurozero0 The Dyson website and this component appear to use different servers, so the issues are probably unrelated

Possible that Dyson changed something internally that broke HA and themselves?

[DarKOrange75]

While @shenxn achieve the new component development, here is a temporary workaround which does not need to communicate with Dysons servers (talks directly with the fan's MQTT broker):

https://community.home-assistant.io/t/dyson-pure-cool-link-local-mqtt-control/217263

The down side is that you no longer have a fan entity to expose to HomeKit for example...

[kurozero0]

@googanhiem is right. The website and app appear to be working now but still nothing on HA.

[strandlie]

I have the same issue.

Tried Charles to try to find out if the app had added some new header etc. It seems that Dyson uses SSL Certificate Pinning in their iOS app (this is good, but a little impractical here), which makes Charles unable to decrypt SSL traffic, even with a custom root certificate installed. So my only conclusion so far is that they haven't changed the endpoint URL.

But if someone could shed some light on what the issue is I would be happy to make a PR for this.

[bfayers]

It seems that Dyson uses SSL Certificate Pinning in their iOS app (this is good, but a little impractical here), which makes Charles unable to decrypt SSL traffic, even with a custom root certificate installed. So my only conclusion so far is that they haven't changed the endpoint URL.

Unfortunately they do the same in their Android app so I couldn't get it to work from that direction either.

Anyway - I have a half-okay method of having this work reliably which you can see here: https://github.com/etheralm/libpurecool/issues/37#issuecomment-777699816

It would appear the library will need to add a Content-Type Header of application/json as well as requiring you to re-authenticate the official app just before running it, however no one else has yet been able to confirm if what I'm doing works.

[strandlie]

@bfayers Great. I will test that tomorrow and report back. Nice work.

[jerryzou]

It seems that Dyson uses SSL Certificate Pinning in their iOS app (this is good, but a little impractical here), which makes Charles unable to decrypt SSL traffic, even with a custom root certificate installed. So my only conclusion so far is that they haven't changed the endpoint URL.

Unfortunately they do the same in their Android app so I couldn't get it to work from that direction either.

Anyway - I have a half-okay method of having this work reliably which you can see here: etheralm/libpurecool#37 (comment)

It would appear the library will need to add a Content-Type Header of application/json as well as requiring you to re-authenticate the official app just before running it, however no one else has yet been able to confirm if what I'm doing works.

It did work for me. Thanks for your effort.

[strandlie]

Can confirm it also works for me. Just make sure you re-authenticate the app right before making the POST-request.

[ZeGuigui]

Looks like you need to make an account verification before login: https://github.com/etheralm/libpurecool/issues/37#issuecomment-777949022 If confirmed libpurecool will probably get updated in no time

[mgaeremynck]

Following, same issue for my two fans.

Error: Logger: homeassistant.components.dyson Source: components/dyson/__init__.py:60 Integration: dyson (documentation, issues) First occurred: 8:41:16 (1 occurrences) Last logged: 8:41:16 Not connected to Dyson account. Unable to add devices

Log entries: 2021-02-12 08:41:16 ERROR (SyncWorker_1) [homeassistant.components.dyson] Not connected to Dyson account. Unable to add devices 2021-02-12 08:41:16 ERROR (MainThread) [homeassistant.setup] Setup failed for dyson: Integration failed to initialize.

All worked well in the past, I hope a solution is found. I learned a lot with this integration and the automation possibilities this provided.

[bfayers]

I assume you two, @ElMatthews and @kritin81 , where not actually just putting HTTPBasicAuth("account", "password") and instead replacing account and password with the values you can sometimes manage to get out of the Dyson API?

Anyway, it looks like a fix may have been found at this point so hopefully it won't be long before a new library release.

[qdel]

This login method modification seems to work for me:

   def __init__(self):
        [... existing code...]
        self._authNextTime = None

   def login(self):
        self._logged = False
        if self._authNextTime != None and self._authNextTime > datetime.datetime.now():
            print("Could not identify for the moment, next authent will be made at: %s" % (self._authNextTime))
        else:
            """Login to dyson web services."""
            urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
            print("Disabling insecure request warnings since "
                  "dyson are using a self signed certificate.")
            pre_login = requests.get(
                "https://{0}/v1/userregistration/userstatus?country={1}&email={2}".format(
                    self._dyson_api_url, self._country, self._email),
                headers=self._headers,
                verify=False
            )
            print("pre_login return code: %d, text: %s" % (pre_login.status_code, pre_login.text))
            if pre_login.status_code == requests.codes.ok:
                json_response = pre_login.json()
                if 'accountStatus' in json_response and json_response['accountStatus'] == 'ACTIVE':
                    request_body = {
                        "Email": self._email,
                        "Password": self._password
                    }
                    login = requests.post(
                        "https://{0}/v1/userregistration/authenticate?country={1}".format(
                            self._dyson_api_url, self._country),
                        headers=self._headers,
                        json=request_body,
                        verify=False
                    )
                    # pylint: disable=no-member
                    print("login return code: %d, text: %s" % (login.status_code, login.text))
                    if login.status_code == requests.codes.ok:
                        json_response = login.json()
                        print(json_response)
                        self._auth = HTTPBasicAuth(json_response["Account"],
                                                   json_response["Password"])
                        self._logged = True
                    elif 'Retry-After' in login.headers:
                        self._authNextTime = datetime.datetime.now() + datetime.timedelta(seconds=int(login.headers['Retry-After']))
        print("Return %d" % self._logged, flush=True)
        return self._logged

If i can, this week-end, i will make a proper PR. This code use print, this is not safe obviously

[kritin81]

Upgrade Homeassistant #https://github.com/etheralm/libpurecool/issues/37#issuecomment-778270175

[slyoldfox]

I've changed the code in https://github.com/etheralm/libpurecool/pull/38/files in my venv and can confirm it works after this change. ❤️ to everyone working on this!

[arifroni]

i am running a supervised os. i have portainer installed as addon. can anyone tell me how to get it working?

[romquenin]

i am running a supervised os. i have portainer installed as addon. can anyone tell me how to get it working?

Here are the steps to replace the dyson.py on Pi4 running Home Assistant OS:

• install Portainer Addon, disable protection mode, start addon and open web-ui.
• in Portainer settings remove core and homeassistant from the disabled devices
• now from the containr list select homeassistant, clic the console button and enter the three lines above one after the other:

cd /usr/local/lib/python3.8/site-packages/libpurecool/
rm dyson.py
curl -O https://raw.githubusercontent.com/bfayers/libpurecool/fix_auth/libpurecool/dyson.py

• Reboot core and Voilà !

Thanks to everyone involved in this integration !

[DreadN]

Thanks, replaced the dyson.py in my home assistant docker container on RPI4 and it returned online working.

[curt7000]

Trying to fix this. I'm running HA in a VENV, Python3.8, but cannot find the dyson.py file in /usr/local/lib/python3.8/site-packages/libpurecool. Anyone have an idea where the dyson.py file would be located?

[curt7000]

Trying to fix this. I'm running HA in a VENV, Python3.8, but cannot find the dyson.py file in /usr/local/lib/python3.8/site-packages/libpurecool. Anyone have an idea where the dyson.py file would be located?

Found it in /srv/homeassistant/lib/python3.8/site-packages/libpurecool/dyson.py , replaced it with https://raw.githubusercontent.com/bfayers/libpurecool/auth_customdeps/libpurecool/dyson.py

But it doesn't appear to be working. Any thoughts?

[bfayers]

For people in this thread - don't use the auth_customdeps branch - use the fix_auth one @romquenin @curt7000

[mgaeremynck]

I assume you two, @ElMatthews and @kritin81 , where not actually just putting HTTPBasicAuth("account", "password") and instead replacing account and password with the values you can sometimes manage to get out of the Dyson API?

Anyway, it looks like a fix may have been found at this point so hopefully it won't be long before a new library release.

I didn't modify any files (yet) and simply used the standard integration with the account and password in my secret file.

username: !secret dyson_username
password: !secret dyson_pwd
language: GB
devices:
  - device_id: ABC-EU-123456
    device_ip: 10.0.0.64
  - device_id: ABC-EU-123456
    device_ip: 10.0.0.65

Indeed, maybe better for me to wait until the new library is released.

[curt7000]

For people in this thread - don't use the auth_customdeps branch - use the fix_auth one @romquenin @curt7000

Up and running! Thank you! @bfayers

[googanhiem]

@mgaeremynck you might want to remove your serial numbers ("device_id").. as someone could mess with your dyson account with them.

[mgaeremynck]

@mgaeremynck you might want to remove your serial numbers ("device_id").. as someone could mess with your dyson account with them.

Thanks, done

[oneseventhree]

i am running a supervised os. i have portainer installed as addon. can anyone tell me how to get it working?

Here are the steps to replace the dyson.py on Pi4 running Home Assistant OS:

• install Portainer Addon, disable protection mode, start addon and open web-ui.
• in Portainer settings remove core and homeassistant from the disabled devices
• now from the containr list select homeassistant, clic the console button and enter the three lines above one after the other:

cd /usr/local/lib/python3.8/site-packages/libpurecool/
rm dyson.py
curl -O https://raw.githubusercontent.com/bfayers/libpurecool/auth_customdeps/libpurecool/dyson.py

• Reboot core and Voilà !

Thanks to everyone involved in this integration !

You are a life saver! Thank you!!!!!

[RonnyWinkler]

Will this fix be included in a coming HA version?

[bfayers]

Will this fix be included in a coming HA version?

For this to happen this will need to be merged: https://github.com/etheralm/libpurecool/pull/38

then @etheralm will need to push a new release to PyPi and HASS will need to be updated to reference the new version.

[DavidBurela]

That seems to make sense why we're all experiencing the issue since the HA update. The integration authenticates once on HA startup, and so is able to communicate locally. We have all been running out HA for days/weeks, and have had that working credential. Then as we all upgraded and HA rebooted, we're all now hitting the existing Auth issue as it tries to auth on startup.

[M1sterGlass]

i am running a supervised os. i have portainer installed as addon. can anyone tell me how to get it working?

Here are the steps to replace the dyson.py on Pi4 running Home Assistant OS:

* install Portainer Addon, disable protection mode, start addon and open web-ui.

* in Portainer settings remove core and homeassistant from the disabled devices

* now from the containr list select homeassistant, clic the console button and
  enter the three lines above one after the other:

cd /usr/local/lib/python3.8/site-packages/libpurecool/
rm dyson.py
curl -O https://raw.githubusercontent.com/bfayers/libpurecool/fix_auth/libpurecool/dyson.py

* Reboot core and Voilà !

Thanks to everyone involved in this integration !

Running HA 2021.2.3 image in Hyper-V. This fixed it for me, thanks!

[kintaroju]

Hi,

This might sound like a super noob question, but with HassOS, how you do the above?

Thanks!

[bfayers]

Hi,

This might sound like a super noob question, but with HassOS, how you do the above?

Thanks!

connect to the console for it and run docker exec -t homeassistant pip3 install --upgrade git+https://github.com/bfayers/libpurecool.git@fix_auth Then restart hass.

You will need to do this after every HASS update until a new library version can be released and set in hass.

[kintaroju]

@bfayers , that worked like a charm, after restarting HassOS, it now sees all 4 of my Dyson Purifiers now. Hopefully they will integrate this fix in the near future.

[lolongan]

Is it possible to implement this fix remotely using Nabucasa or should I do that locally ?

[ghost]

May I know where is the dyson.py location if I am using HA Supervised? I tried to search this file inside my Debian but I can’t locate it, thank you