SSL Certificate Validation Failure talking to API

[MeeperBeeper]

Hey Folks,

I have HAOS installed under Virtualbox and one of the integrations is to my BlueAir 480i. Until about a few hours ago, it's been great.

I just recently (as in a few hours ago) updated from 2023.2.1 to 2023.2.2. About the same time I noticed that my sensor information wasn't updating with new detail (happens occasionally and I need to restart the device - meh) but I believe the updates stopped coming in from before the update to HAOS.

After restarting both HAOS and the 480i, the integration didn't continue like it normally does. Instead, I got an error about the entity being unavailable. I've since tried to remove and re-add the device but I'm getting the below error showing an SSL Verification error when trying to add my device back and a similar one appears from before I removed the device:

This error originated from a custom integration.

Logger: homeassistant.config_entries
Source: custom_components/blueair/blueair/blueair.py:81
Integration: BlueAir (documentation, issues)
First occurred: 10:38:43 AM (2 occurrences)
Last logged: 12:12:04 PM

Error setting up entry BlueAir [redacted] for blueair
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.10/site-packages/urllib3/connection.py", line 414, in connect
    self.sock = ssl_wrap_socket(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
  File "/usr/local/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/local/lib/python3.10/ssl.py", line 1071, in _create
    self.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 1342, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/usr/local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/usr/local/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.blueair.io', port=443): Max retries exceeded with url: /v2/user/[redacted]/homehost/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 382, in async_setup
    result = await component.async_setup_entry(hass, self)
  File "/config/custom_components/blueair/__init__.py", line 28, in async_setup_entry
    client = await hass.async_add_executor_job(
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/config/custom_components/blueair/__init__.py", line 29, in <lambda>
    lambda: blueair.BlueAir(
  File "/config/custom_components/blueair/blueair/blueair.py", line 66, in __init__
    self.home_host = self.get_home_host()
  File "/config/custom_components/blueair/blueair/blueair.py", line 81, in get_home_host
    response = requests.get(
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 563, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='api.blueair.io', port=443): Max retries exceeded with url: /v2/user/[redacted]/homehost/ (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:997)')))

Something worthy of note is that if you access the URL yourself ( https://api.blueair.io/v2/user/[redacted]/homehost/ ) and look at the SSL certificate, it looks like it was created four days ago. Is this addon doing certificate pinning? If so it might need an update. Otherwise, I'm open to suggestions to get HA to talk to the BlueAir API again.

Meep

[lensherm]

Confirming, same issue here

[hufman]

I'm particularly bad about updating my Home Assistant, and I confirm seeing this issue in my logs under HaOS 8.5 and Home Assistant Core 2022.11.4 :sweat_smile: Looking at openssl s_client -showcerts -connect api-us-east-1.blueair.io:443, it looks like they put the leaf cert twice and didn't put an intermediate cert:

openssl s_client -showcerts -connect api-us-east-1.blueair.io:443
CONNECTED(00000003)
depth=0 C = SE, L = Stockholm, O = Blueair AB, CN = *.blueair.io
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = SE, L = Stockholm, O = Blueair AB, CN = *.blueair.io
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 C = SE, L = Stockholm, O = Blueair AB, CN = *.blueair.io
verify return:1
---
Certificate chain
 0 s:C = SE, L = Stockholm, O = Blueair AB, CN = *.blueair.io
   i:C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
-----BEGIN CERTIFICATE-----
MIIGnzCCBYegAwIBAgIQCjj6jS4iP5VWaKPRQ4MpvDANBgkqhkiG9w0BAQsFADBP
...
bqYRxCHfthAT3oxAJS9we041acvgRbjt/eDkX2tRQkrn3WgCEL7I0ktrbMs52vdW
c50/L4nJZSlJk/PCOtwms3tqEQ==
-----END CERTIFICATE-----
 1 s:C = SE, L = Stockholm, O = Blueair AB, CN = *.blueair.io
   i:C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1
-----BEGIN CERTIFICATE-----
MIIGnzCCBYegAwIBAgIQCjj6jS4iP5VWaKPRQ4MpvDANBgkqhkiG9w0BAQsFADBP
...
bqYRxCHfthAT3oxAJS9we041acvgRbjt/eDkX2tRQkrn3WgCEL7I0ktrbMs52vdW
c50/L4nJZSlJk/PCOtwms3tqEQ==
-----END CERTIFICATE-----
---
Server certificate
subject=C = SE, L = Stockholm, O = Blueair AB, CN = *.blueair.io

issuer=C = US, O = DigiCert Inc, CN = DigiCert TLS RSA SHA256 2020 CA1

[MeeperBeeper]

@hufman, you know, once I saw that Firefox worked I didn't think to check the certificate further. Using one of those online SSL Certificate checkers confirms what you have above.

I opened up their app on my tablet and low and behold, it's very much not working. Can someone else confirm their app also isn't working in case it's just me.

I wonder if they'll notice or if someone's going to have to tell them...

[otw]

I just noticed the same issue for my own custom integration, checked the certificate and confirmed findings above. I can also confirm that the app doesn't work either (Android + iOS). So hopefully they'll realize this sooner rather than later :-|

[lensherm]

I got the original app working by signing out of it and signing back in and then cycling the purifier.

[MeeperBeeper]

Confirmed @lensherm's findings in that logging out of the app (multiple times in my case) and back in while also power cycling the purifier worked.

Blast... Now I have more questions. Mainly if they even do proper certificate checking in their products. I'm not sure I want to know.

[MeeperBeeper]

I've sent an email to their info@blueair.com address listed on their site telling them of the issue (without pointing to this ticket, repo, or that we do this at all). Let's hope they're willing to actually fix the issue and don't just ban me from their service.

[lensherm]

Thanks for the legwork @MeeperBeeper

[otw]

@lensherm Unfortunately logout/cycling only works partially for me. I have 8 x80i total and I'm only able to get 1 or 2 online at a time using this method.

@MeeperBeeper Thanks for reaching out to them!

[spikeygg]

bah, I was using a separately downloaded custom_component version of this add-on from way back. Now that it is available in HACS and I'm also experiencing this problem, I decided I'd try to revive it by reinstalling the works. So, I removed the integration then deleted the custom_component directory, restarted HA, reinstalled the HACS version of the add-on, restarted HA and then tried to reinstall the integration but it won't accept my login information. :(

I logged out of the app on my phone and logged back in and it accepted my credentials (same values).

[chrisdfw]

They updated the cert did not include the intermediate cert in the chain. I open a ticket also with the information below. It looks like its now fixed

https://www.ssllabs.com/ssltest/analyze.html?d=api.blueair.io&s=52.5.90.17&hideResults=on&latest

[hufman]

My openssl command confirms that the intermediate is in the chain, and the HASS integration works again without errors in the logs :)

[MeeperBeeper]

While they haven't (yet) responded to my email, I can confirm that the SSL certificate appears to have been fixed and that my own integrations are now working again.

Thanks everyone for the help and confirmations. Glad it wasn't anything on our end.

[lensherm]

Thank you all!