search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.68k stars 30.81k forks source link

Certificate Expiry integration does not handle self signed or personal cert authorities #91170

Open ferg22 opened 1 year ago

ferg22 commented 1 year ago

The problem

I have my own CA for my own certificates.

  1. Certificate Expiry integration reports the cert as invalid as it cannot validate it against my root certs. However the cert is perfectly valid, it is just the check logic cannot locate the correct root cert. In every other similar case(outside of HA) there is an option to provide or point to additional root certs. There is no option to add a local root cert in HA.
  2. Even if the cert chain cannot be validated, this integration is not doing what it says on the tin. The expiry dates should still be able to be checked even if the cert chain cannot be validated.

Please fix.

I do sysadmin + coding for a living, some of which involves cert administration. So I thought I should be able to look at the code and fix this myself. Having done so I havnt a clue how this thing works or hangs together. Is that good?

What version of Home Assistant Core has the issue?

core-2023.4.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Certificate Expiry

Link to integration documentation on our website

https://www.home-assistant.io/integrations/cert_expiry/

Diagnostics information

None, checking a perfectly valid cert produces the following in HA "Is valid false, Error: unable to get local issuer certificate."

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @cereal2nd, @jjlawren, mind taking a look at this issue as it has been labeled with an integration (cert_expiry) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `cert_expiry` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign cert_expiry` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)

cert_expiry documentation cert_expiry source (message by IssueLinks)

nekodojo commented 1 year ago

I have the same issue. Do I need to add my root cert as trusted, somewhere in the homeassistant server? An option to add root certs in the integration would be great... if not, I could solve it myself if I knew which library is being used and how to configure trust.

Thanks

nekodojo commented 1 year ago

I was able to work around this by adding the cert directly in the homeassistant server (in my case a homeassistant docker container).

Create /etc/ssl/certs/ca-cert-myspecial-ca.pem with your trusted root plus any intermediates (worked for me with root only) Cat the file you just added onto the end of /etc/ssl/cert.pem (in my case this was a symlink to /etc/ssl/certs/ca-certificates.crt but cat worked anyway) cat >> /etc/ssl/cert.pem /etc/ssl/certs/ca-cert-myspecial-ca.pem

If using docker you will need to build the /etc/ssl/cert.pem ahead of time and inject it into the container, overwriting /etc/ssl/cert.pem or /etc/ssl/certs/ca-certificates.crt if symlink is present

The following was helpful to me to see what Python ssl lib is currently doing for CA python -c "import ssl; print(ssl.get_default_verify_paths())"

Some linux installations have a process that builds a new /etc/ssl/certs/ca-certificates.crt when OS packages get updated. If your change to the master ca certs file gets reverted, look for a tool in your OS to mark specific ones as trusted.

Above text is probably way too long to include in documentation but someone may be able to condense it down.

Good luck

ferg22 commented 1 year ago

Thanks for the help. However....

I used ssh to connect to my HA core and added my root CA pem file to /etc/ssl/cert.pem (actually added to /etc/ssl/certs/ca-certificates.crt ) It did not seem to affect the

I could not get 'python -c "import ssl; print(ssl.get_default_verify_paths())"' to work. I got '-bash: python: command not found'. So where is python installed?

ckuethe commented 1 year ago

One thing you can do when using docker is to bind mount your additional CA cert into /etc/ssl/certs so that all the certificate verifiers can see it, eg.

-v /docker/homeassistant/ssl/my_ca.crt:/etc/ssl/certs/my_ca.crt:ro 
-v /docker/homeassistant/ssl/my_ca.crt:/etc/ssl/certs/12345678.0:ro 
-v /docker/homeassistant/ssl/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro 
-v /docker/homeassistant/ssl/ca-certificates.crt:/etc/ssl/certs/cert.pem:ro

where

It feels kinda dirty, but it works.

That said, I kind of think that CA verification should be optional given the description of this integration: "... fetches the certificate from a configured host and displays its expiration in a timestamp sensor." It says nothing about checking whether or not the certificate has been signed by a known CA.

GrumpyMeow commented 1 year ago

I have the same issue about that "local... not trusted". My setup is not Docker-based. I have an official certificate for my domain. With this certificate I enounctered the same error using other online services, which complained about my certificate. I read online that these kinds of problems are commonly caused by using certificates which didn't contain the full chain. I never got around to looking into this.

My plans are two fold:

Hopefully i get around to this soon..

ferg22 commented 1 year ago

Chris,

Thanks for this response and sorry about the slow reply. This sounds exactly what I need, but I cant follow your directions.

I am using a raspberry pi based HA install, with about 10 containers.

Having snuck around I can find no equiv of the /docker/homeassistant you mention. Are you (or somebody) able to suggest where the "/config docker volume” is on a rasp pi. By the way I am logged into the hypervisor using "ssh -p22222 @.***"

On 8 Jul 2023, at 02:02, Chris Kuethe @.***> wrote:

One thing you can do when using docker is to bind mount your additional CA cert into /etc/ssl/certs so that all the certificate verifiers can see it, eg.

-v /docker/homeassistant/ssl/my_ca.crt:/etc/ssl/certs/my_ca.crt:ro -v /docker/homeassistant/ssl/my_ca.crt:/etc/ssl/certs/12345678.0:ro -v /docker/homeassistant/ssl/ca-certificates.crt:/etc/ssl/certs/ca-certificates.crt:ro -v /docker/homeassistant/ssl/ca-certificates.crt:/etc/ssl/certs/cert.pem:ro where

/docker/homeassistant is wherever you store the /config docker volume for homeassistant /docker/homeassistant/ssl is where the custom certificates will be stored. The location is chosen to indicate that these ssl additions are for the benefit of the docker container, and not the host my_ca.crt is, of course, your custom CA certificate 12345678.0 is the hash name of your custom CA cert from openssl rehash ca-certificates.crt is the concatenation of /etc/ssl/certs/ca-certificates.crtand your custom CA cert. It feels kinda dirty, but it works.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/91170#issuecomment-1626401077, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOIEBHEBOM5QF2NOYIRMJTXPCWUHANCNFSM6AAAAAAWZBAD54. You are receiving this because you authored the thread.

============================================================ Fergus

ckuethe commented 1 year ago

I use /docker/ as shorthand for wherever it is that you store your persistent docker data. Maybe you just create volumes under /var/lib/docker/volumes. Maybe you create directories under /var/lib/docker-volumes/. Maybe you have a whole separate SSD/NAS/SAN mounted at /mnt/docker...

docker inspect will tell you many things about a given container, including where the volumes are. Look for /config in the output.

As an aside, https://github.com/portainer/portainer is a nice gui way to manage your containers.

ferg22 commented 1 year ago

Thanks again Chris for the prompt reply.

My HA was just an image I downloaded and flashed to my PI. I think they call it the "Home assistant operating system" variant. It contained an OS, Supervisor and Core all within the image. Logging into the hypervisor and looking around I saw a '/mnt/data/docker’ hier which contained several Dockerfiles for add ons. But I could find a Dockerfile for HA core itself and there is no docker-compose anywhere, I sniffed around the contents of the /etc and /mnt/supvervisor and count not see how the docker containers where launched at all

Using inspect on the container built from image ghcr.io/home-assistant/raspberrypi4-64-homeassistant:2023.7.3 I do see lots of mounts But I dont have a docker file for it, though I did manage to find https://github.com/home-assistant/core/blob/dev/Dockerfile but it has no mounts.

Using docker inspect I See the following mounts.

"Mounts": [
{ "Type": "bind", "Source": "/dev", "Target": "/dev", "ReadOnly": true },
{ "Type": "bind", "Source": "/run/dbus", "Target": "/run/dbus", "ReadOnly": true },
{ "Type": "bind", "Source": "/run/udev", "Target": "/run/udev", "ReadOnly": true },
{ "Type": "bind", "Source": "/mnt/data/supervisor/homeassistant", "Target": "/config" },
{ "Type": "bind", "Source": "/mnt/data/supervisor/ssl", "Target": "/ssl", "ReadOnly": true },
{ "Type": "bind", "Source": "/mnt/data/supervisor/share", "Target": "/share", "BindOptions": { "Propagation": "rslave" } },
{ "Type": "bind", "Source": "/mnt/data/supervisor/media", "Target": "/media", "BindOptions": { "Propagation": "rslave" } },
{ "Type": "bind", "Source": "/mnt/data/supervisor/tmp/homeassistant_pulse", "Target": "/etc/pulse/client.conf", "ReadOnly": true },
{ "Type": "bind", "Source": "/mnt/data/supervisor/audio/external", "Target": "/run/audio", "ReadOnly": true },
{ "Type": "bind", "Source": "/mnt/data/supervisor/audio/asound", "Target": "/etc/asound.conf", "ReadOnly": true },
{ "Type": "bind", "Source": "/etc/machine-id", "Target": "/etc/machine-id", "ReadOnly": true }
],

I feel that the supervisor much be launching the container somewhere but I can t see where.

Hopefully yourself or someone else can explain how I can add to the home assistant mounts.

On 27 Jul 2023, at 01:42, Chris Kuethe @.***> wrote: I use /docker/ as shorthand for wherever it is that you store your persistent docker data. Maybe you just create volumes under /var/lib/docker/volumes. Maybe you create directories under /var/lib/docker-volumes/. Maybe you have a whole separate SSD/NAS/SAN mounted at /mnt/docker...

docker inspect will tell you many things about a given container, including where the volumes are. Look for /config in the output.

As an aside, https://github.com/portainer/portainer https://github.com/portainer/portainer is a nice gui way to manage your containers.

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/91170#issuecomment-1652741760, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAOIEBCCWDWLYX5MFEBAGXDXSG2OVANCNFSM6AAAAAAWZBAD54. You are receiving this because you authored the thread.

============================================================ Fergus

issue-triage-workflows[bot] commented 1 year ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

aebgit commented 1 year ago

This issue is still relevant.

nekodojo commented 1 year ago

I still have the issue and I had to apply the workaround given by ckuethe Jul 7 https://github.com/home-assistant/core/issues/91170#issuecomment-1626401077

This is basically overriding the set of root certs that come with homeassistant docker container and replacing the files with master root CA certs from your linux host. So, tricking homeassistant to see your in-house cert as part of the world-wide CA ecosystem.

Still I agree with ckuethe:

That said, I kind of think that CA verification should be optional given the description of this integration: "... fetches the certificate from a configured host and displays its expiration in a timestamp sensor." It says nothing about checking whether or not the certificate has been signed by a known CA.

Hopefully the PR will get some review and get added in a future HA: https://github.com/home-assistant/core/pull/97242

If this cannot be solved with a code change then I would at least suggest updating the documentation to let people know that custom self-signed or in-house company PKI is not supported, but that they can work around this by overriding the master CA certs of their Home Assistant system (or in docker if homeassistant docker is used)

ferg22 commented 1 year ago

A recently added HACS "Additional CA for Home Assistant" does address this. However the whole thing is still broken- for me at least. See https://github.com/Athozs/hass-additional-ca for details.

I think the fix is working, however the "Certificate Expiry" now fails to run giving a message "certificate key too weak". I think the lib the code is based on has all its knobs turned upto max. This plugin should do what it says on the tin, it must report cert expiry, other isses such a unknown root or weak security can be mentioned in logs perhaps. But they should not affect the core function.

I came across the following ... in case it helps. https://superuser.com/questions/1640089/ssl-certificate-ee-certificate-key-too-weak

By default, Debian has configured OpenSSL at security level 2, which provides 112 bits of security. That means that if one of the keys involved in the TLS connection, in this case the server's key (the end-entity certificate), provides a security level less than 112 bits (usually because the certificate is an RSA key smaller than 2048 bits), then it will be rejected.

Since a 112 bit security level is even below the recommended 128-bit minimum these days and this server is below even that, the best thing to do is to contact the server administrator and ask them to generate a new TLS certificate. With such an insecure certificate, a major corporation or a government could probably crack the key with some effort, and consequently spoof the connection.

If you can't do that, you can lower the security level by using curl --ciphers DEFAULT@SECLEVEL=1. Note that by doing this, you're essentially accepting that your connection is not completely secure and is subject to tampering.

PS> I am fixing my Cert chain :-)

ferg22 commented 11 months ago

Having added "Additional CA for Home Assistant" HACS I see no improvment at all. I used the debug logging to obtain the following

2023-11-24 10:40:34.894 WARNING (SyncWorker_1) [homeassistant.loader] We found a custom integration hacs which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2023-11-24 10:41:08.106 ERROR (SyncWorker_1) [homeassistant.components.onewire.onewire_entities] Error fetching HIH5030 humidity data: [Errno 1] Startup - command line parameters invalid: '/26.7E857D000000/HIH5030/humidity'
2023-11-24 10:41:10.420 ERROR (SyncWorker_7) [homeassistant.components.onewire.onewire_entities] Error fetching HIH5030 humidity data: [Errno 1] Startup - command line parameters invalid: '/26.F3867D000000/HIH5030/humidity'
2023-11-24 10:41:11.433 ERROR (SyncWorker_7) [homeassistant.components.onewire.onewire_entities] Error fetching HIH5030 humidity data: [Errno 1] Startup - command line parameters invalid: '/26.CB7A25020000/HIH5030/humidity'
2023-11-24 10:41:15.480 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform onewire is taking over 10 seconds.
2023-11-24 10:41:15.481 WARNING (MainThread) [homeassistant.components.sensor] Setup of sensor platform onewire is taking over 10 seconds.
2023-11-24 10:41:17.502 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [547599877952] Fergus from 192.168.47.89 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36): Client unable to keep up with pending messages. Reached 4096 pending messages. The system's load is too high or an integration is misbehaving; Last message was: {"type":"event","event":{"a":{"device_tracker.41_e5_a4_5b_ae_c1":{"s":"not_home","a":{"source_type":null,"friendly_name":"41-E5-A4-5B-AE-C1"},"c":"01HG0EPTNY8E6PDQV3XTRN00X4","lc":1700822477.502233}}},"id":2}
2023-11-24 10:41:17.502 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [547410023488] Fergus from 192.168.47.89 (Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36): Client unable to keep up with pending messages. Reached 4096 pending messages. The system's load is too high or an integration is misbehaving; Last message was: {"type":"event","event":{"a":{"device_tracker.41_e5_a4_5b_ae_c1":{"s":"not_home","a":{"source_type":null,"friendly_name":"41-E5-A4-5B-AE-C1"},"c":"01HG0EPTNY8E6PDQV3XTRN00X4","lc":1700822477.502233}}},"id":40}
2023-11-24 10:41:21.773 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [CA certificate key too weak]
2023-11-24 10:41:23.409 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [CA certificate key too weak]
2023-11-24 10:41:33.055 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 332, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 557, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message:
  Bad status line "Invalid method encountered:\n\n  b'\\x16\\x03\\x01\\x02'\n    ^"
2023-11-24 10:42:18.849 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):

  Bad status line "Invalid method encountered:\n\n  b'\\x16\\x03\\x01\\x02'\n    ^"
2023-11-25 09:30:40.982 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 332, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 557, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message:
  Bad status line "Invalid method encountered:\n\n  b'\\x16\\x03\\x01\\x02'\n    ^"
2023-11-25 09:30:53.034 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 332, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 557, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message:
  Bad status line "Invalid method encountered:\n\n  b'\\x16\\x03\\x01\\x02'\n    ^"
2023-11-25 09:30:56.103 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiohttp/web_protocol.py", line 332, in data_received
    messages, upgraded, tail = self._request_parser.feed_data(data)
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "aiohttp/_http_parser.pyx", line 557, in aiohttp._http_parser.HttpParser.feed_data
aiohttp.http_exceptions.BadStatusLine: 400, message:
  Bad status line "Invalid method encountered:\n\n  b'\\x16\\x03\\x01\\x02'\n    ^"
2023-11-25 10:27:22.725 ERROR (MainThread) [custom_components.tapo.coordinators] Error fetching tapo data: Unexpected exception:
2023-11-25 10:41:22.729 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [self-signed certificate in certificate chain]
2023-11-25 11:35:47.474 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [self-signed certificate in certificate chain]
2023-11-25 11:35:47.516 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [self-signed certificate in certificate chain]
2023-11-25 11:44:48.836 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [self-signed certificate in certificate chain]
2023-11-25 11:44:48.837 DEBUG (MainThread) [homeassistant.components.cert_expiry.coordinator] Finished fetching isobel.twig.me.uk:993 data in 0.031 seconds (success: True)
2023-11-25 11:44:48.873 ERROR (MainThread) [homeassistant.components.cert_expiry.coordinator] Certificate validation error: isobel.twig.me.uk [self-signed certificate in certificate chain]
2023-11-25 11:44:48.874 DEBUG (MainThread) [homeassistant.components.cert_expiry.coordinator] Finished fetching isobel.twig.me.uk:993 data in 0.035 seconds (success: True)

The cert that is being checked is installed on my dovecot imap server and works fine; even apple devices are happy with the server once I had added my root cert to apples profile.

issue-triage-workflows[bot] commented 8 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

cromefire commented 8 months ago

Screenshot_20240223-132130.png

Still doesn't work. (HA 2024.2.2)

ferg22 commented 8 months ago

As a followup. It is all working fine for me now.

As mentioned above the trick was to install and configure a recently added HACS "Additional CA for Home Assistant". I also had to "upgrade" my CA with stronger root keys. That bit was a real pain. While perhaps unnessicary - I have ended up with a new much stronger CA.

issue-triage-workflows[bot] commented 5 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

cromefire commented 5 months ago

Still an issue.

ferg22 commented 5 months ago

The "Certificate Expiry integration" prerequisite "Additional CA" is broken due to some issue in the HA core!

issue-triage-workflows[bot] commented 2 months ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

cromefire commented 2 months ago

Still an issue

oracleguy commented 1 month ago

It looks like the PR was abandoned before it could get properly reviewed and merged in. I agree with others that the changes in the PR seem to be correct route: This addon shouldn't care about valid CAs or key strength, it should simply tell you when the certificate is going to expire.