search

Arksine / moonraker

Web API Server for Klipper
https://moonraker.readthedocs.io
GNU General Public License v3.0
1.02k stars 392 forks source link

Home Assistant self-signed certificate support for integration with power component #799

Closed paulo-serrao closed 5 months ago

paulo-serrao commented 5 months ago

The current version of moonraker has integration with Home Assistant allowing to control remote switches. There is the option to set the authentication protocol to HTTPS, but there was no option to define a certificate (.crt) to be used. This is required when using self-signed certificates, otherwise validation will fail and moonraker.log will log following error:

[iostream.py:_do_ssl_handshake()] - SSL Error on 12 ('192.168.1.X', 8123): [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)

This merge request allows a new variable called ca_certs (kept the same variable name used by the tornado httpclient library) to be defined inside the [power homeassistant_switch] config section, any request made to homeassistant will then use the certificate and succeed. 

[power homeassistant_switch]
type: homeassistant
address: homeassistant.local
protocol: https
port: 8123
device: switch.1234567890abcdefghij
token: home-assistant-very-long-token
domain: switch
ca_certs: /home/pi/homeassistant.crt

The ca_certs variable is optional and moonraker will behave the same if not defined. This variable should be used together with setting the protocol to https (instead of the default http).

PS: I tried appending the self-signed certificate directly into the system (wget starts working) and into python venv thru certifi (python requests lib starts working), but I could not make the tornado httpclient to work other than passing the certificate on the code.

Signed-off-by: Paulo Serrão paulo.serrao@gmail.com

Arksine commented 5 months ago

Thanks. I wonder if its possible for you to install your own root certificate and have it work with no modifications. I'm not sure if the above procedure is what you meant by "appending the certificate directly."

If the above does not work we could look into merging. We would need to change the ca_certs option to take a file name relative to the {printer_data}/certs path. The file could be a symbolic link, all that is important is that it exists in the correct location.

paulo-serrao commented 5 months ago

@Arksine As I mentioned on the PS, I tried adding the self-signed certificate to the system exactly as described on that page (sudo cp my.crt /usr/local/share/ca-certificates/ ; sudo update-ca-certificates). wget stops complaining and working just fine, but thats not the case for moonraker (even after process restart). I did not try a system restart.

After that if I launch a moonraker venv and import python request, it also keeps complaining. I could work around that by appending my self-signed certificate to venv python-certifi package as described on the first answer here https://stackoverflow.com/questions/34931378/certificate-verification-when-using-virtual-environments (which is already too much of a hack for my taste, since it would break on python-certifi update or venv rebuild), it starts working with the requests lib, but moonraker doesn't use that.

With tornado, I could not figure out a way to hack this, I don't know which CA bundle it uses so I can append my self-signed there...

pi@octopi:~ $ wget https://homeassistant.local:8123
--2024-02-01 17:35:37--  https://homeassistant.local:8123/
Resolving homeassistant.local (homeassistant.local)... 192.168.1.110
Connecting to homeassistant.local (homeassistant.local)|192.168.1.110|:8123... connected.
ERROR: The certificate of ‘homeassistant.local’ is not trusted.
ERROR: The certificate of ‘homeassistant.local’ doesn't have a known issuer.
pi@octopi:~ $ sudo cp homeassistant_rootca.crt /usr/local/share/ca-certificates/
pi@octopi:~ $ sudo update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
pi@octopi:~ $ wget https://homeassistant.local:8123 
--2024-02-01 17:37:33--  https://homeassistant.local:8123/
Resolving homeassistant.local (homeassistant.local)... 192.168.1.165
Connecting to homeassistant.local (homeassistant.local)|192.168.1.165|:8123... connected.
HTTP request sent, awaiting response... 200 OK
Length: 4148 (4.1K) [text/html]
Saving to: ‘index.html’

index.html                                             100%[============================================================================================================================>]   4.05K  --.-KB/s    in 0.001s  

2024-02-01 17:37:33 (2.71 MB/s) - ‘index.html’ saved [4148/4148]
pi@octopi:~ $ /home/pi/moonraker-env/bin/python
Python 3.9.2 (default, Mar 12 2021, 04:06:34) 
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tornado
>>> request = tornado.httpclient.HTTPRequest(url = 'https://homeassistant.local:8123', method='GET')
>>> print(tornado.httpclient.HTTPClient().fetch(request))
Uncaught exception, closing connection.
Traceback (most recent call last):
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 691, in _handle_events
    self._handle_read()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1427, in _handle_read
    self._do_ssl_handshake()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1367, in _do_ssl_handshake
    self.socket.do_handshake()
  File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1123)
Exception in callback None()
handle: <Handle cancelled>
Traceback (most recent call last):
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 202, in _handle_events
    handler_func(fileobj, events)
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 691, in _handle_events
    self._handle_read()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1427, in _handle_read
    self._do_ssl_handshake()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1367, in _do_ssl_handshake
    self.socket.do_handshake()
  File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1123)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/httpclient.py", line 134, in fetch
    response = self._io_loop.run_sync(
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/ioloop.py", line 539, in run_sync
    return future_cell[0].result()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/simple_httpclient.py", line 340, in run
    stream = await self.tcp_client.connect(
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/tcpclient.py", line 292, in connect
    stream = await stream.start_tls(
  File "/usr/lib/python3.9/asyncio/events.py", line 80, in _run
    self._context.run(self._callback, *self._args)
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/platform/asyncio.py", line 202, in _handle_events
    handler_func(fileobj, events)
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 691, in _handle_events
    self._handle_read()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1427, in _handle_read
    self._do_ssl_handshake()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1367, in _do_ssl_handshake
    self.socket.do_handshake()
  File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:1123)

I am open to do more testing if desired.

Regarding the ca_certs path, isn't the {printer_data}/certs only used for the certs for moonraker to sign his pages? thats what I understood after reading the docs after trying to put my HA self-signed public key there in the hope magic could happen by itself, but I agree we can add it there and on the config you just specify the filename instead of the whole path.

Arksine commented 5 months ago

Interesting. There seems to be something strange about your system, as I can't reproduce. On Mainsailos (Debian Bullseye):

pi@mainsailos:~ $ wget https://pi-server.home:8123
--2024-02-02 05:38:40--  https://pi-server.home:8123/
Resolving pi-server.home (pi-server.home)... 10.0.0.11
Connecting to pi-server.home (pi-server.home)|10.0.0.11|:8123... connected.
ERROR: The certificate of ‘pi-server.home’ is not trusted.
ERROR: The certificate of ‘pi-server.home’ doesn't have a known issuer.
pi@mainsailos:~ $ ~/moonraker-env/bin/python -c 'from tornado.httpclient import HTTPClient; print(HTTPClient().fetch("https://pi-server.home:8123"))'
SSL Error on 6 ('10.0.0.11', 8123): [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/httpclient.py", line 134, in fetch
    response = self._io_loop.run_sync(
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/ioloop.py", line 527, in run_sync
    return future_cell[0].result()
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/simple_httpclient.py", line 340, in run
    stream = await self.tcp_client.connect(
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/tcpclient.py", line 292, in connect
    stream = await stream.start_tls(
  File "/home/pi/moonraker-env/lib/python3.9/site-packages/tornado/iostream.py", line 1367, in _do_ssl_handshake
    self.socket.do_handshake()
  File "/usr/lib/python3.9/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)
pi@mainsailos:~ $ sudo cp pi-server-ca.pem /usr/local/share/ca-certificates/extra/pi-server-ca.crt
pi@mainsailos:~ $ sudo update-ca-certificates
Updating certificates in /etc/ssl/certs...
1 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
pi@mainsailos:~ $ wget https://pi-server.home:8123
--2024-02-02 05:39:42--  https://pi-server.home:8123/
Resolving pi-server.home (pi-server.home)... 10.0.0.11
Connecting to pi-server.home (pi-server.home)|10.0.0.11|:8123... connected.
HTTP request sent, awaiting response... 200 OK
Length: 9294 (9.1K) [text/html]
Saving to: ‘index.html’

index.html                    100%[=================================================>]   9.08K  --.-KB/s    in 0.004s

2024-02-02 05:39:42 (2.15 MB/s) - ‘index.html’ saved [9294/9294]

pi@mainsailos:~ $ ~/moonraker-env/bin/python -c 'from tornado.httpclient import HTTPClient; print(HTTPClient().fetch("https://pi-server.home:8123"))'
HTTPResponse(_body=None,_error_is_response_code=False,buffer=<_io.BytesIO object at 0x7fb8429f40>,code=200,effective_url='https://pi-server.home:8123',error=None,headers=<tornado.httputil.HTTPHeaders object at 0x7fb8242eb0>,reason='OK',request=<tornado.httpclient.HTTPRequest object at 0x7fb82ed340>,request_time=0.12515902519226074,start_time=1706870390.0247965,time_info={})

It also works on RPOS Bookworm for me. One thing I notice is that your exception is unhandled. It isn't providing the certificate verify failed: unable to get local issuer certificate message.

I think the best path forward is to attempt to determine why the handshake is failing on your system. It appears as if Python is picking up another cert for homeassistant.local that is invalid.

paulo-serrao commented 5 months ago

@Arksine I did a bit more testing and you were right, there was a problem on my side, not with the system (2 weeks old Raspberry PI OS + KIAUH install) but with the certificate itself. I had previously manipulated the cert in order to be able to import it into my windows machine as well as my android phone and ended up copying to my raspberry not the original rootCA.pem but a converted version that had "Bag Attributes" prepended to the cert. Apparently that is enough to cause this behavior on my Rasp.

I am now able to use the current moonraker code without requiring any further modification 👍

I think it is reasonable to assume who is using moonraker will have root access and be able to add the rootCA directly into the system, so perhaps this can be closed, since this merge request would only make sense if that was not the case or if someone is not using the certificate in the proper format 😇

Do you agree?

Arksine commented 5 months ago

Yes, I agree. From a security perspective it is preferable to require root access in order add a ca root certificate. If in the future there is some compelling reason to add support for the ca_certs option we can revisit it, but I don't see the need for it now.

paulo-serrao commented 5 months ago

Closing as this should not be required. Instead it is recommended to add the ca root certificate directly into the OS.