xannor / ha_reolink_rest

ReoLink REST/Web Camera Integration for Home Assistant
24 stars 3 forks source link

SSL error when trying to add device #13

Closed quentindavid closed 1 year ago

quentindavid commented 2 years ago

I installed the two integrations from HACS (Reolink Discovery and Reolink IP Device), and it discover correctly two cameras. For one (DUO) it seems to work fine, but for the other (RLC-510A), when I click on "Configure", it ask for Host and Port. If I enter the IP of the camera, port 443 and I check https, then I have "Unknown error occurred". Here is the error I can find:

Logger: aiohttp.server
Source: custom_components/reolink_rest/config_flow.py:317
Integration: Reolink IP Device ([documentation](https://github.com/xannor/ha_reolink_rest), [issues](https://github.com/xannor/ha_reolink_rest/issues))
First occurred: 11:00:53 (1 occurrences)
Last logged: 11:00:53

Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 435, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 222, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 79, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 236, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 136, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 177, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 62, in wrapper
    result = await method(view, request, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 109, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 277, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 359, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/config/custom_components/reolink_rest/config_flow.py", line 317, in async_step_connection
    self.data.update(user_input)
ValueError: dictionary update sequence element #0 has length 3; 2 is required

If I try then to add it manually, I have another error, that seems related to https (SSLv3 handshake failed):

Logger: custom_components.reolink_rest.config_flow
Source: custom_components/reolink_rest/config_flow.py:203
Integration: Reolink IP Device ([documentation](https://github.com/xannor/ha_reolink_rest), [issues](https://github.com/xannor/ha_reolink_rest/issues))
First occurred: 00:15:56 (1 occurrences)
Last logged: 00:15:56

Unhanled exception occurred
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 986, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore[return-value]  # noqa
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1089, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1119, in _create_connection_transport
    await waiter
  File "/usr/local/lib/python3.10/asyncio/sslproto.py", line 534, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/local/lib/python3.10/asyncio/sslproto.py", line 188, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 975, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/reolink_rest/config_flow.py", line 203, in async_step_user
    if not await client.login(
  File "/usr/local/lib/python3.10/site-packages/async_reolink/api/security.py", line 136, in login
    return await self._process_token(responses)
  File "/usr/local/lib/python3.10/site-packages/async_reolink/api/security.py", line 107, in _process_token
    async for response in responses:
  File "/usr/local/lib/python3.10/site-packages/async_reolink/api/commands.py", line 274, in async_trap_errors
    async for response in responses:
  File "/usr/local/lib/python3.10/site-packages/async_reolink/rest/connection.py", line 279, in _generator
    async for response in _generator():
  File "/usr/local/lib/python3.10/site-packages/async_reolink/rest/connection.py", line 245, in _generator
    response = await context
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 535, in _request
    conn = await self._connector.connect(
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 542, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 907, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
    raise last_exc
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 990, in _wrap_create_connection
    raise ClientConnectorSSLError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorSSLError: Cannot connect to host 192.168.1.25:443 ssl:default [[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)]

When I try to add the camera manually, I have no error but it is looping on the same screen (as other issue mention).

In any case thanks for you work, it's very valuable ! I would be happy if I can help you to solve this issue :)

xannor commented 2 years ago

I finally got a new version up, though this is "very" alpha, please give it a go and see if you still get this issue. If you get a different one, please close this and open a new issue.

xannor commented 2 years ago

No follow up after a few days, assuming fixed, create a new issue if you still have problems.

quentindavid commented 2 years ago

Hi @xannor ! I am sorry I was not available in the last 2 weeks. Thanks for the upgrade ! But it seems it did not solve the issue. Camera is detected but I can't add it :

File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 990, in _wrap_create_connection
    raise ClientConnectorSSLError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorSSLError: Cannot connect to host 192.168.1.17:443 ssl:default [[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)]
xannor commented 2 years ago

Unfortunately that error is outside of my code. I believe it is due to something ReoLink is doing wrong with their self signed certs on some SSL setups. I have seen reports of this happening with the NVR, but this is the first with an individual device. Unless you have a dire, specific, need for SSL encryption on your local network, I would recommend make in sure port 80 (HTTP) is enabled in the network configuration, also while in the settings, make sure ONVIF, and RTMP are enabled as well so motion events and streams are available as well.

xannor commented 2 years ago

I think I found the cause/answer to the SSL issue. I have a device that can reproduce it and I found out the cause of the error. Python 3.10 changes its TLS requirements in 3.10 and it looks like the key types used by the camera's (and possibly the NVR's) are using a very old RSA format that python is no longer accepting.

The issue is filed here: https://bugs.python.org/issue43998 and I found it via this Stack Overflow question: https://stackoverflow.com/questions/71006708/getting-sslv3-alert-handshake-failure-when-trying-to-connect-to-imap

however the solution may not be practical on current Home Assistant installs. I will have to investigate this, so I re-opened this issue.

for the time being I strongly recommend not using SSL if possible (i.e. not sharing the camera directly on the internet) or updating the firmware to a more recent version that supports TLS1.2+ (if possible) not using the self signed certificate generated by the camera, install a real one or generate a strong self signed cert somewhere else.

update: I do have a solution for this that I will include in the next release, but I do not have a release date at this time.

ThomasCr commented 1 year ago

hi, today I got a Reolink RLC-410W - I also cant add the camera. But after finding this issue I generated a own self signed cert, which meets all current standards:

For sure I imported the cert & key in the camera and verified in the browser.

I also imported the cert in my docker image for HA as /etc/ssl/certs/camera1.pem with its symlinks to the hashes (c_rehash):

But also after all this - I get:

Unhanled exception occurred

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 986, in _wrap_create_connection
    return await self._loop.create_connection(*args, **kwargs)  # type: ignore[return-value]  # noqa
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1089, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/local/lib/python3.10/asyncio/base_events.py", line 1119, in _create_connection_transport
    await waiter
  File "/usr/local/lib/python3.10/asyncio/sslproto.py", line 534, in data_received
    ssldata, appdata = self._sslpipe.feed_ssldata(data)
  File "/usr/local/lib/python3.10/asyncio/sslproto.py", line 188, in feed_ssldata
    self._sslobj.do_handshake()
  File "/usr/local/lib/python3.10/ssl.py", line 975, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/config/custom_components/reolink_rest/config_flow.py", line 204, in async_step_user
    if not await client.login(
  File "/usr/local/lib/python3.10/site-packages/async_reolink/api/security/__init__.py", line 71, in login
    async for response in self._execute(
  File "/usr/local/lib/python3.10/site-packages/async_reolink/rest/connection.py", line 252, in __execute
    response = await context
  File "/usr/local/lib/python3.10/site-packages/aiohttp/client.py", line 535, in _request
    conn = await self._connector.connect(
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 542, in connect
    proto = await self._create_connection(req, traces, timeout)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 907, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1206, in _create_direct_connection
    raise last_exc
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 1175, in _create_direct_connection
    transp, proto = await self._wrap_create_connection(
  File "/usr/local/lib/python3.10/site-packages/aiohttp/connector.py", line 990, in _wrap_create_connection
    raise ClientConnectorSSLError(req.connection_key, exc) from exc
aiohttp.client_exceptions.ClientConnectorSSLError: Cannot connect to host camera1.guest.lan:443 ssl:default [[SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:997)]

Are u sure, that a new cert fixed the issue? It looks to me, that the Webserver of the Reolink maybe uses a weak cipher - not only the cert?

ThomasCr commented 1 year ago

Hmm.. with openssl s_client -connect camera1.guest.lan:https I can see, thats the cipher used is not weak:

So, maybe a python issue?

$ openssl s_client -connect camera1.guest.lan:https
CONNECTED(00000003)
depth=0 CN = camera1
verify return:1
---
Certificate chain
 0 s:CN = camera1
   i:CN = camera1
   a:PKEY: rsaEncryption, 4096 (bit); sigalg: RSA-SHA256
   v:NotBefore: Dec 12 00:00:00 2022 GMT; NotAfter: Dec 11 23:59:59 2027 GMT
---
Server certificate
-----BEGIN CERTIFICATE-----
MIIFJDCCAwygAwIBAgIIAqIzUMZg6rMwDQYJKoZIhvcNAQELBQAwEjEQMA4GA1UE
AxMHY2FtZXJhMTAeFw0yMjEyMTIwMDAwMDBaFw0yNzEyMTEyMzU5NTlaMBIxEDAO
BgNVBAMTB2NhbWVyYTEwggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQDj
gX+r0OBPLixhpC9R4voaPJLl17WivpCouOVqqpes5t7+Aai0lXDbF31u+PGPlMQd
LbtsLxuv9vZIyrlJ7s0NlNNNDE+/xaN3XWyIJ+yLTLhfVIbI5xuP3baOt+rHk9y2
SZWbth8Lw9/YqKfUZGXptLV3TTrgPI+2435tOs8NhK2L2tMdRdfxGpKyi/CMLqPz
EORVOj7tZEGJgVyT6ssdj7/CYYg/QsLFDBJegHoat/DdY6r41gGYkQXBPJqatwZ+
OVN9uULxwbjgco5KeMK/eYQD5NANT4ohEZ+xZIYsS1Q1mY45ZjeudPNeL+tOIZe3
NURGq6yK+LeXDdmCQfRVkv1bXe8eNj5cLGaLbK5nlUqpj7OjsQzqk3TOTVVQjaZ4
AjDPavSqDRyR0FHsAj683lDGEoT/RrN3x9eafzhugD2w29PehzXaKoca4f5pIks7
fgFP+xFWC+LgZcqbjcaRRM0HfmQ9d93X6yn1L2xDT/xyhuDo7VrlKIXimHS/hUvt
XthGpBzuHHms6JEOpcmj58jGpiC5fZsR+01fnheFKt9iTvfre+64/bydVNOM6K5r
IH0MUP1nzhUPpQu5L51MXr44yxjHYlz2+xxdczY4It8BdAvXnYXusHaSisOkb+VD
oZZshwZigHBKgfa+RhWcWkAUwW3wzFva4IEofSrFXwIDAQABo34wfDAMBgNVHRMB
Af8EAjAAMB0GA1UdDgQWBBQ7hTcMjqaIY6i3KA0WPx4gsri8lTALBgNVHQ8EBAMC
A+gwEwYDVR0lBAwwCgYIKwYBBQUHAwEwKwYDVR0RBCQwIoIHY2FtZXJhMYIRY2Ft
ZXJhMS5ndWVzdC5sYW6HBMCovYgwDQYJKoZIhvcNAQELBQADggIBAE7vibekojgH
COcquSYLJ+pVvaklPhlufu2Dkg7EtIvFSxBQi9wZNpsqH3XcBRTlWZTAuS6pfadC
3TslZqYwY7eiU8YNOWr31zK3cqf1z9JqdywjxGRckbcLR6J9ugX46KkugbGq0eFC
5fAsCZK5UBHKAG352L38K79/67JJu6SUSXKRx/terhLWlTMjIvqnVJlnZvK12vX0
qUVDG9rHR98e3LO4Mr/pvYa7fSspYJ4Ix5lUIqmm4KrxSd5mjzAVZT8B9h3cEh/n
N0r+c3oPqze+Gi2tUQiNVP/ZZIEHZDWHxjGTsWEf6HA14HiVEqGWEi7hGP59XNEv
FVVPA88OE0voTwJyRwRBXEXnYvtjEfyJReIRQNcE+vJ6wp0lzndxm1HbQlHZbKNO
fMMgD+drtXPKlowHd/fLplyERL0V/24Gq87kYi5XY03VpQ8G53n2m1yJMA+BFu8p
a81u6/I1m0wz3IC9fKWeFSf4KJebOJRYhPhpgYu8HUmJ9G89wjyi3ZRQWpcSJkEx
oayRjMZvE2R9beJ/RKJT1Kldr5q1d/8nf4ejF6XBmZWoR4hXEwdknmw3FSTvC7sb
Xa2RSO/ArhxZFv5AV+aeHWMjF4JBVMXJ85+rwvpjg88+TDstU/2FBYTFw7E+9IDj
arpUpA9JazDPnEa5eZO+Hw0OjpMBser4
-----END CERTIFICATE-----
subject=CN = camera1
issuer=CN = camera1
---
No client certificate CA names sent
---
SSL handshake has read 1660 bytes and written 893 bytes
Verification: OK
---
New, TLSv1.2, Cipher is AES128-GCM-SHA256
Server public key is 4096 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
SSL-Session:
    Protocol  : TLSv1.2
    Cipher    : AES128-GCM-SHA256
    Session-ID: C4E1BF776B6D566BA43716689E889F93192C8581E24AB30ACE68963BEEEB42EE
    Session-ID-ctx: 
    Master-Key: 35426D9E331AECE8445CCEADBEC64C27B6C5CCC589516E1380E36CD640296C68B702B400DD7E5A12A2830C8628423819
    PSK identity: None
    PSK identity hint: None
    SRP username: None
    TLS session ticket lifetime hint: 300 (seconds)
    TLS session ticket:
    0000 - 43 72 db 46 99 02 0e 36-29 4d d2 e3 cb f3 4d 53   Cr.F...6)M....MS
    0010 - d5 ba 16 67 8f 9c db 48-0b 14 b1 04 51 67 44 62   ...g...H....QgDb
    0020 - 74 e1 ef c6 11 6a 0b 42-ad 37 f7 04 e1 40 77 e6   t....j.B.7...@w.
    0030 - 74 49 d8 4a 8e 83 2b a5-6d 5a cf 34 f4 38 cb 9a   tI.J..+.mZ.4.8..
    0040 - 78 aa 1d ba 20 1f b1 ce-4c 0a 81 ea 3f 3d b4 76   x... ...L...?=.v
    0050 - 6e fd eb 5f c9 ae b3 4a-e5 4c 31 6e b8 6d 85 ed   n.._...J.L1n.m..
    0060 - f6 a7 78 f8 a3 f2 f3 29-90 8f 67 b5 a7 7e a6 09   ..x....)..g..~..
    0070 - 8c 3c 33 ed 16 9a 2b 01-ee 7c ed fa 80 2b 5f ef   .<3...+..|...+_.
    0080 - e8 3b de 2e 56 59 d3 c1-ef 49 de 3c e9 40 2c 4a   .;..VY...I.<.@,J
    0090 - 63 96 0e 0e bf 33 b4 61-89 74 a5 e4 5f f6 38 e2   c....3.a.t.._.8.
    00a0 - 17 a8 8b 92 04 48 b4 86-ae 4c d4 6e af 46 4a 44   .....H...L.n.FJD
    00b0 - 54 16 0f 1e 72 de e0 92-df ff bd f7 6c 5d 0c f5   T...r.......l]..

    Start Time: 1670872766
    Timeout   : 7200 (sec)
    Verify return code: 0 (ok)
    Extended master secret: no
---
ThomasCr commented 1 year ago

hmm.. okay it needs fixed for python3.10 as xannor already pointed out.

I found also this:

I was able to solve it by adjusting the SSL context:

            context = ssl.create_default_context()
            context.set_ciphers("DEFAULT")
            context.check_hostname = False
            context.verify_mode = ssl.CERT_NONE
            self.ssl = context

https://community.home-assistant.io/t/2022-7-a-stunning-performance/437364/227?u=tcr82

ThomasCr commented 1 year ago

okay, I found where to fix it:

--- /tmp/connection.py
+++ /usr/local/lib/python3.10/site-packages/async_reolink/rest/connection.py
@@ -14,6 +14,7 @@
     overload,
 )
 from urllib.parse import urlparse
+import ssl
 import aiohttp

 from async_reolink.api.commands import (
@@ -42,10 +43,16 @@

 def _default_create_session(base_url: str, timeout: int):
+
+    context = ssl.create_default_context()                                                 
+    context.set_ciphers("DEFAULT")                                                         
+    context.check_hostname = False                                                         
+    context.verify_mode = ssl.CERT_NONE
+
     return aiohttp.ClientSession(
         base_url=base_url,
         timeout=aiohttp.ClientTimeout(total=timeout),
-        connector=aiohttp.TCPConnector(ssl=False),
+        connector=aiohttp.TCPConnector(ssl=context),
     )

also I modified this, but I think, it is not needed:

--- /tmp/push.py
+++ /config/custom_components/reolink_rest/push.py
@@ -16,6 +16,7 @@

 from xml.etree import ElementTree as et

+import ssl
 from aiohttp import ClientSession, TCPConnector, client_exceptions
 from aiohttp.web import Request

@@ -277,7 +278,12 @@

     async def _send(self, url: str, headers, data):

-        async with ClientSession(connector=TCPConnector(verify_ssl=False)) as client:
+        context = ssl.create_default_context()                                                                     
+        context.set_ciphers("DEFAULT")                                                                             
+        context.check_hostname = False                                                                             
+        context.verify_mode = ssl.CERT_NONE  
+
+        async with ClientSession(connector=TCPConnector(ssl=context)) as client:
             _LOGGER.debug("%s->%r", url, data)

             headers.setdefault("content-type", "application/soap+xml;charset=UTF-8")
ThomasCr commented 1 year ago

the question is, if it is really ALWAYS a good idea, to ignore ssl checks...?! Maybe it would be nice to have a option to ignore or check against /etc/ssl/certs/ like shown on those examples there: https://python.hotexamples.com/examples/ssl/-/create_default_context/python-create_default_context-function-examples.html

    if cafile:
      ctx = ssl.create_default_context(cafile=cafile)
    else:
      ctx = ssl.create_default_context()

    ctx.set_ciphers("DEFAULT")

    if not verify:
      ctx.check_hostname = False
      ctx.verify_mode = ssl.CERT_NONE
xannor commented 1 year ago

No, its not. I have a modifed version I am working with that supports setting the more extreme ignores, but realistically, disabling these defeats the purpose of SSL and unless you really need encryption to talk the the camera (i.e. you dont trust your network at all, or you have to expose it directly to the internet) SSL is just extra overhead degrading the cameras performance with little to no benefit.

see #23 for my thoughts on SSL and for an example of allowing the SSL

sezlony commented 1 year ago
> hi, today I got a Reolink RLC-410W - I also cant add the camera. But after finding this issue I generated a own self signed cert, which meets all current standards:
> 
>     * RSA 4096 bit private key
> 
>     * all possible sub-alternative names and ips
> 
>     * sha256 signed
> 
> 
> For sure I imported the cert & key in the camera and verified in the browser.
> 
> I also imported the cert in my docker image for HA as /etc/ssl/certs/camera1.pem with its symlinks to the hashes (c_rehash):

excuse me for being offtopic, but how is it possible to use and upload own certificates onto the cameras? i'm very much interested in that (method and tools)

ThomasCr commented 1 year ago

excuse me for being offtopic, but how is it possible to use and upload own certificates onto the cameras? i'm very much interested in that (method and tools)

On my Reolink Cam, I go to Network Settings -> Advanced -> HTTPS Settings - There it is possible to upload a cert file and key.

There are many ways to generate a ssl cert. For me I like to use Xca - but maybe u also like those: OpenSSL, MakeCert, PowerShell - for sure they are more possible ways ;-)

But there is a lot background information you should know about - file formats (unencrypted DER/PEM/PK12 - encrypted / container p7b/pfx), key type and strengths, signing algorithm, key usage, trust structures and mush more... there a many things to study :)

sezlony commented 1 year ago

just to clarify things, are you still talking about an RLC-410W cam?

ThomasCr commented 1 year ago

Yes my cam is a Reolink RLC-410W - I use the Webinterface - not any App.

sezlony commented 1 year ago

ok, than it has to do something with FW, because on v3.0.0.389_21062202 there's no option for that

thanx anyway