search

fwestenberg / reolink_dev

Home Assistant Reolink addon
MIT License
550 stars 102 forks source link

No motion detection when internal SSL is used #116

Open andriej opened 3 years ago

andriej commented 3 years ago

Describe the bug After having integration running for some time I've noticed errors in log regarding the connectivity and rising CPU usage - which has dropped after rebooting the cameras (2 pcs). Usually before updating the library (so before the github-name migration in HACS, previously I had them in YAML) - CPU stood at max 9% for weeks. Now it's slowly rising, so could be some issue inside the code.

To Reproduce Currently noticed, will look at the issue further

Screenshots Marked yellow the moment of cameras reboot: image (high CPU at night is backup, ignore it). image

Environment: Please provide useful information about your environment, like:

Additional context Happenes more often for one out of two cameras - the one that had auto reboot disabled. Reolink client don't have any issues and can connect without problem so it's no connectivity issue (LAN cable from HA to camera all the time)

Will add new findings in comment to this issue if any.

andriej commented 3 years ago

First finding - they are on old firmware... I was sure that I've upgraded them recently. I did right now to latest available (v3.0.0.136_20121112), will see if there are improvements.

andriej commented 3 years ago

Ok, so after update - no further errors regarding timeout's/connectivity problems. What persist is CPU problem: image

CPU rising didn't happen before (with the 'YAML' way integration). That drop was moment of HA restart.

fwestenberg commented 3 years ago

Thanks for your analysis. I think it's not sure the integration way leading to higher CPU usage in your case. There also has changed a lot in the integration. If I look at my HA supervisor, CPU is only 1.2% with two camera's connected. Is everything working well at least? Motion detection working?

andriej commented 3 years ago

@fwestenberg thank you for your question - I've checked and indeed, there may be still problem :-) as motion detection didn't show any motion since 24hours and i.e. IR-lights looks like: image every 1 hour and 1 minute there's an second of 'unavailable'

Shall I re-add those cameras or dig something further? (Motion detection were working previously of course)

fwestenberg commented 3 years ago

Yes this woud be great. IR light are working? I can't understand your language. ;-)

andriej commented 3 years ago

Right, I've switched to english. Before removal, just to have history:

It seems like the integration didn't went through whole process and there's something going in background that makes CPU rise and also not report all functionalities like it should. Will remove and re-add right now

fwestenberg commented 3 years ago

Ok, also I found some errors in the code. I will update a.s.a.p.

andriej commented 3 years ago

While removing cameras this entry appeared in log (maybe useful?): image

fwestenberg commented 3 years ago

This is probably the reason why motion detection is not working. But I already fixed this one. I will create a PR later today.

andriej commented 3 years ago

Yes, now I've confirmed - in app (reolink client) I see motion detected and video recorded - in HA it's no-motion. Will wait for PR - thanks for fast investigation and great integration! :-) Glad I can help to hunt some bugs.

fwestenberg commented 3 years ago

Can you update the files in custom_components/reolink_dev with the files from this branch? Please let me know the results.

andriej commented 3 years ago

Deleted files, added linked one and restarted (and readded cameras) - still no motion reported, despite that they seem to work: image

fwestenberg commented 3 years ago

The availability is best to find in the log (debug mode). Is the subscription working? And did it register a webhook?

andriej commented 3 years ago

Could you please provide what components to put in 'debug' mode for logger? I always have trouble guessing it. ;)

fwestenberg commented 3 years ago

Yes, I described this and some more interesting things here

fwestenberg commented 3 years ago

I would suggest next time to mask your webhook id (last part). If you just reload the integration, it will generate another one and delete the old. What I see, is you are using https internal also. And this does not work with the Reolink camera yet (unfortunately). It's described in another post also.

andriej commented 3 years ago

Ok, so that's the thing that changed between versions? Will it be possible in any future soon? Just to make sure: it's 100% proper SSL that can be validated (LE).

fwestenberg commented 3 years ago

It works totally different now, and should lead to less http calls etc. Read more here. Maybe it can work, but needs more investigation first.

andriej commented 3 years ago

I'd be happy to help investigate and test it out, as at the moment I can't motivate myself to drop SSL support (I have it done same way as in linked thread - forced via nginx proxy)

fwestenberg commented 3 years ago

Can you check the log if it still uses the https, since you updated the files it should now use http...

andriej commented 3 years ago

I re-did everything, so removed integration, deleted files, did git clone https://github.com/fwestenberg/reolink_dev/ --branch NVR-Webhook-subscription and in logs I see: [custom_components.reolink_dev.base] Host 192.168.x.102 subscribed successfully to webhook https://xxx/api/webhook/a180bxxxx

fwestenberg commented 3 years ago

The answer to that is in this post, I think!

andriej commented 3 years ago

Yes, could be - but integrations I rely on doesn't allow me to have HTTP also inside my network and I'd rather like not to drop some level of security too. And there's no way I can declare any kind of workaround too (as it's taken from HA, not configurable parameter).

Is there something that can be changed on library/integration level or it's rather something to debug on reolink's side? Bugreport? Any way to see/check what exactly is the issue for reolink, debug that?

jjeremia commented 3 years ago

Just a comment to this... My cameras worked perfect, with motion detection, prior to moving config from configuration.yaml to integration setup. After that I have not gotten any motion detection. I have always used internal SSL.

andriej commented 3 years ago

So it's the same issue as mine. Good, one more to potential debug. :-)

fwestenberg commented 3 years ago

It works totally different now, and should lead to less http calls etc. Read more here. Maybe it can work, but needs more investigation first.

Like I wrote here, it's useless to compare to the old version. Things have changed significantly and we are not polling for motion every second. With both of my camera's this works great. I have motion detected within a second and without missing any motion. So this is definitely the way to go. Also, the official Reolink Windows application works this way. So we can only fix your issue by:

andriej commented 3 years ago

Why would camera need SSL cert when it has proper non-self signed one? Is there any API/way to test those calls from camera to HA? Or just get some more debug codes? I'd like to investigate more so potential solution will be more user friendly than creating custom webhooks (and HA doesn't allow to create custom webhooks anymore too?)

fwestenberg commented 3 years ago

Why would camera need SSL cert when it has proper non-self signed one? Is there any API/way to test those calls from camera to HA? Or just get some more debug codes? I'd like to investigate more so potential solution will be more user friendly than creating custom webhooks (and HA doesn't allow to create custom webhooks anymore too?)

This is just one of the possible things to try. It's not about the camera's certificate, but it should probably first trust your HA certificate?

andriej commented 3 years ago

But why would it not trust that certificate? How can we check if it's problem with cert or other problem? I'm not a programmer, tried to look at api.cgi but failed as I don't know internals how to check and check response codes :-)

My SSL cert is verified by proper CA, in browsers it's shown as 'secured' so camera shouldn't have any issues with that

badabing2005 commented 3 years ago

But why would it not trust that certificate? How can we check if it's problem with cert or other problem? I'm not a programmer, tried to look at api.cgi but failed as I don't know internals how to check and check response codes :-)

My SSL cert is verified by proper CA, in browsers it's shown as 'secured' so camera shouldn't have any issues with that

It all depends on which certificate authority signed your certificate and if that authority is in root CA of the camera / NVR How are you accessing api.cgi? where are you accessing this?

fwestenberg commented 3 years ago

Thanks for renaming this one. And I thing @badabing2005 you are right about that. But I also did not look any further. I am using the ONVIF event subscription for motion detection. You can check it out in the other repository (fwestenberg/reolink). I can also help you get this in soapui for some testing.

andriej commented 3 years ago

I can try to help and debug/see possible options, but for sure assistance is needed with the library and/or API :-)

badabing2005 commented 3 years ago

Thanks for renaming this one. And I thing @badabing2005 you are right about that. But I also did not look any further. I am using the ONVIF event subscription for motion detection. You can check it out in the other repository (fwestenberg/reolink). I can also help you get this in soapui for some testing.

@fwestenberg Let me know what you need and how I can get you the info you need.

andriej commented 3 years ago

Does the library use onvif subscription to get that notifications or it's more reolink's cgi file?

fwestenberg commented 3 years ago

It uses onvif subscription for notifications. Because NVR is not working well at the moment, after a NVR motion notification we call the motion service from the cgi to find out which camera has motion detected.

andriej commented 3 years ago

Not sure if I get it properly - first to fix is the NVR issue and then it might help with this one, right?

fwestenberg commented 3 years ago

Indeed, the first NVR issue should be fixed. Then we will look further for a certificate fix or whatever.

fwestenberg commented 3 years ago

I sent @badabing2005 a SoapUI project for testing Onvif services. If you are interested as well, please share your email with me.

Jonnehs commented 3 years ago

I'm using HTTPS for HASS and would like the motion sensor to be available for triggering automations etc, is that possible somehow?

AndyVRD commented 3 years ago

Hi,

First of all thanks for this amazing integration. I'm also using HTTPS for HASS and the motion sensor is also not working for me, all other things working fine.

Jonnehs commented 3 years ago

has there been any progress on this, or is there a work around? Other than using http for your whole HASS system?

andriej commented 3 years ago

Not yet, unfortunately

Jonnehs commented 3 years ago

is there any way to get the webhook for this integration to use http as a temporary measure? Or can such a thing be implemented as a work around until an SSL fix is available? Really need to be able to get motion events to solve an issue