search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.72k stars 30.45k forks source link

still_image_url: not work, Source: components/generic/camera.py:149 error. #47511

Closed gabriel600r closed 3 years ago

gabriel600r commented 3 years ago

Checklist

Describe the issue you are experiencing

I wanted to tell you that after the last update some of my cameras stopped showing the snapshot, this is the error message that I get in the logs.

Error getting new camera image from Camera2: peer closed connection without sending complete message body (received 11200 bytes, expected 11248)

Before updating they worked correctly. my configuration is the following,

Describe the behavior you expected

If I paste the still_image url in my browser it shows me the snapshot. so communication to the DVR is there and it works in the browser, but not within Homeassistant.

Steps to reproduce the issue

Before updating they worked correctly. my configuration is the following,

platform: generic name: Camera2 stream_source: “rtsp: // admin:password@10.10.0.150: 554 / cam / realmonitor? channel = 1 & subtype = 00” still_image_url: “http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=0” username: admin password:! secret dahuapw verify_ssl: false authentication: digest

What version of Home Assistant Core has the issue?

core-2021.3.1

What was the last working version of Home Assistant Core?

core 2021.2.3

In which browser are you experiencing the issue with?

No response

Which operating system are you using to run this browser?

Kubuntu and Windows 10

State of relevant entities

# Paste your state here.

Problem-relevant frontend configuration

# Paste your YAML here.

Javascript errors shown in your browser console/inspector

Logger: homeassistant.components.generic.camera
Source: components/generic/camera.py:149
Integration: generic (documentation, issues)
First occurred: 22:59:11 (9 occurrences)
Last logged: 23:01:46

Error getting new camera image from Camara4: peer closed connection without sending complete message body (received 31388 bytes, expected 31436)
Error getting new camera image from Camara3: peer closed connection without sending complete message body (received 50168 bytes, expected 50216)
Error getting new camera image from Camara8: peer closed connection without sending complete message body (received 9804 bytes, expected 9852)
Error getting new camera image from Camara5: peer closed connection without sending complete message body (received 64172 bytes, expected 64220)
Error getting new camera image from Camara7: peer closed connection without sending complete message body (received 6200 bytes, expected 6248)# Paste your logs here.

Hello good evening, The same thing happened to me with a Dahua DVR that I have configured as generic. After the 2021.3.0 update, the "still_image_url:" parameter stopped working, it doesn't show me the snapshots anymore. sorry for my bad english

My configuration:

platform: generic name: Camara4 stream_source: "rtsp://admin:#######@10.10.0.150:554/cam/realmonitor?channel=4&subtype=00" still_image_url: "http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=4&subtype=0" username: admin password: !secret dahuapw verify_ssl: false authentication: digest Logs: Logger: homeassistant.components.generic.camera Source: components/generic/camera.py:149 Integration: generic (documentation, issues) First occurred: 22:59:11 (9 occurrences) Last logged: 23:01:46

Error getting new camera image from Camara4: peer closed connection without sending complete message body (received 31388 bytes, expected 31436) Error getting new camera image from Camara3: peer closed connection without sending complete message body (received 50168 bytes, expected 50216) Error getting new camera image from Camara8: peer closed connection without sending complete message body (received 9804 bytes, expected 9852) Error getting new camera image from Camara5: peer closed connection without sending complete message body (received 64172 bytes, expected 64220) Error getting new camera image from Camara7: peer closed connection without sending complete message body (received 6200 bytes, expected 6248)

bramkragten commented 3 years ago

This is an issue with the integration in core, not the frontend. I'm going to transfer it.

bramkragten commented 3 years ago

And why don't you use the onvif https://www.home-assistant.io/integrations/onvif/ integration?

probot-home-assistant[bot] commented 3 years ago

generic documentation generic source (message by IssueLinks)

gabriel600r commented 3 years ago

And why don't you use the onvif https://www.home-assistant.io/integrations/onvif/ integration?

Hi Bramkragten, before reporting this problem I tried the ONVIF integration but it is not compatible with my Dahua DVR.

Thank you very much for your answer

uvjustin commented 3 years ago

The library used to grab the still images was changed from a combination of requests and aiohttp to httpx. httpx is more strict with some http protocol violations than the other two libraries - my guess is your NVR might be sending an invalid content-length. Can you try running the curl command below (please fill in your pw and redact it on the output) and sharing the output? curl --digest -u admin:yourdahuapw -X -HEAD -I http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=0

gabriel600r commented 3 years ago

Hello uvjustin, good morning, here I am attaching the output of the command you are asking for:

css@csspc:~$ curl --digest -u admin:mypass -X -HEAD -I "http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=00" HTTP/1.1 401 Unauthorized Connection: close WWW-Authenticate: Digest realm="Login to 2A00559PAEX2SQF",qop="auth",nonce="ed520af2df5b40f943561c21e67cee02"

HTTP/1.1 200 OK Content-Type: image/jpeg Content-Transfer-Encoding: binary Content-Length: 54608 Connection: close

Please let me know if you need more information and thank you very much for your response!

uvjustin commented 3 years ago

Thanks, can you run the line below? curl --digest -u admin:yourdahuapw -H "Accept-Encoding: br, gzip, deflate" -I http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=0

gabriel600r commented 3 years ago

Yes, attach the result,

css@csspc:~$ curl --digest -u admin:mypass -H "Accept-Encoding: br, gzip, deflate" -I "http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=0" HTTP/1.1 401 Unauthorized Connection: close WWW-Authenticate: Digest realm="Login to 2A00559PAEX2SQF",qop="auth",nonce="cd6dd912f64218c3cdb06a584292fb55"

HTTP/1.1 200 OK Content-Type: image/jpeg Content-Transfer-Encoding: binary Content-Length: 54956 Connection: close

uvjustin commented 3 years ago

Hmm, I thought it might show some more helpful information :-(. From your logs it shows that all the images you requested came back 48 bytes shorter than expected. I thought that might be due to a simple mixup (perhaps it was using gzip length and not deflate length), but I can't much more from that curl output. The camera does use a slightly weird http header ("Content-Transfer-Encoding"), but that doesn't seem to be the problem. If I put up a draft PR are you able to get the files and test them?

gabriel600r commented 3 years ago

Hmm, I thought it might show some more helpful information :-(. From your logs it shows that all the images you requested came back 48 bytes shorter than expected. I thought that might be due to a simple mixup (perhaps it was using gzip length and not deflate length), but I can't much more from that curl output. The camera does use a slightly weird http header ("Content-Transfer-Encoding"), but that doesn't seem to be the problem. If I put up a draft PR are you able to get the files and test them?

Hello sorry for the delay in reply, I do whatever it takes to help resolve it. I am at your service.

I wanted to see if it is possible to go back to the previous version of camera.py? it worked without problems.

sorry for my bad english.

uvjustin commented 3 years ago

The previous version had a problem with digest authentication across many cameras. But if it works for you and we can't resolve this then you can use the old version on your own install. Can you try the following 2 lines?

curl --digest -u admin:yourdahuapw -vvvv http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=0 > image.jpg
stat -c %s image.jpg
gabriel600r commented 3 years ago

I attached the output,

css@csspc:~$ curl --digest -u admin:mypass -vvvv "http://10.10.0.150/onvifsnapshot/media_service/snapshot?channel=1&subtype=0" > image.jpg

############

css@csspc:~$ stat -c %s image.jpg 55168

gabriel600r commented 3 years ago

That message is the one you mentioned of the 48bytes that are missing ...

gabriel600r commented 3 years ago

Hi Justin, i use an older version of camera.py and it works again.

Attached link of the version that I put.

https://github.com/home-assistant/core/blob/e1060f154e1dd70e80276eeedef2eb039ba3af29/homeassistant/components/generic/camera.py

I hope this information will serve you. Please let me know if you need anything else

Thanks again.

uvjustin commented 3 years ago

Thanks. It looks like it's what we suspected - the server is declaring a Content-Length of 55216 bytes while only sending 55168 bytes, which is a protocol violation. Unfortunately I don't know what we should do about this. Currently httpx doesn't seem to have an option to continue despite this violation, but it doesn't make sense to switch back to requests since the main reason for switching was because requests has a bug when using digest across different threads. I would say for now continue using the old version if it's working fine for you, and we'll keep an eye out to see if there are any updates to httpx or requests which allow us to work around this issue.

gabriel600r commented 3 years ago

Okay I understand. I was thinking if it is possible until it is fixed, create two camera.py one with httpx and one with the old version. In this way the people who are having my problem could choose.

Example: generic-camera-v1 (without httpx) generic-camera-v2 (with httpx) It's just an idea. From what I read on the community forum, there are quite a few people who are having this problem.

I hope you understand what I am trying to explain.

Thank you very much for your time Justin.

uvjustin commented 3 years ago

I had a look at the forum and the other problems don't seem to be the same as that one you have. Since the problem seems to ultimately come from an incorrect content-length from your NVR device, maybe you can try updating the NVR's firmware to see if that fixes anything. I will keep an eye on the httpx library to see if there is any change to allow us to workaround this issue.

gabriel600r commented 3 years ago

Ok Justin, I'm going to try to update the firmware, but I don't think there is since it is a pretty old DVR. Can we leave this ticket open in case you find a change in the HTTPS library? or do you prefer me to close it?

Thanks again for your time.

uvjustin commented 3 years ago

Sure, I will keep an eye on this

mkanet commented 3 years ago

@uvjustin I started getting having this issue too recently. Something definitely changed in Home Assistant to generate these errors in my log.

2021-04-30 07:30:32 ERROR (MainThread) [homeassistant.components.generic.camera] Error getting new camera image from Desktop: peer closed connection without sending complete message body (received 81681 bytes, expected 154705)
2021-04-30 08:16:35 ERROR (MainThread) [homeassistant.components.generic.camera] Error getting new camera image from Desktop: peer closed connection without sending complete message body (received 81681 bytes, expected 154705)
2021-04-30 08:20:56 ERROR (MainThread) [homeassistant.components.generic.camera] Error getting new camera image from Desktop: peer closed connection without sending complete message body (received 81681 bytes, expected 154705)
2021-04-30 08:24:41 ERROR (MainThread) [homeassistant.components.generic.camera] Error getting new camera image from Desktop: peer closed connection without sending complete message body (received 81681 bytes, expected 154705)
2021-04-30 08:25:34 ERROR (MainThread) [homeassistant.components.generic.camera] Error getting new camera image from Desktop: peer closed connection without sending complete message body (received 65297 bytes, expected 154705)

In my case, the endpoint is just a regular, fully-patched Microsoft Windows IIS 10.0 web server serving http://mywebsite.com/today/screenshot.jpg. It's behind an Nginx proxy. That image gets updated every 20 seconds using the below simple batch file:

@ECHO ON
:START

nircmd.exe savescreenshot "C:\inetpub\wwwroot\today\screenshot.jpg"
sleep 20

GOTO START

CURL:

curl --digest -X -HEAD -I https://mywebsite.com/today/screenshot.jpg

HTTP/2 405
server: nginx/1.18.0
date: Sat, 01 May 2021 02:10:11 GMT
content-type: text/html
content-length: 157

NOTE: My server does get busy sometimes; which I can't avoid.

BTW: Yesterday I created a new issue about this: https://github.com/home-assistant/core/issues/49885 If you think the issue I opened is a duplicate, please close it.

Anyway, can we at least have an option to avoid getting these errors for the people being affected recently? I don't think using an old version of camera.py is a good long-term solution. Maybe, you have a suggestion on how I can avoid getting these errors by changing the update interval or scan interval or something?

Below, is the camera configuration for Home Assistant:

camera:
  - platform: generic
    name: 'Desktop'
    still_image_url: http://mywebsite.com/today/screenshot.jpg
    verify_ssl: false
uvjustin commented 3 years ago

I suspect your setup is reporting an incorrect content-length, but the CURL output you shared doesn't have any information because your NGINX is set to block HEAD requests. I don't know much about IIS, but is it possible there is a race condition where you are overwriting the file as it is being served? Or perhaps there is a Content-Length issue with the nginx cache. Try turning off the cache to try and isolate the issue? https://forum.nginx.org/read.php?15,279202

Anyway, can we at least have an option to avoid getting these errors for the people being affected recently? I don't think using an old version of camera.py is a good long-term solution. Maybe, you have a suggestion on how I can avoid getting these errors by changing the update interval or scan interval or something?

Let's try to get to the bottom of your issue first. If your server is indeed returning an incorrect Content-Length, then that's a protocol violation, ie the server is doing something wrong. One of the motivations for the change in the http library was to get around a bug in the old library, ie HA was doing something wrong. If we have to choose between "fixing" these we need to prioritize fixing the cases where the issue stems from HA.

mkanet commented 3 years ago

Hi @uvjustin, thank you so much for your help. I am able to avoid this issue completely by pointing the generic camera integration directly to my internal IIS URL... http://192.168.1.2:8022/today/screenshot.jpg

I suspect the issue is caused by going through the reverse proxy; which can introduce issues. I tried upgrading to the latest build of Nginx; however, the problem still occurs if I use the nginx reverse proxy URL. I'm guessing I probably won't be able to "fix" Nginx to avoid this issue. So, maybe it's best to just use IIS; bypassing Nginx in this case?

Thanks again for your help.

uvjustin commented 3 years ago

Yeah, if possible it makes sense to just access your internal IIS server directly. Otherwise, you could try disabling the NGINX cache for that location or try purging the file from the NGINX cache just before reloading it.

dwarfer166 commented 3 years ago

I have the same issue and would like to use the previous version of camera.py to see if that resolves it like it has for other people. But I am running HASSIO and cannot see how I can access the components folder? I am using the File Browse plugging to browse the config directory. Is this even possible via this method when using HASSIO?

Thanks.

gabriel600r commented 3 years ago

I have the same issue and would like to use the previous version of camera.py to see if that resolves it like it has for other people. But I am running HASSIO and cannot see how I can access the components folder? I am using the File Browse plugging to browse the config directory. Is this even possible via this method when using HASSIO?

Thanks.

Hello dwarfer166, I recently migrated to HassOs and you can reach the components folder, I leave you my method:

Install SSH & Web Terminal addon, disable protection mode.

disble proteccion docker exec -t -i homeassistant /bin/bash cd /usr/src/homeassistant/homeassistant/components/generic/

and there you edit the camera.py.

I hope it helps you.

github-actions[bot] commented 3 years ago

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.