search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
73.78k stars 30.88k forks source link

No images in Lovelace from generic cameras. #47624

Closed RaveGun closed 3 years ago

RaveGun commented 3 years ago

The problem

Since the update to core-2021.3.x the generic IP cameras are no longer displayed in Lovelace. There is no issue or anything else in the log.

What is version of Home Assistant Core has the issue?

core-2021.3.2

What was the last working version of Home Assistant Core?

core-2021.2.x

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

integration: camera

Link to integration documentation on our website

https://www.home-assistant.io/integrations/generic/

Example YAML snippet

  ################
  ### Front property camera
  ################
  - platform: generic
    name: 'Front'
    still_image_url: !secret camera_front_img
    stream_source: !secret camera_front_vid
    verify_ssl: false

Anything in the logs that might be useful for us?

None

Upon inspecting the page I saw that there are errors reported in the browser console.

GEThttp://192.168.1.122:8123/api/camera_proxy/camera.front?authSig=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJlMWM5YWE5MzYxMjc0MWQ2YTUwM2FhMmVhNjk1NDQwYyIsInBhdGgiOiIvYXBpL2NhbWVyYV9wcm94eS9jYW1lcmEuZnJvbnQiLCJpYXQiOjE2MTUyMTM2ODcsImV4cCI6MTYxNTIxMzcxN30.HX9lKRjes4lUwxGv7Y0AltSMRSIrVX9Xq7qWRnmeQ5g
[HTTP/1.1 500 Internal Server Error 10840ms]

After a Core restart it works for a while. The same behavior is on any web browser and also on the companion app.

probot-home-assistant[bot] commented 3 years ago

generic documentation generic source (message by IssueLinks)

RaveGun commented 3 years ago

Actually I just realize this shall be more of a: "integration: camera" label.

uvjustin commented 3 years ago

If you are using a username/password, please try including them separately in the yaml instead of embedding them in still_image_url.

RaveGun commented 3 years ago

Has anything changed in respect to generic camera implementation?

The proposed solution will not work because is not a normal login. This is how the secret URL looks like:

camera_front_img: http://192.168.1.70/cgi-bin/api.cgi?cmd=Snap&channel=0&user=1ooser&password=muchs3cur3w0w

The link above works just fine on the browser.

There must be something with the tockens on the Core side.

uvjustin commented 3 years ago

I see. That url should be fine. Yes, there were changes in both the generic camera integration as well as in the base camera integration itself. The generic camera integration was changed to use the httpx library, but any failures there should leave log messages. The changes to the base camera integration were mostly to do with stream, so I wouldn't expect that those would affect the still images. Also interesting is the fact that it works for a while after a reboot. To rule out the generic component, can you try reverting the generic/camera.py to the previous working version?

RaveGun commented 3 years ago

I have changed the generic/camera.py in the Core container with this version: https://github.com/home-assistant/core/blob/0e98bc5ea23cd15ecdc17a8318250b4a8fbe9478/homeassistant/components/generic/camera.py

For the moment is works after the restart. I will report back when / if it stops working again.

RaveGun commented 3 years ago

With the changed camera.py, as above described, it is still working fine. No issues until now.

uvjustin commented 3 years ago

That's interesting, thanks for the update. For the version with problems, how long does it take for the images to not work?

RaveGun commented 3 years ago

Last time it was after about an hour or so, but I did not tracked this actively.

uvjustin commented 3 years ago

See if you can try this version of camera.py to see if it helps.

RaveGun commented 3 years ago

Hi,

Thanks for the update. I saw what you did there. This might have been the issue. I just made the changes, restarted the Core and so far it is running well. Will report back in a couple of hours.

Thanks again.

RaveGun commented 3 years ago

I spoke too soon. These errors showed up in the log:

Logger: aiohttp.server
Source: components/generic/camera.py:152
First occurred: 7:31:48 PM (3 occurrences)
Last logged: 7:32:11 PM

Error handling request
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/generic/camera.py", line 140, in async_camera_image
    response = await async_client.get(
  File "/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1548, in get
    return await self.request(
  File "/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1371, in request
    response = await self.send(
  File "/usr/local/lib/python3.8/site-packages/httpx/_client.py", line 1416, in send
    await response.aread()
  File "/usr/local/lib/python3.8/site-packages/httpx/_models.py", line 1234, in aread
    self._content = b"".join([part async for part in self.aiter_bytes()])
  File "/usr/local/lib/python3.8/site-packages/httpx/_models.py", line 1234, in <listcomp>
    self._content = b"".join([part async for part in self.aiter_bytes()])
  File "/usr/local/lib/python3.8/site-packages/httpx/_models.py", line 1247, in aiter_bytes
    async for chunk in self.aiter_raw():
  File "/usr/local/lib/python3.8/site-packages/httpx/_models.py", line 1286, in aiter_raw
    async for part in self.stream:
  File "/usr/local/lib/python3.8/site-packages/httpcore/_async/connection_pool.py", line 61, in __aiter__
    async for chunk in self.stream:
  File "/usr/local/lib/python3.8/site-packages/httpcore/_bytestreams.py", line 73, in __aiter__
    async for chunk in self._aiterator:
  File "/usr/local/lib/python3.8/site-packages/httpcore/_async/http11.py", line 155, in _receive_response_data
    event = await self._receive_event(timeout)
  File "/usr/local/lib/python3.8/site-packages/httpcore/_async/http11.py", line 172, in _receive_event
    data = await self.socket.read(self.READ_NUM_BYTES, timeout)
  File "/usr/local/lib/python3.8/site-packages/httpcore/_backends/asyncio.py", line 144, in read
    return await asyncio.wait_for(
  File "/usr/local/lib/python3.8/asyncio/tasks.py", line 481, in wait_for
    await waiter
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request
    resp = await self._request_handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_app.py", line 499, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 56, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 18, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 72, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 129, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 129, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 496, in get
    return await self.handle(request, camera)
  File "/usr/src/homeassistant/homeassistant/components/camera/__init__.py", line 513, in handle
    image = await camera.async_camera_image()
  File "/usr/src/homeassistant/homeassistant/components/generic/camera.py", line 152, in async_camera_image
    response.close()
UnboundLocalError: local variable 'response' referenced before assignment
uvjustin commented 3 years ago

I see, I updated the file to fix that. Can you try again?

RaveGun commented 3 years ago

It is now up and running. There are no log entries either. I just want to mention that the previous fix showed the same issue.

RaveGun commented 3 years ago

Unfortunately it stopped working after a while. The last update did not fixed the issue.

I have a doorbell automation that is sending a snapshot to my phone from the same URL as in the camera configuration and it is working fine. This means that getting the snapshot works even if the generic:camera stopped. But these are anyhow not related.

Is there anything else I should try? Is there something else on my side that is misbehaving? Is only my installation affected? Hmm..

uvjustin commented 3 years ago

How long did it work before not working? Any logs? I made another update to the file, see if that helps anything.

RaveGun commented 3 years ago

Thanks for the info. The update is up and running.

Last time it was again around one hour. I cannot tell exactly as it was in the middle of the night. There were also no logs, or nothing special nor related to the issue. All of my cameras are scheduled to restart once a week overnight on a different day and time. So there are the usual cannot get image from "insert_camera_name".

I will let you know how it went overnight.

RaveGun commented 3 years ago

Hi, It sopped after 2,5 Hours and now there are old previews. No information in the logs but I do have the default log level set to error. So the asyncio.CancelledError will not be logged. I have added now these two lines to the logger:

  homeassistant.components.camera: debug
  homeassistant.components.generic.camera: debug

Maybe this will show more.

RaveGun commented 3 years ago

After the restart it stopped after 1 hour and 15 minutes but now in the log are 1900+ cancelling getting camera image warnings for all generic cameras.

uvjustin commented 3 years ago

Did it always show old previews? It seems like at some point the client stops working and subsequent timeouts get triggered at the higher level (in camera as opposed to generic). I changed the file to log out all unknown exceptions to error - let's see if we can pick up anything.

RaveGun commented 3 years ago

No, it did not always shown old previews. Only the latest updates with the return self._last_image are doing this.

RaveGun commented 3 years ago

There were about 12k cancel warnings but no unknown error so far. I disabled now the logging for the warnings and restarted the Core.

Is the CancelledError exception triggered by the 500 Internal Server Error?

uvjustin commented 3 years ago

The CancelledError exception comes when the camera component times out when getting an image from generic. It's not strange to get that once or twice if there is some connectivity issue, but the problem is that whatever is causing the timeout keeps on happening. There seem to be a number of outstanding issues with the httpx async client. I changed the file to use the sync client instead. Can you see if that fixes things?

RaveGun commented 3 years ago

LOL, now I am getting 500: Internal Server Error also from raw.githubusercontent.com trying to download the updated camera.py.

RaveGun commented 3 years ago

After GitHub went back online I did the update and is working normally for the last 12 hours. The sync access did the trick.

Thank you.

uvjustin commented 3 years ago

@RaveGun It was suggested we try to work around the issue while still using the async api. I've updated the camera.py file. Can you try it again?

RaveGun commented 3 years ago

Update done and restarted. The cameras are working after the restart. I will be back with more information in a couple of hours. Thanks.

RaveGun commented 3 years ago

Hello,

Until now everything looks good. I guess that the issue has been fixed with the latest update.

Thank you for your time and support.

Joc007 commented 3 years ago

Hello,

Logger: homeassistant.components.generic.camera Source: components/generic/camera.py:149 Error getting new camera image from imou_cue2c: Unsupported URL protocol 'rtsp'

I am on HA 2021.3.4.

Any ideas, what can I check?

Thanks, Joc.

HAnoobhacker commented 3 years ago

Same issue here. Strange that the camera image comes up fine on the iphone Home Assistant app. I see that there is an updated file "camera.py". unfortunately, I have no idea what to do with the file or where it needs to go. Searched HA file system and found no file wit the same name. Is there any docs that detail what needs to be done in order to update the file?

Thank you, Andy

RaveGun commented 3 years ago

@HAnoobhacker Is the image on the app also actual? Depending on your installation the file is located here: /core/homeassistant/components/generic/

In have a docker installation so I had to connect to the container's console and change the file from in the image from within.

@Joc007 sorry, I cannot help with this. Maybe @uvjustin knows how is the rtsp protocol handled.

HAnoobhacker commented 3 years ago

Yes. The image is "Live Broadcast". I'm running core-2021.3.4 Home Assistant OS 5.12. in VirtualBox 6.1 on Ubuntu Desktop 20.4. I'm not seeing that directory structure in the HA VM.

uvjustin commented 3 years ago

@Joc007 It looks like your setup is incorrect. RTSP urls are camera feeds which are used in stream_source, while the still_image_url should be a http or https image url.

rpitera commented 3 years ago

I've been having this same problem and for me it coexists with my REST sensors failing. I've been moving my REST calls over to NodeRed and while this has solved the issues with REST, I am still having the issue with generic cams. It's frustrating because I've had a stable HA for years and these cams have worked forever until a month or two ago (maybe three).

Here's the info I posted in issue #43600 :

Been moving my REST calls to Node Red as per @tiagofreire-pt and while it's helped with my REST issues, I am also having issues with generic based URL cameras, such as ones for radar/weather conditions as well as those that make a REST call to get a snapshot (e.g. getting a snap from a tablet cam via Fully Kiosk). When the existing HA REST sensors start to fail on me, so do these types of cameras. It's making me wonder if the issue isn't REST but the http/https calls. It doesn't matter whether these calls are internal or external to my network. Even after these fail, I can still ping or do an nslookup from the HA console so I know there isn't a network/DNS issue involved. I've tried debugging aiohttp but see nothing there. It's really getting frustrating. I even built a new VM in Virtualbox and restored my snapshot to it thinking there might be some corruption in the core but it's still there.

System Health

version core-2021.3.4
installation_type Home Assistant OS
dev false
hassio true
docker true
virtualenv false
python_version 3.8.7
os_name Linux
os_version 5.4.99
arch x86_64
timezone America/New_York
Home Assistant Community Store GitHub API | ok -- | -- Github API Calls Remaining | 4719 Installed Version | 1.11.3 Stage | running Available Repositories | 779 Installed Repositories | 54
AccuWeather can_reach_server | ok -- | -- remaining_requests | 49
Home Assistant Cloud logged_in | true -- | -- subscription_expiration | April 16, 2021, 8:00 PM relayer_connected | true remote_enabled | true remote_connected | true alexa_enabled | true google_enabled | true can_reach_cert_server | ok can_reach_cloud_auth | ok can_reach_cloud | ok
Hass.io host_os | Home Assistant OS 5.12 -- | -- update_channel | stable supervisor_version | supervisor-2021.03.6 docker_version | 19.03.13 disk_total | 30.8 GB disk_used | 21.6 GB healthy | true supported | true board | ova supervisor_api | ok version_api | ok installed_addons | Terminal & SSH (9.1.0), ADB - Android Debug Bridge (0.7.0), ESPHome (1.16.2), Visual Studio Code (3.3.0), Let's Encrypt (4.11.0), Node-RED (8.2.1), Home Assistant Google Drive Backup (0.103.1), MariaDB (2.2.2), Check Home Assistant configuration (3.6.0), Glances (0.11.2), Bitwarden RS (0.9.0), Portainer (1.4.0), Tautulli (2.0.5), phpMyAdmin (0.2.0), ozwcp (1.1.2), php-nginx Docker Image (latest), Eufy Home Assistant MQTT Bridge (1.13.0), ioBroker (0.8.0), rest980 Docker Image (20200205)
Lovelace dashboards | 2 -- | -- resources | 45 views | 28 mode | storage
uvjustin commented 3 years ago

@rpitera The issue with generic was introduced in https://github.com/home-assistant/core/pull/46576 when we switched the http library to use httpx. Unfortunately it looks like there is an outstanding issue with httpx not freeing up resources when the request times out and gets cancelled. We have a PR to work around this here: https://github.com/home-assistant/core/pull/47852 . There is some info at the top of the PR with links to the httpx issues. It's possible that the rest component has a similar issue with the library.

rpitera commented 3 years ago

@uvjustin Thanks so much for the reply; running out of places to look for resources so I'll read the issues/PRs you referenced. Appreciate the pointer. I've been setting up NR to now capture my images and save them to a local file and displaying them using the local file camera. But I'd really like to get to the bottom of it since I still have a few cams that won't work with my work-around.