search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.7k stars 30.45k forks source link

ONVIF Integration suddenly lost a number of entities. #90199

Closed GreyGnome closed 1 year ago

GreyGnome commented 1 year ago

The problem

I was using the ONVIF integration for my Amcrest camera. It was 1 device, 14 entities. I was creating some automations for it, and they were mostly working just the way I wanted.

Then they stopped. I went to the Integrations page and looked at my device, and half of my entities were disabled. They said "entity is no longer being provided by the Integration."

So I deleted the device entirely, then readded it.

It came back with only 7 entities. For example, any motion detection entities were not there.

I am able to log into my camera from the Amcrest app and all looks well, so I don't know why or how this suddenly changed. As I said, I was in automations, they were working, then they stopped.

What version of Home Assistant Core has the issue?

core-2023.3.6

What was the last working version of Home Assistant Core?

core-2023.3.6

What type of installation are you running?

Home Assistant OS

Integration causing the issue

ONVIF

Link to integration documentation on our website

https://www.home-assistant.io/integrations/onvif/

Diagnostics information

Sorry I cannot do it as I deleted the Integration.

Furthermore, now I cannot add my ONVIF device. HomeAssistant says it has an unknown error.

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Not sure if it's related, but I saw this in my log while I was trying to add on ONVIF integration again:

2023-03-23 16:28:24.555 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 34, in read
    return await self._stream.receive(max_bytes=max_bytes)
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1265, in receive
    await self._protocol.read_event.wait()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 32, in read
    with anyio.fail_after(timeout):
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
    raise TimeoutError
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 112, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 91, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 155, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 191, in _receive_event
    data = await self._network_stream.read(
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 31, in read
    with map_exceptions(exc_map):
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc)
httpcore.ReadTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 67, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 80, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 234, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 146, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 180, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 72, in wrapper
    result = await method(view, request, data, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 271, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 367, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/onvif/config_flow.py", line 157, in async_step_configure
    return await self.async_setup_profiles()
  File "/usr/src/homeassistant/homeassistant/components/onvif/config_flow.py", line 197, in async_setup_profiles
    await device.update_xaddrs()
  File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 294, in update_xaddrs
    capabilities = await devicemgmt.GetCapabilities({"Category": "All"})
  File "/usr/local/lib/python3.10/site-packages/zeep/proxy.py", line 64, in __call__
    return await self._proxy._binding.send_async(
  File "/usr/local/lib/python3.10/site-packages/zeep/wsdl/bindings/soap.py", line 156, in send_async
    response = await client.transport.post_xml(
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 235, in post_xml
    response = await self.post(address, message, headers)
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 220, in post
    response = await self.client.post(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout
2023-03-23 16:29:22.183 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:29:52.185 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:29:59.371 WARNING (Thread-6) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.468 WARNING (Thread-2) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.474 WARNING (Thread-4) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.479 WARNING (Thread-6) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.579 WARNING (Thread-2) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.584 WARNING (Thread-4) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.590 WARNING (Thread-6) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.689 WARNING (Thread-2) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.693 WARNING (Thread-4) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.696 WARNING (Thread-6) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.801 WARNING (Thread-2) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.804 WARNING (Thread-4) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.812 WARNING (Thread-6) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.911 WARNING (Thread-2) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:29:59.919 WARNING (Thread-4) [daemon] could not find handler for: _handle_resolve
2023-03-23 16:30:01.493 WARNING (Thread-2) [daemon] could not find handler for: _handle_probe
2023-03-23 16:30:01.508 WARNING (Thread-6) [daemon] could not find handler for: _handle_probe
2023-03-23 16:30:01.512 WARNING (Thread-4) [daemon] could not find handler for: _handle_probe
2023-03-23 16:30:22.189 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:30:52.188 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:31:22.189 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:31:52.191 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:32:22.192 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:32:52.193 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:33:22.193 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:33:52.200 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:34:01.125 ERROR (MainThread) [frontend.js.latest.202303091] :0:0 ResizeObserver loop completed with undelivered notifications.
2023-03-23 16:34:05.212 WARNING (Thread-4) [daemon] could not find handler for: _handle_probe
2023-03-23 16:34:05.215 WARNING (Thread-2) [daemon] could not find handler for: _handle_probe
2023-03-23 16:34:05.222 WARNING (Thread-6) [daemon] could not find handler for: _handle_probe
2023-03-23 16:34:22.202 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:34:52.203 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:35:22.204 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:35:52.205 WARNING (MainThread) [homeassistant.helpers.entity] Update of media_player.spotify_greygnome is taking over 10 seconds
2023-03-23 16:35:52.252 ERROR (MainThread) [aiohttp.server] Error handling request
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 112, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 91, in handle_async_request
    ) = await self._receive_response_headers(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 155, in _receive_response_headers
    event = await self._receive_event(timeout=timeout)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http11.py", line 205, in _receive_event
    raise RemoteProtocolError(msg)
httpcore.RemoteProtocolError: Server disconnected without sending a response.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_protocol.py", line 433, in _handle_request
    resp = await request_handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_app.py", line 504, in _handle
    resp = await handler(request)
  File "/usr/local/lib/python3.10/site-packages/aiohttp/web_middlewares.py", line 117, in impl
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/security_filter.py", line 67, in security_filter_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/forwarded.py", line 100, in forwarded_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/request_context.py", line 28, in request_context_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/ban.py", line 80, in ban_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/auth.py", line 234, in auth_middleware
    return await handler(request)
  File "/usr/src/homeassistant/homeassistant/components/http/view.py", line 146, in handle
    result = await result
  File "/usr/src/homeassistant/homeassistant/components/config/config_entries.py", line 180, in post
    return await super().post(request, flow_id)
  File "/usr/src/homeassistant/homeassistant/components/http/data_validator.py", line 72, in wrapper
    result = await method(view, request, data, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/helpers/data_entry_flow.py", line 110, in post
    result = await self._flow_mgr.async_configure(flow_id, data)
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 271, in async_configure
    result = await self._async_handle_step(
  File "/usr/src/homeassistant/homeassistant/data_entry_flow.py", line 367, in _async_handle_step
    result: FlowResult = await getattr(flow, method)(user_input)
  File "/usr/src/homeassistant/homeassistant/components/onvif/config_flow.py", line 157, in async_step_configure
    return await self.async_setup_profiles()
  File "/usr/src/homeassistant/homeassistant/components/onvif/config_flow.py", line 197, in async_setup_profiles
    await device.update_xaddrs()
  File "/usr/local/lib/python3.10/site-packages/onvif/client.py", line 294, in update_xaddrs
    capabilities = await devicemgmt.GetCapabilities({"Category": "All"})
  File "/usr/local/lib/python3.10/site-packages/zeep/proxy.py", line 64, in __call__
    return await self._proxy._binding.send_async(
  File "/usr/local/lib/python3.10/site-packages/zeep/wsdl/bindings/soap.py", line 156, in send_async
    response = await client.transport.post_xml(
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 235, in post_xml
    response = await self.post(address, message, headers)
  File "/usr/local/lib/python3.10/site-packages/zeep/transports.py", line 220, in post
    response = await self.client.post(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1848, in post
    return await self.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: Server disconnected without sending a response.

Additional information

No response

home-assistant[bot] commented 1 year ago

Hey there @hunterjm, mind taking a look at this issue as it has been labeled with an integration (onvif) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `onvif` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign onvif` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)

onvif documentation onvif source (message by IssueLinks)

GreyGnome commented 1 year ago

BTW, the camera is an IP4M-1055E.

ohkaja commented 1 year ago

FYI: My "Lupus" cams after updating to 2023.3.5 came back with too much entities wich are not supported by the hardware (turn on/off IR light, turn on/off wipers, and one other wich I not remeber jet).

GreyGnome commented 1 year ago

I'm going to be away from my computer for Spring Break so I won't be able to look at this issue until April 2.

bdraco commented 1 year ago

httpx.RemoteProtocolError: Server disconnected without sending a response.

Protocol errors need to be retried since HTTP/1.1 is allowed to disconnect at any time

bdraco commented 1 year ago

This needs to be fixed in zeep

GreyGnome commented 1 year ago

Sorry, what’s zeep? Is there anything I need to do to advance this issue?

On Mon, Mar 27, 2023 at 2:04 PM J. Nick Koston @.***> wrote:

This needs to be fixed in zeep

— Reply to this email directly, view it on GitHub https://github.com/home-assistant/core/issues/90199#issuecomment-1485859448, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA2KFGPOTFNQNINCWDE4U6LW6H6FXANCNFSM6AAAAAAWFYSBCY . You are receiving this because you authored the thread.Message ID: @.***>

-- -Mike Schwager

bdraco commented 1 year ago

Wait for https://github.com/mvantellingen/python-zeep/pull/1369 to be merged and released

bdraco commented 1 year ago

https://github.com/agronholm/anyio/issues/374 might actually be the root cause here

bdraco commented 1 year ago

Please try 2023.4.5

We added some more retry logic while waiting for the above fixes. It might help

bdraco commented 1 year ago

Since I haven't heard back, I'm going to assume the issue is resolved. Please reopen this issue if there is still a problem.