zoneminder integration doesn't retry if server not available upon startup

spacelama commented 8 months ago

The problem

Just like auto-closed #47489 #58538 #68162 etc, zoneminder integration doesn't attempt to restart if the server isn't available at startup (eg, your VM host infrastructure is still in the process of rebooting)

It would good if it simple automatically retried in an appropriate exponential backoff fashion upon every connection failure, either automatically or via some flag (default: true). Or simply documented how one might achieve this with separate plugins such as this is hinting might be able to be done: https://community.home-assistant.io/t/configurable-auto-restart-of-components-that-failed-setup/91483 https://community.home-assistant.io/t/wth-doesnt-ha-retry-a-failed-service-call/467850/25 eg, maybe this might be promising: http://homeassistant/hacs/repository/612717689 but I don't know enough to know where to start configuring that for my zoneminder camers.

What version of Home Assistant Core has the issue?

core-2023.12.0

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

zoneminder

Link to integration documentation on our website

No response

Diagnostics information

No response

Example YAML snippet

configuration.yaml:
zoneminder:
  - host: zm.rather.puzzling.org
    username: homeass
    password: 'blahblah'
    ssl: true
    verify_ssl: false
camera:
  - platform: zoneminder

sensor.yaml:
- platform: zoneminder
  include_archived: false

Anything in the logs that might be useful for us?

2023-12-07 19:17:49.579 WARNING (MainThread) [homeassistant.helpers.system_info] No Home Assistant Supervisor info available
2023-12-07 19:18:04.057 ERROR (MainThread) [homeassistant.setup] Error during setup of component zoneminder
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
OSError: [Errno 113] Host is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 404, in _make_request
    self._validate_conn(conn)
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn
    conn.connect()
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 363, in connect
    self.sock = conn = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8c84e73d50>: Failed to establish a new connection: [Errno 113] Host is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='zm.rather.puzzling.org', port=443): Max retries exceeded with url: /zm/api/host/login.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8c84e73d50>: Failed to establish a new connection: [Errno 113] Host is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/setup.py", line 332, in _async_setup_component
    result = await task
             ^^^^^^^^^^
  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/zoneminder/__init__.py", line 78, in setup
    success = zm_client.login() and success
              ^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/zoneminder/zm.py", line 53, in login
    req = requests.post(
          ^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 115, in post
    return request("post", url, data=data, json=json, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='zm.rather.puzzling.org', port=443): Max retries exceeded with url: /zm/api/host/login.json (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8c84e73d50>: Failed to establish a new connection: [Errno 113] Host is unreachable'))
2023-12-07 19:18:04.186 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform 'zoneminder.camera': Unable to set up component.
2023-12-07 19:18:05.595 ERROR (MainThread) [homeassistant.setup] Unable to prepare setup for platform 'zoneminder.sensor': Unable to set up component.



### Additional information

_No response_

home-assistant[bot] commented 8 months ago

Hey there @rohankapoorcom, mind taking a look at this issue as it has been labeled with an integration (zoneminder) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of `zoneminder` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Renames the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign zoneminder` Removes the current integration label and assignees on the issue, add the integration domain after the command. - `@home-assistant add-label needs-more-information` Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue. - `@home-assistant remove-label needs-more-information` Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

_{^{(message by CodeOwnersMention)}}

zoneminder documentation zoneminder source _{^{(message by IssueLinks)}}

moddroid94 commented 8 months ago

Hi @spacelama
I'm new to contributions but i think this is related to the way the integration handles setup itself, I could not make a real test, but since the error appears to be because an error connection i could just setup the integration without the actual server, then if you want you could try it with your working setup.

also i think they are migrating to another service called frigate? if I'm not wrong, so maybe this is just not relevant for long.

edit: sorry Michael I tagged the wrong person.

spacelama commented 8 months ago

I'm up for that @moddroid94 . Looks like the zoneminder integration and the upstream zm-py code hasn't been touched by the original author since 2020 or so, so it likely won't be fixed unless you or someone is able to work it out.

moddroid94 commented 8 months ago

I saw, but i think that is because they're gonna switch to frigate and the dev is busy on that, so idk about your setup but maybe you should give it a check to future proof your system. :)

Anyway I've managed to get this to setup without failing and retry the connection every 30 sec. or so, it shouldn't break anything, but I'm not sure if it's going to work as intended when devices are already present, or if the retry on connection would be catched by HA, because there's a flow to allow HA to handle the setup of a failing integration as explained in Handling Setup Failures But i don't understand how to edit this to make it work that way.

Anyway if you want to test it, please make a backup if you haven't yet, the link to the repo folder is this: https://github.com/moddroid94/core/tree/dev/homeassistant/components/zoneminder

you should copy the whole zoneminder folder into /custom_components/ Then on reboot HA should load the custom integration without touching the original, if you have any problem just delete the the folder and it falls back to the integrated one.

hope it helps :)

nabbi commented 7 months ago

Hello- I submitted #107519 to improve the ZoneMinder connection retry behavior.

I've tested this with ZM offline as HA start up and also with ZM restarting after HA is already up and running.

Thank you

home-assistant / core