search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.81k stars 30.51k forks source link

iAqualink - Error doing job: Task exception was never retrieved #87795

Closed smshapiro85 closed 1 year ago

smshapiro85 commented 1 year ago

The problem

Suddenly getting errors with the iAqualink integration. Seems to work once I restart home assistant, but it quickly breaks after, making controlling any of the devices not possible.

Linked debug log for more information.

What version of Home Assistant Core has the issue?

2023.2.0

What was the last working version of Home Assistant Core?

A few versions before

What type of installation are you running?

Home Assistant Supervised

Integration causing the issue

iAqualink

Link to integration documentation on our website

https://www.home-assistant.io/integrations/iaqualink

Diagnostics information

home-assistant_iaqualink_2023-02-09T16-58-14.623Z.log

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

Suddenly getting errors with the iAqualink integration. Seems to work once I restart home assistant, but it quickly breaks after.

home-assistant[bot] commented 1 year ago

Hey there @flz, mind taking a look at this issue as it has been labeled with an integration (iaqualink) you are listed as a code owner for? Thanks!

Code owner commands Code owners of `iaqualink` can trigger bot actions by commenting: - `@home-assistant close` Closes the issue. - `@home-assistant rename Awesome new title` Change the title of the issue. - `@home-assistant reopen` Reopen the issue. - `@home-assistant unassign iaqualink` Removes the current integration label and assignees on the issue, add the integration domain after the command.

(message by CodeOwnersMention)

iaqualink documentation iaqualink source (message by IssueLinks)

PRGeno commented 1 year ago

I'm seeing the same problem here. A reload of the iAqualink integration brings it back on line again, but only temporarily. As reported in my log, the problem started occurring on February 3rd so maybe the Aqualink service may have changed something on their end that needs accommodation in the integration to be fully operational again.

Here is a copy of my log entry when it fails:

Logger: homeassistant
Source: components/iaqualink/__init__.py:152
First occurred: February 3, 2023 at 11:53:03 AM (13663 occurrences)
Last logged: 9:31:24 AM

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 130, in _call_sslobject_method
    result = func(*args)
  File "/usr/local/lib/python3.10/ssl.py", line 917, in read
    v = self._sslobj.read(len)
ssl.SSLWantReadError: The operation did not complete (read) (_ssl.c:2548)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 34, in read
    return await self._stream.receive(max_bytes=max_bytes)
  File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 195, in receive
    data = await self._call_sslobject_method(self._ssl_object.read, max_bytes)
  File "/usr/local/lib/python3.10/site-packages/anyio/streams/tls.py", line 137, in _call_sslobject_method
    data = await self.transport_stream.receive()
  File "/usr/local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 1265, in receive
    await self._protocol.read_event.wait()
  File "/usr/local/lib/python3.10/asyncio/locks.py", line 214, in wait
    await fut
asyncio.exceptions.CancelledError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 10, in map_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 32, in read
    with anyio.fail_after(timeout):
  File "/usr/local/lib/python3.10/site-packages/anyio/_core/_tasks.py", line 118, in __exit__
    raise TimeoutError
TimeoutError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 144, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 112, in handle_async_request
    status, headers = await self._receive_response(
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 229, in _receive_response
    event = await self._receive_stream_event(request, stream_id)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 260, in _receive_stream_event
    await self._receive_events(request, stream_id)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 281, in _receive_events
    events = await self._read_incoming_data(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 341, in _read_incoming_data
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 327, in _read_incoming_data
    data = await self._network_stream.read(self.READ_NUM_BYTES, timeout)
  File "/usr/local/lib/python3.10/site-packages/httpcore/backends/asyncio.py", line 31, in read
    with map_exceptions(exc_map):
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_exceptions.py", line 14, in map_exceptions
    raise to_exc(exc)
httpcore.ReadTimeout

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/iaqualink/__init__.py", line 152, in _async_systems_update
    await system.update()
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/system.py", line 92, in update
    r2 = await self._send_devices_screen_request()
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/system.py", line 79, in _send_devices_screen_request
    r = await self._send_session_request(IAQUA_COMMAND_GET_DEVICES)
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/system.py", line 72, in _send_session_request
    return await self.aqualink.send_request(url)
  File "/usr/local/lib/python3.10/site-packages/iaqualink/client.py", line 98, in send_request
    r = await self._client.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.ReadTimeout
LGH531 commented 1 year ago

It appears that the 2 other open tickets are also related to this issue. I also think that Jandy/ iAqualink have made a change that has caused things to break. My integration was working well last week, and so I restored HA back to a good known state. The issue still persists though.

smshapiro85 commented 1 year ago

I did notice that they recently updated their app interface, so it is certainly possible that the issue is on the Jandy/iAqualink side. It is frustrating that it works for a period of time, and then stops working.

markiper commented 1 year ago

I am observing the same behavior; seems to have started after Jandy pushed their new mobile app.

gregbm99 commented 1 year ago

I also have the same issue. Also nothing is recorded in Logbook when a Jandy switch is open/closed

PRGeno commented 1 year ago

Updated to 2023.2.4 which supposedly has this fix but I am still experiencing the same issue. iAqualink requires a restart to get it to start responding again. So it appears this issue is not resolved

kevinkahn commented 1 year ago

Same here. Upgraded to . 4 and I got the NO ERROR error message. Had to add a programmatic reload of the integration to be able to turn the pool lights on. Not fixed so bug should probably be reopened.

gregbm99 commented 1 year ago

Also have the same issue after upgrading. On a positive note my spa pump is no longer randomly starting.

markiper commented 1 year ago

Same here, updated to 2023.2.4 and the issue persists

Logger: homeassistant.components.websocket_api.http.connection
Source: components/iaqualink/utils.py:14
Integration: Home Assistant WebSocket API (documentation, issues)
First occurred: 8:55:39 AM (3 occurrences)
Last logged: 8:55:59 AM

[140030559352624] <ConnectionTerminated error_code:ErrorCodes.NO_ERROR, last_stream_id:961, additional_data:None>
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/h2/connection.py", line 224, in process_input
    func, target_state = self._transitions[(self.state, input_)]
KeyError: (<ConnectionState.CLOSED: 3>, <ConnectionInputs.SEND_HEADERS: 0>)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 106, in handle_async_request
    await self._send_request_headers(request=request, stream_id=stream_id)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 203, in _send_request_headers
    self._h2_state.send_headers(stream_id, headers, end_stream=end_stream)
  File "/usr/local/lib/python3.10/site-packages/h2/connection.py", line 766, in send_headers
    self.state_machine.process_input(ConnectionInputs.SEND_HEADERS)
  File "/usr/local/lib/python3.10/site-packages/h2/connection.py", line 228, in process_input
    raise ProtocolError(
h2.exceptions.ProtocolError: Invalid input ConnectionInputs.SEND_HEADERS in state ConnectionState.CLOSED

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 60, in map_httpcore_exceptions
    yield
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 353, in handle_async_request
    resp = await self._pool.handle_async_request(req)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 253, in handle_async_request
    raise exc
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection_pool.py", line 237, in handle_async_request
    response = await connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/connection.py", line 90, in handle_async_request
    return await self._connection.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpcore/_async/http2.py", line 139, in handle_async_request
    raise RemoteProtocolError(self._connection_error_event)
httpcore.RemoteProtocolError: <ConnectionTerminated error_code:ErrorCodes.NO_ERROR, last_stream_id:961, additional_data:None>

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 200, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1787, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1824, in _execute_service
    await cast(Callable[[ServiceCall], Awaitable[None]], handler.job.target)(
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 213, in handle_service
    await service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 680, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 968, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 720, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/light/__init__.py", line 571, in async_handle_light_on_service
    await light.async_turn_on(**filter_turn_on_params(light, params))
  File "/usr/src/homeassistant/homeassistant/components/iaqualink/__init__.py", line 192, in wrapper
    await func(self, *args, **kwargs)
  File "/usr/src/homeassistant/homeassistant/components/iaqualink/light.py", line 65, in async_turn_on
    await await_or_reraise(self.dev.turn_on())
  File "/usr/src/homeassistant/homeassistant/components/iaqualink/utils.py", line 14, in await_or_reraise
    await awaitable
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/device.py", line 181, in turn_on
    await self.set_effect_by_id(1)
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/device.py", line 221, in set_effect_by_id
    await self.system.set_light(data)
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/system.py", line 178, in set_light
    r = await self._send_session_request(IAQUA_COMMAND_SET_LIGHT, data)
  File "/usr/local/lib/python3.10/site-packages/iaqualink/systems/iaqua/system.py", line 72, in _send_session_request
    return await self.aqualink.send_request(url)
  File "/usr/local/lib/python3.10/site-packages/iaqualink/client.py", line 98, in send_request
    r = await self._client.request(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1533, in request
    return await self.send(request, auth=auth, follow_redirects=follow_redirects)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1620, in send
    response = await self._send_handling_auth(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1648, in _send_handling_auth
    response = await self._send_handling_redirects(
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1685, in _send_handling_redirects
    response = await self._send_single_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_client.py", line 1722, in _send_single_request
    response = await transport.handle_async_request(request)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 352, in handle_async_request
    with map_httpcore_exceptions():
  File "/usr/local/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/site-packages/httpx/_transports/default.py", line 77, in map_httpcore_exceptions
    raise mapped_exc(message) from exc
httpx.RemoteProtocolError: <ConnectionTerminated error_code:ErrorCodes.NO_ERROR, last_stream_id:961, additional_data:None>
Vortec4800 commented 1 year ago

See #88033 for more info related to the recurrence.

MartinHjelmare commented 1 year ago

Ok, let's track the issue in #88033 instead. Thanks!

wheindl commented 1 year ago

I am observing the same behavior; seems to have started after Jandy pushed their new mobile app.

That is an interesting observation, something I hadn't noticed.