handling DTLS client reconnection after aiocoap server restart

[litinoveweedle]

Hello, as you are probably aware aiocoap is used in pytradfri API which is used in Homeassistant integration for IKEA Tradfri devices. Due to current abandon-ish state of both tradfri and pytradfri code I am trying to fix some long standing issues with its functionality. My knowledge is very limited in all aspects which are required for achieving such target so I am sorry for bothering you with something which is maybe not aiocoap issue at all.

this is related issue in HA repo - due to several issues being tracked there please start reading at given comment.

We observed, that after Ikea Tradfri GW (aiocoap server) is restarted Homeassistant is unable to restore communication with it, causing both devices commands and states monitoring to fail. This was observed with HA using aicoap version 0.4b3. It seems that as no exception is raised from aicoap to pytradfri it is not able to create new connection towards rebooted Tradfri GW and to reestablish communication. (pytradfri utilize DTLS using aiocoap tinydtls transport)

We also tried to use latest aiocoap version, which made situation little bit better - at least after GW restart new command are executed successfully - but async devices states monitoring is not working. This is in my limited knowledge because in current implementation it requires exception to be raised from aiocoap -> pytradfri -> HA to restart devices observation thread. But there is no exception like that so new connection to rebooted GW is never established. It seems that GW after restart is replying to old connection requests with:

TimeoutError: [Errno 110] Operation timed out

My question is: is there a way how pytradfri could be notified from the aiocoap (raising exception?) that current connection is not working anymore - so it can destroy it and create new one?

Thank you for any help.

[GCC 9.3.0]
aiocoap version: 0.4b3.post0
Modules missing for subsystems:
    dtls: everything there
    oscore: missing filelock
    linkheader: missing LinkHeader
    prettyprint: missing LinkHeader, termcolor
Python platform: linux
Default server transports:  tinydtls:tcpserver:tcpclient:tlsserver:tlsclient:ws:udp6
Selected server transports: tinydtls:tcpserver:tcpclient:tlsserver:tlsclient:ws:udp6
Default client transports:  tinydtls:tcpclient:tlsclient:ws:udp6
Selected client transports: tinydtls:tcpclient:tlsclient:ws:udp6
SO_REUSEPORT available (default, selected): True, True

[litinoveweedle]

I have no idea if my previous explanation is understood as there was no reaction, so I will try my best to rephrase it.

What is current error reporting mechanism of aiocoap library to report to client (implemented using this library), that COAP server was restarted, existing DTLS session lost and server will not continues to communicate with client anymore? Such trigger than could be used to destroy existing client connection to server and create new session including new DTLS handshake.

I am asking as it is currently observed that aiocoap client implementation in pytradfri doesn't receives any protocol exception/notification when such situation happens.

[chrysn]

I just didn't get to it over the holiday backlog yet. I thin I get the issue, trying to come up with sth over the weekend.

[chrysn]

What is current error reporting mechanism of aiocoap library to report to client (implemented using this library), that COAP server was restarted, existing DTLS session lost and server will not continues to communicate with client anymore?

To the application, this is ideally not shown at all (given that any state between the client and the server should be a matter of optimization, and it's the CoAP library's task to hide that and show only the abstract stateless mechanism).

aiocoap should transparently create a new DTLS session if an old one becomes unusable; that should raise an error flying out of the request in which it was discovered as well as any other requests currently active on the same DTLS connection (including observations). Subsequent requests sent after the error should establish a new connection, and ideally all use the same DTLS connection again.

That's the theory, at least -- the issues with pytradfri show that there's a bug in there that prevents the observations from going visibly dead when the DTLS connection fails, I'm looking into why that is.

[chrysn]

I've fixed a few missing pieces in the DTLS error handling, please try whether that solves your immediate issue.

Of the three error paths I've observed with the libcoap example server, two now behave well:

• If the server just goes away (eg. a kill -KILL pid-of-example-server) and stays down, the next request triggers an ICMP error, which ends both the triggering request and the pending observation.
• If the server shuts down regularly (eg. Ctrl-C-ing the libcoap server), it sends out a DTLS shutdown that ends any pending observation. (I think it even sends a final 4.04 on the individual observations before taking down the DTLS connection).

• Trouble is when the server is hard-killed and then restarted before a request goes out. In this situation, it appears that the libcoap DTLS server does not send a DTLS WTF packet (however it's called), but just silently ignores the request. Consequently, the client doesn't see anything wrong with the server, it just doesn't get responses. (I could add code that at least makes a CON retransmission timeout kill the transport, but neither am I sure that's a good idea architecturally, nor would it work with NON requests).

  What makes issues a bit harder is that as far as I've observed the DTLS connections never fall out of the WeakDict that's holding them, indicating some kind of practical memory leak (but we the whole mechanism shouldn't rely on that, especially because if they do start falling out of the WeakDict, I'd probably make them stick around for some time a la five minutes at least).

  If you could report on how this behaves at tradfri, this may help narrow down whether there's actually anything to do (if it works there, it may be just a bug in libcoap); if it doesn't, we'd have to think about when it's legitimate for the client to try to establish a new DTLS connection because the old one may-or-may-not-be broken.

[litinoveweedle]

First thank you very much for reply, all the information and especially for such fast fixes. I am going to test them ASAP.

aiocoap should transparently create a new DTLS session if an old one becomes unusable; that should raise an error flying out of the request in which it was discovered as well as any other requests currently active on the same DTLS connection (including observations). Subsequent requests sent after the error should establish a new connection, and ideally all use the same DTLS connection again.

Regarding your intention to handle COAP sessions mechanism in aiocoap bellow level of the implemented client, I saw several places where pytradfri is handling some issues on its level - specially when testing results of the the .request method. But that seems to be fine for me as in such case it is good idea to report error from aiocoap to pytradfri. In most cases pytradfri is handling it by dropping and creating new aiocoap instance and therefore recreating new connection (network timeouts, missing credentials, etc.). I think, that handling some of the issues will be required on the level above aiocoap anyway, as this shall depend on the indented client logic.

That's the theory, at least -- the issues with pytradfri show that there's a bug in there that prevents the observations from going visibly dead when the DTLS connection fails, I'm looking into why that is.

Just one thing which could be confusing is term "observation", definitely when reading both tickets (aiocoap and pytradfri). In pytradfri context it refers to HomeAssistant threaded mechanism to monitor IOT device states asynchronously. In aiocoap it is IMHO way to implement COAP sessions transparently in the state machine. So to put that into current situation, HA/pytradfri observation fails to work as aiocoap observation fails to catch erroneous state triggered by COAP server (Tradfri GW) restart.
Anyway based on our independent tests it realy seems that in some cases aiocoap is neither able to solve session lost caused by server restart, neither to report such state to client implementation (pytradfri).

Trouble is when the server is hard-killed and then restarted before a request goes out. In this situation, it appears that the libcoap DTLS server does not send a DTLS WTF packet (however it's called), but just silently ignores the request. Consequently, the client doesn't see anything wrong with the server, it just doesn't get responses. (I could add code that at least makes a CON retransmission timeout kill the transport, but neither am I sure that's a good idea architecturally, nor would it work with NON requests).

What makes issues a bit harder is that as far as I've observed the DTLS connections never fall out of the WeakDict that's holding them, indicating some kind of practical memory leak (but we the whole mechanism shouldn't rely on that, especially because if they do start falling out of the WeakDict, I'd probably make them stick around for some time a la five minutes at least).

If you could report on how this behaves at tradfri, this may help narrow down whether there's actually anything to do (if it works there, it may be just a bug in libcoap); if it doesn't, we'd have to think about when it's legitimate for the client to try to establish a new DTLS connection because the old one may-or-may-not-be broken.

I will validate how aiocoap now behaves against IKEA Tradfri GW server implementation and let you know ASAP. But as timing could be crucial condition I do not know how my tests will be accurate.

As described above, client implementation on the level above aiocoap (pytradfri) needs to handle some edge cases anyway, as not everything could by decided/handled in the session state machine in the aiocoap. So I think that at least possibility to triggering network timeout exception from aiocoap to pytradfri in such case (server restart) will help, maybe even straight away, as pytradfri will probably just drop existing aicocap instance and created new one. I would prefer such implementation over recreating transport in the aiocoap (as you mentioned) as I agree, this is not good place to be at.

Signing out for testing. ;-)

[chrysn]

On the topic of reconnecting and where to handle errors:

A network error does need to be handled by the application, simply because aiocoap can't (generally) be certain that "just trying again" is legal. Shutting all of aiocoap down and trying anew is certainly a way to deal with it, although I hope that (at least with the current improvements) just sending a new request should be fine in many cases. As pytradfri is expected to know the semantics of the operations, I think that pytradfri could already do that retry handling, and pass just a cleaned up "either the latest state of a clear statement that even with retries, that device is gone right now" up to HA.

Ad observation: Good point -- when I'm talking of observation here, it's always about the CoAP Observe mechanism. That mechanism does have a kind of inherent mechanism to indicate when it breaks (based on Max-Age) -- does the Tradfri GW use that consistently? (Ideally, it should sent a Max-Age option that indicates when it will next send a state, or send something every 60 seconds which is default for Max-Age, but I don't know if it does that).

On validating and timing: Timing should not be too crucial. If HA/pytradfri only uses observe, the reboot may take quite some time and it'd be OK (bringing us back to the Max-Age above). If they additionally do any polling, then that polling sets how much time there is for the restart.

[litinoveweedle]

A network error does need to be handled by the application, simply because aiocoap can't (generally) be certain that "just trying again" is legal. Shutting all of aiocoap down and trying anew is certainly a way to deal with it, although I hope that (at least with the current improvements) just sending a new request should be fine in many cases.

I fully agree that network, credentials etc error shall be handled by application. I did some first test bu it seems that retransmission mechanism doesn't work so far, I will post results bellow.

As pytradfri is expected to know the semantics of the operations, I think that pytradfri could already do that retry handling, and pass just a cleaned up "either the latest state of a clear statement that even with retries, that device is gone right now" up to HA.

I agree that pytradfri shall handle it as well, the only issue was, that so far no event was passed out from aiocoap and I don't think it is good idea to implement timers in the pytradfri to handle "no response" state.

Ad observation: Good point -- when I'm talking of observation here, it's always about the CoAP Observe mechanism. That mechanism does have a kind of inherent mechanism to indicate when it breaks (based on Max-Age) -- does the Tradfri GW use that consistently? (Ideally, it should sent a Max-Age option that indicates when it will next send a state, or send something every 60 seconds which is default for Max-Age, but I don't know if it does that).

Thank you, I think that we have now good understanding of the aiocoap/pytradfri/HA interactions. I have no idea if Tradfri GW uses Max-Age, and even worse I have no idea how to check it out. (as it is DTLS packet dump will not help). I will repeat my test with at least 60s window to see if something will triggers (so far all observed error where triggered only by new request from client)

On validating and timing: Timing should not be too crucial. If HA/pytradfri only uses observe, the reboot may take quite some time and it'd be OK (bringing us back to the Max-Age above). If they additionally do any polling, then that polling sets how much time there is for the restart.

Tradfri GW restart seems to be actually swift, I am able to use IOT devices again in less than 60s from HA, therefore I would assume Tradfri GW could reboot bellow 60s.

[litinoveweedle]

This is result from HA logs after:

• restart of the Tradfri GW
• issue of command to one IOT device

To me it seems that now smething being triggered but exceptions are not handled fully in aiocoap...? Also it seems as some race condition between aiocoap retransmission mechanism and pytradfri aiocoap exception handling.

ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Open/Close Remote Dinning
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Open/Close Remote Living
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 3 Living
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 1 Living
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind Bedroom
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 2 Living
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 2 Dinning
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 1 Dinning
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused
2021-01-09 15:20:35 INFO (MainThread) [coap] No DTLS connection active to (192.168.3.55, 5684, b'1bed0ae16434463d8758f8b1a451787f'), creating one
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65557')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65571')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65561')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65565')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65563')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65559')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65564')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65575')
2021-01-09 15:20:35 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65574')
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65557]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65557'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65571]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65571'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65561]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65561'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65565]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65565'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65563]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65563'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [139913901108096] 'DTLSClientConnection' object has no attribute '_retransmission_task'
ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 135, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1445, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1480, in _execute_service
    await handler.job.target(service_call)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service
    await self.hass.helpers.service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 593, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 664, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 630, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/tradfri/cover.py", line 50, in async_open_cover
    await self._api(self._device_control.set_state(0))
  File "/usr/src/homeassistant/homeassistant/components/tradfri/base_class.py", line 22, in wrapper
    await func(command)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 168, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 152, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 111, in _get_response
    await self._reset_protocol(e)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 78, in _reset_protocol
    await protocol.shutdown()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 322, in shutdown
    await item
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 54, in shutdown
    await self.token_interface.shutdown()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 71, in shutdown
    await self.message_interface.shutdown()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 384, in shutdown
    c.shutdown()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 223, in shutdown
    self._retransmission_task.cancel()
AttributeError: 'DTLSClientConnection' object has no attribute '_retransmission_task'
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65559]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65559'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65564]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65564'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65575]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65575'))
2021-01-09 15:20:35 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65574]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65574'))

• Subsequent command from HA trigers different errors:

2021-01-09 15:20:52 INFO (MainThread) [coap] No DTLS connection active to (192.168.3.55, 5684, b'1bed0ae16434463d8758f8b1a451787f'), creating one
2021-01-09 15:20:52 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: TypeError("'NoneType' object does not support item assignment")
2021-01-09 15:20:52 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback _SelectorDatagramTransport._read_ready()
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 1011, in _read_ready
    data, addr = self._sock.recvfrom(self.max_size)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 1015, in _read_ready
    self._protocol.error_received(exc)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 318, in error_received
    parent._inject_error(exc)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 248, in _inject_error
    self.coaptransport.ctx.dispatch_error(e.errno, self)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 129, in dispatch_error
    self.token_manager.dispatch_error(errno, remote)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 77, in dispatch_error
    for key, request in self.outgoing_requests.items():
AttributeError: 'NoneType' object has no attribute 'items'
2021-01-09 15:20:52 ERROR (MainThread) [homeassistant.components.websocket_api.http.connection] [139913901108096] 'NoneType' object does not support item assignment
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 135, in handle_call_service
    await hass.services.async_call(
  File "/usr/src/homeassistant/homeassistant/core.py", line 1445, in async_call
    task.result()
  File "/usr/src/homeassistant/homeassistant/core.py", line 1480, in _execute_service
    await handler.job.target(service_call)
  File "/usr/src/homeassistant/homeassistant/helpers/entity_component.py", line 204, in handle_service
    await self.hass.helpers.service.entity_service_call(
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 593, in entity_service_call
    future.result()  # pop exception if have
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 664, in async_request_call
    await coro
  File "/usr/src/homeassistant/homeassistant/helpers/service.py", line 630, in _handle_entity_call
    await result
  File "/usr/src/homeassistant/homeassistant/components/tradfri/cover.py", line 50, in async_open_cover
    await self._api(self._device_control.set_state(0))
  File "/usr/src/homeassistant/homeassistant/components/tradfri/base_class.py", line 22, in wrapper
    await func(command)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 168, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 152, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 345, in send
    request_interface.request(plumbing_request)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 232, in request
    self.outgoing_requests[key] = request
TypeError: 'NoneType' object does not support item assignment
2021-01-09 15:20:54 ERROR (MainThread) [homeassistant] Error doing job: Exception in callback _SelectorDatagramTransport._read_ready()
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 1011, in _read_ready
    data, addr = self._sock.recvfrom(self.max_size)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.8/asyncio/selector_events.py", line 1015, in _read_ready
    self._protocol.error_received(exc)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 318, in error_received
    parent._inject_error(exc)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 248, in _inject_error
    self.coaptransport.ctx.dispatch_error(e.errno, self)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 129, in dispatch_error
    self.token_manager.dispatch_error(errno, remote)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 77, in dispatch_error
    for key, request in self.outgoing_requests.items():
AttributeError: 'NoneType' object has no attribute 'items'
2021-01-09 15:21:05 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: TypeError("'NoneType' object does not support item assignment")
2021-01-09 15:21:05 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/tradfri/__init__.py", line 162, in async_keep_alive
    await api(gateway.get_gateway_info())
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 168, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 152, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 345, in send
    request_interface.request(plumbing_request)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 232, in request
    self.outgoing_requests[key] = request
TypeError: 'NoneType' object does not support item assignment

[chrysn]

Just to make sure as I see a few "connection refused" and "no credentials" errors:

• With the same setup but aiocoap of this morning (say, version e181700a93a2f8ee5778cc3c8022861cc9454aba), do things generally work?
• Which version (git commit hash) did you test this with? (Just because I've been pushing a few in the last hours)

[chrysn]

as it is DTLS packet dump will not help

When you receive the responses, you can look into response.opt.max_age -- that's either None or a number of seconds. Generally, when you do CoAP observations against a property of the GW, does it ever send updates if the state doesn't change / how often does it?

[litinoveweedle]

Just to make sure as I see a few "connection refused" and "no credentials" errors:

My previous test were using version Dec19 2020 (e181700). With this version acicoap and pytradfri where able to recover to state, when new IOT command were processed after GW restart, but no HA observations worked afterward.

Current test were done with code downloaded as zip using github download menu aprox. 1hr before (as far as I understand github this is master/HEAD), but I can check commit, as code is downloaded without git metadata. But I am going to reproduce tests with git clone of the repo now.

When you receive the responses, you can look into response.opt.max_age -- that's either None or a number of seconds. Generally, when you do CoAP observations against a property of the GW, does it ever send updates if the state doesn't change / how often does it?

I will do my best, but as I stated on the beginning my Python skills are ~0 (Perl person here), and pytradfri code lacks any support from previous developers. :-o

[litinoveweedle]

Ok, results seems to be same as in previous test (so it seems I was using latest code), so I also performed slightly different test:

• reboot Tradfri GW
• no command issued, only waited (for HA observation)

2021-01-09 16:41:29 INFO (MainThread) [coap] Retransmission, Message ID: 19302.
2021-01-09 16:41:33 INFO (MainThread) [coap] Retransmission, Message ID: 19302.
2021-01-09 16:41:43 INFO (MainThread) [coap] Retransmission, Message ID: 19302.
2021-01-09 16:42:02 INFO (MainThread) [coap] Retransmission, Message ID: 19302.
2021-01-09 16:42:41 INFO (MainThread) [coap] Exchange timed out trying to transmit <aiocoap.Message at 0x7f23f4cc63d0: Type.CON GET (MID 19302, token 4098) remote <aiocoap.transports.tinydtls.DTLSClientConnection object at 0x7f23fdc9d8e0>, 2 option(s)>
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Open/Close Remote Living
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Open/Close Remote Bedroom
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Open/Close Remote Dinning
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 2 Living
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 2 Dinning
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind Bedroom
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 1 Living
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 1 Dinning
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [homeassistant.components.tradfri.base_class] Observation failed for Blind 3 Living
TimeoutError: [Errno 110] Operation timed out

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65561')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65557')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65571')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65564')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65575')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65559')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65563')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65574')
2021-01-09 16:42:41 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65565')
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65571]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65571'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri] Keep-alive failed
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri] Keep-alive failed
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65561]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65561'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65557]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65557'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65564]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65564'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65575]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65575'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65559]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65559'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65563]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65563'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65574]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65574'))
2021-01-09 16:42:41 ERROR (MainThread) [homeassistant.components.tradfri.base_class] Unable to execute command <Command get ['15001', 65565]>: ('There was an error with the request.', CredentialsMissingError('No suitable credentials for coaps://192.168.3.55/15001/65565'))
2021-01-09 16:43:26 INFO (MainThread) [coap] No DTLS connection active to (192.168.3.55, 5684, b'1bed0ae16434463d8758f8b1a451787f'), creating one

• as I saw the last message I tested to execute IOT command - it fired one more log message (bellow) and it actually worked. But status of IOT devices is still not updated afterwards in HA - probably due to some unhadled exception HA observation thread died???

2021-01-09 16:44:23 INFO (MainThread) [coap] No DTLS connection active to (192.168.3.55, 5684, b'1bed0ae16434463d8758f8b1a451787f'), creating one

So it seems to works (but it was working similarly with version from Dec17 but without any exceptions) but:

• some exception seems aren't handled properly (where???)
• results are different if IOT command reqest is sent before timeout in aiocoap or after it.
• following command are slow being passed to device compared to situation before any exception
• some exceptions keeps to continue to popup from now on:

2021-01-09 17:06:16 WARNING (MainThread) [coap] An exception that is not an aiocoap Error was raised from a transport; please report this as a bug in aiocoap: TypeError("'NoneType' object does not support item assignment")
2021-01-09 17:06:16 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/components/tradfri/__init__.py", line 162, in async_keep_alive
    await api(gateway.get_gateway_info())
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 168, in request
    result = await self._execute(api_commands)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 152, in _execute
    _, res = await self._get_response(msg)
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 345, in send
    request_interface.request(plumbing_request)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 232, in request
    self.outgoing_requests[key] = request
TypeError: 'NoneType' object does not support item assignment
Exception ignored in: 'DTLSSocket.dtls._read'
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 266, in _read
    self.coaptransport.ctx.dispatch_message(message)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 109, in dispatch_message
    success = self._process_response(message)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 326, in _process_response
2021-01-09 17:06:16 ERROR (MainThread) [root] Uncaught exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/transports/tinydtls.py", line 266, in _read
    self.coaptransport.ctx.dispatch_message(message)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 109, in dispatch_message
    success = self._process_response(message)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/messagemanager.py", line 326, in _process_response
    return self.token_manager.process_response(response)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 147, in process_response
    if key not in self.outgoing_requests:
TypeError: argument of type 'NoneType' is not iterable
    return self.token_manager.process_response(response)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/tokenmanager.py", line 147, in process_response
    if key not in self.outgoing_requests:
TypeError: argument of type 'NoneType' is not iterable

• after decribed tests performed there are new aiocoap errors triggered by later HA shutdown:

Client error on /homeassistant/restart request Server disconnected
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  [Previous line repeated 1 more time]
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  [Previous line repeated 2 more times]
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  [Previous line repeated 3 more times]
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  [Previous line repeated 4 more times]
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  [Previous line repeated 5 more times]
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out
Exception ignored in: <function ClientObservation._Iterator.__del__ at 0x7f240045dc10>
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1093, in __del__
    self._future.result()
  [Previous line repeated 6 more times]
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/pytradfri/api/aiocoap_api.py", line 96, in _get_response
    r = await pr.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 823, in _run_outer
    await cls._run(app_request, response, weak_observation, protocol, log)
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 875, in _run
    blockresponse = await blockrequest.response
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 976, in _run_observation
    async for block1_notification in lower_observation:
  File "/usr/local/lib/python3.8/site-packages/aiocoap/protocol.py", line 1074, in __anext__
    result = await self._future
aiocoap.error.NetworkError: [Errno 110] Operation timed out

[chrysn]

OK, I think I'll have to try to reproduce this locally. I have a tradfri GW and lamp around but haven't used it much yet; can this whole trouble be also observed using pytradfri and no HA? (It'd just help making the whole setup a lot more managable.)

[litinoveweedle]

OK, I think I'll have to try to reproduce this locally. I have a tradfri GW and lamp around but haven't used it much yet; can this whole trouble be also observed using pytradfri and no HA? (It'd just help making the whole setup a lot more managable.)

I understand that it would help significantly, but unfortunately my skills are not up to the task, I am very sorry. :-o

[chrysn]

I'll give it a try anyway (but could take some time), thanks for bringing all this to my attention.

[wrobell]

I am using master of aiocoap (1af4c3f) and face the reconnection issues IKEA Tradfri gateway as well.

To connect to the gateway I am using the script in #236:

1. Start the script.
2. Trigger state change of a device to see that the script is working.
3. Disable wifi on the laptop running the script.
4. Wait few seconds.
5. Enable the wifi on the laptop again.
6. Trigger state change of a device - neither data nor error is reported by the script.

Once the script is restarted, then everything works fine again.

[wrobell]

Also, there might be a more generic problem here.

1. Start "server.py" provided in the repository.
2. Run "aiocoap-client --observe coap://localhost/time".
3. Wait few seconds to see the time being reported.
4. Stop "server.py" and start it again.
5. The client does not report timestamps anymore.

[wrobell]

Checking with few Tradfri devices, the max age sent by the gateway is 604800 seconds (168 hours or a week).

According to https://datatracker.ietf.org/doc/html/rfc7641#section-3.3.1

To make sure it has a current representation and/or to re-register its interest in a resource, a client MAY issue a new GET request with the same token as the original at any time. All options MUST be identical to those in the original request except for the set of ETag Options. It is RECOMMENDED that the client does not issue the request while it still has a fresh notification/response for the resource in its cache. Additionally, the client SHOULD at least wait for a random amount of time between 5 and 15 seconds after Max-Age expired to reduce collisions with other clients.

By default, aiocoap could wait the recommended time and re-register observation with the server. However, it could be made configurable and an end-user of an application could decide the timeout value for an observation.

[anyc]

Is there a workaround for this? I am seeing the same issue.