KeyError: "latestPulse"

[uphillbattle]

The problem

Since a few days ago, I get no information from Easee. I get a message about KeyError: "latestPulse". When enabling debugging, I also see messages about an SR 429 Connection error. Maybe a similar problem to #429?

config_entry-easee-ee456ed0020c4278578e43650b6df79a.json home-assistant_2024-09-09T07-29-01.728Z.log

Version of Easee integration having the issue?

easee-0.9.59

Version of Home Assistant Core having the issue?

2024.9.1

Anything in the logs that might be useful for us?

Please see attached files.

Additional information

Only a problem on my home instance of HA. Easee (0.9.59) on my cabin instance of HA (still running 2024.8.3) is running fine.

[olalid]

I do not think this has anything to do with the other issue you are referring too. From what I can see in the log file you sent, the connection to the cloud was not successful, so it never retrieves any data. Which in turn causes the "lastestPusle" KeyError. There is a PR that removes that error here https://github.com/nordicopen/easee_hass/pull/452 but I do not think that will help in any other way than suppressing that error.

If you turn on debug logging (see readme) and generate a new log file with a fresh restart, it would probably give us more to go on.

[uphillbattle]

The log seemed to include a bunch of personal information - tried to redact it.

home-assistant_easee_2024-09-10T08-44-29.268Z.log

[uphillbattle]

It does seem, from the log, that data are fetched from the API, but that the connection to Signal does not succeed?

No sensors become populated with other data than unknown.

[olalid]

To me it looks perfectly normal up to the point that the signalr connection is about to be established. The easee cloud then returns error 429 which means "Too Many Requests". Which seems odd since we are only sending one request, then waiting for increasing amount of time after each try, up to 300 seconds, so even if this happens once, it should eventually start working. Are you possibly using some other software/integration that is also using the easee API that could be causing the cloud to block you out? If not, I could ask Easee about it if you provide me with a serial number of one of your units, you can send it to me in mail at olal (a) plea.se.

Normally, when the signalr connection never is sucessfully established the integration should resort to polling (which works but is less responsive and slower), but it might be that this "lastestPulse" bug stops that from working correctly. So if you try to apply the PR I talked about above you might get some improvement. (https://github.com/nordicopen/easee_hass/pull/452) But again, the real problem seems to be that cloud does not allow you to connect the signalr stream.

[uphillbattle]

Thanks, I've sent you an email with a the serial numbers monitored by the Home Assistance instance in question.

I will try the PR mentioned.

[uphillbattle]

The PR works. As you said, without the SignalR connection, the integration is slow (due to resorting to polling, presumably), but it works. It would be great if you could ask Easee about the SignalR connection.

As mentioned above, I have two Home Assistant instances running the Easee integration and using the same login credentials:

• One at home (HA 2024.9.1) - the problem one, monitoring my two Easee chargers at home
• One at my cabin (HA 2024.8.3) - which does not have the SignalR problem, monitoring my Easee charger at the cabin

A third HA instance (running HA 2024.8.3) is located at home - for my father's Easee charger using his Easee credentials, so two HA instances at home with the same public IP.

Don't know if any of that matters.

I did disable the problematic Easee integration (at my home HA instance) for >24 hours to see if a lock-out would be lifted, but that didn't work.

[olalid]

I received your e-mail and have asked about it, so lets wait and see if there is any ideas from Easee what can be wrong.

Can you try to disable the installation using your fathers credentials for a while and then try to connect with yours? I would assume rate-limiting would not look only at public IP though, but who knows.

[uphillbattle]

Thanks! I’ve disabled the installation using my father’s credentials.

I did try, this afternoon, to go to https://streams.easee.com/hubs (I believe this is the signalr URI for easee?) and got Too Many Requests as a result, so it seems that my public IP is banned.

[olalid]

Well, I think the base URL for negotiation is this https://streams.easee.com/hubs/chargers/negotiate, but it is unclear in what stage the 429 error occurs since that is inside a separate lib that "magic" happens and what exact URL it is then using, it could be redirected elsewhere along the process, who knows. I get a 401 response if I try this one (unauthorised) but that makes sense since I am not even trying to log in... :) The one you listed above gives 404 (not found) as response for me. In any case, yes, it does sound like you have been banned/locked out for some reason, or perhaps some unintentional misconfiguration that locks you out from signalr. Strange though, that you are locked out only from the signalr stream, and not the API entirely I must say... I will let you know if Easee responds, I can not guarantee if/when that happens. Feel free to talk to their support also if you want.

[olalid]

@uphillbattle Easee is asking for your external IP adress to check if they can find anything in logs. Can you send another mail with that info?

[uphillbattle]

@uphillbattle Easee is asking for your external IP adress to check if they can find anything in logs. Can you send another mail with that info?

Thanks a lot for helping me out. I've sent you the IP address by email. I also contacted Easee support earlier today as you suggested (with the external IP address), but I haven't heard back yet.

[astrandb]

There are a couple of 429 errors in the attached HA log. The first is timestamped 2024-09-09 09:22:43.641 and the second 4 seconds later. The 150 s backoff doesn't seem to be in place. Is the SR start callled more than once?

[olalid]

Possibly, this fix might solve that https://github.com/nordicopen/pyeasee/pull/104

[olalid]

@uphillbattle are your external IP same as you sent me in mail earlier? Easee responded saying that they do not see any traffic from that IP at all currently.

[uphillbattle]

@olalid Yes, it hasn't changed. On the other hand, I didn't see any 429 error code on the latest HA restart. A snip from the debug log below. I'm not sure what I should expect. There is a WARNING there, but towards the end, there is also a very nice SignalR stream connected message.

2024-09-16 10:41:53.087 DEBUG (MainThread) [pyeasee.easee] Subscribing to EHxxxxxx
2024-09-16 10:41:53.087 DEBUG (MainThread) [pyeasee.easee] SR connect sleep 0
2024-09-16 10:41:53.087 DEBUG (MainThread) [pyeasee.easee] Subscribing to EHyyyyyy
2024-09-16 10:41:53.087 DEBUG (MainThread) [pyeasee.easee] Already connecting
2024-09-16 10:41:53.087 DEBUG (MainThread) [pyeasee.easee] SR connect loop
2024-09-16 10:41:53.087 DEBUG (MainThread) [pyeasee.easee] verify_updated_token: 2024-09-16 11:40:51.320441, 2024-09-16 10:41:53.087788, False
2024-09-16 10:41:53.469 WARNING (MainThread) [homeassistant.util.loop] Detected blocking call to load_default_certs with args (<ssl.SSLContext object at 0x7fd6046baf50>, <Purpose.SERVER_AUTH: _ASN1Object(nid=129, shortname='serverAuth', longname='TLS Web Server Authentication', oid='1.3.6.1.5.5.7.3.1')>) in /usr/local/lib/python3.12/ssl.py, line 713: context.load_default_certs(purpose) inside the event loop; This is causing stability issues. Please create a bug report at https://github.com/home-assistant/core/issues?q=is%3Aopen+is%3Aissue
For developers, please see https://developers.home-assistant.io/docs/asyncio_blocking_operations/#load_default_certs
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 223, in <module>
    sys.exit(main())
  File "/usr/src/homeassistant/homeassistant/__main__.py", line 209, in main
    exit_code = runner.run(runtime_conf)
  File "/usr/src/homeassistant/homeassistant/runner.py", line 189, in run
    return loop.run_until_complete(setup_and_run_hass(runtime_config))
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 674, in run_until_complete
    self.run_forever()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 641, in run_forever
    self._run_once()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1990, in _run_once
    handle._run()
  File "/usr/local/lib/python3.12/asyncio/events.py", line 88, in _run
    self._context.run(self._callback, *self._args)
  File "/usr/local/lib/python3.12/site-packages/pyeasee/easee.py", line 301, in _sr_connect_loop
    await self.sr_connection.run()
  File "/usr/local/lib/python3.12/site-packages/pysignalr/client.py", line 92, in run
    await self._transport.run()
  File "/usr/local/lib/python3.12/site-packages/pysignalr/transport/websocket.py", line 77, in run
    await self._loop()
  File "/usr/local/lib/python3.12/site-packages/pysignalr/transport/websocket.py", line 102, in _loop
    async for conn in connection_loop:
  File "/usr/local/lib/python3.12/site-packages/pysignalr/__init__.py", line 26, in __aiter__
    async with self as protocol:
  File "/usr/local/lib/python3.12/site-packages/websockets/legacy/client.py", line 629, in __aenter__
    return await self
  File "/usr/local/lib/python3.12/site-packages/websockets/legacy/client.py", line 647, in __await_impl_timeout__
    return await self.__await_impl__()
  File "/usr/local/lib/python3.12/site-packages/websockets/legacy/client.py", line 651, in __await_impl__
    _transport, _protocol = await self._create_connection()
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1149, in create_connection
    transport, protocol = await self._create_connection_transport(
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 1173, in _create_connection_transport
    transport = self._make_ssl_transport(
  File "/usr/local/lib/python3.12/asyncio/selector_events.py", line 83, in _make_ssl_transport
    ssl_protocol = sslproto.SSLProtocol(
  File "/usr/local/lib/python3.12/asyncio/sslproto.py", line 295, in __init__
    sslcontext = _create_transport_context(
  File "/usr/local/lib/python3.12/asyncio/sslproto.py", line 55, in _create_transport_context
    sslcontext = ssl.create_default_context()
  File "/usr/local/lib/python3.12/ssl.py", line 713, in create_default_context
    context.load_default_certs(purpose)

2024-09-16 10:41:53.875 DEBUG (MainThread) [pyeasee.easee] SignalR stream connected
2024-09-16 10:41:53.875 DEBUG (MainThread) [pyeasee.easee] Subscribing to EHxxxxxx
2024-09-16 10:41:53.876 DEBUG (MainThread) [pyeasee.easee] Subscribing to EHyyyyyy

[olalid]

That warning is a separate thing and same as this issue: https://github.com/nordicopen/easee_hass/issues/466 Your issue as stated here originally is most likely solved in latest release v0.9.60, so please update and test.

[uphillbattle]

Updated to v0.9.60 now and restarted. The warning is still there, but the original issue in this thread seems to have been resolved. Will therefore close this issue now. Thanks for your support!

[olalid]

Yes, that was expected. Thanks for confirming.