pydantic.error_wrappers.ValidationError: 1 validation error for DeviceResponse

[lellky]

So I recently upgraded HA to latest version and since then I get errors from tradfri in my log https://github.com/home-assistant/core/issues/106481. Then I forked this repo to try to understand what was could be the error, and this happened.

Where are you using pytradfri (eg stand-alone, Home Assistant etc)

stand-alone

Version of pytradfri

latest on master

Backend used (aiocoap, libcoap)

libcoap_api

What type of environment are you in, eg Dockerized, Raspberry PI, VirtualBox, WiFi network (give as much details as possible)

Docker, as per the docs

Expected behaviour

It would work without raised exceptions

Actual behaviour

I get an exception:

root@docker-desktop:/usr/src/app# python3 -i -m pytradfri 192.168.2.110
DEBUG:pytradfri.util:JSON file not found: tradfri_standalone_psk.conf
Please provide the 'Security Code' on the back of your Tradfri gateway: SECRET
DEBUG:pytradfri.util:JSON file not found: tradfri_standalone_psk.conf
DEBUG:pytradfri.api.libcoap_api:Executing 192.168.2.110 post ['15011', '9063']: {'9090': '671fd3a5f4d74b42aa72989a2aa9ab47'}
DEBUG:pytradfri.api.libcoap_api:Received: {"9091":"SECRET","9029":"1.21.0031"}
Generated PSK:  GENERATED_PSK
DEBUG:pytradfri.api.libcoap_api:Executing 192.168.2.110 get ['15001']
DEBUG:pytradfri.api.libcoap_api:Received: [65626,65640,65538,65639,65644,65622,65619,65637,65647,65628,65631,65655,65641,65654,65645,65657,65608,65658,65638,65650,65649,65624,65652,65600,65560,65559,65558,65557,65646,65561,65580,65616,65537,65547,65592,65594,65610,65648,65653,65605,65601,65543,65563,65614,65593,65546,65660,65584,65609,65598,65583,65548,65627,65632,65642,65621,65651,65636,65603,65602,65582,65549,65613,65623]
DEBUG:pytradfri.api.libcoap_api:Executing 192.168.2.110 get ['15001', 65626]
DEBUG:pytradfri.api.libcoap_api:Received: {"9001":"TRADFRI on/off switch 20","9003":65626,"9002":1612010457,"9020":1651148754,"9054":0,"9019":1,"3":{"0":"IKEA of Sweden","1":"TRADFRI on/off switch","2":"","3":"2.3.079","6":3,"7":4549,"8":0,"9":87},"5750":0,"15009":[{"9003":0}]}
DEBUG:pytradfri.api.libcoap_api:Executing 192.168.2.110 get ['15001', 65640]
DEBUG:pytradfri.api.libcoap_api:Received: {"9001":"TRADFRI on/off switch 25","9003":65640,"9002":1638114646,"9020":1693354814,"9054":0,"9019":1,"3":{"0":"IKEA of Sweden","1":"TRADFRI on/off switch","2":"","3":"2.3.079","6":3,"7":4549,"8":0,"9":87},"5750":0,"15009":[{"9003":0}]}
DEBUG:pytradfri.api.libcoap_api:Executing 192.168.2.110 get ['15001', 65538]
DEBUG:pytradfri.api.libcoap_api:Received: {"9001":"TRADFRI on/off switch","9003":65538,"9002":1575482816,"9020":1703345115,"9054":0,"9019":1,"3":{"0":"IKEA of Sweden","1":"TRADFRI on/off switch","2":"","3":"24.4.6","6":3,"7":4549,"8":0,"9":20},"5750":0,"15009":[{"9003":0}]}
DEBUG:pytradfri.api.libcoap_api:Executing 192.168.2.110 get ['15001', 65639]
DEBUG:pytradfri.api.libcoap_api:Received: {"9001":"TRADFRI on/off switch 22","9003":65639,"9002":1638114398,"9020":1706473768,"9054":0,"9019":1,"3":{"0":"IKEA of Sweden","1":"TRADFRI on/off switch","2":"","6":3,"7":4549,"8":0,"9":50},"5750":0,"15009":[{"9003":0}]}
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/usr/src/app/pytradfri/__main__.py", line 79, in <module>
    devices = api(devices_commands)
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/pytradfri/api/libcoap_api.py", line 147, in request
    result = self._execute(api_command, timeout=timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/pytradfri/api/libcoap_api.py", line 124, in _execute
    api_command.process_result(_process_output(return_value, parse_json))
  File "/usr/src/app/pytradfri/command.py", line 62, in process_result
    self._result = self._process_result(result)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/app/pytradfri/gateway.py", line 141, in process_result
    return Device(result)
           ^^^^^^^^^^^^^^
  File "/usr/src/app/pytradfri/resource.py", line 40, in __init__
    self.raw = self._model_class(**raw)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/pydantic/main.py", line 341, in __init__
    raise validation_error
pydantic.error_wrappers.ValidationError: 1 validation error for DeviceResponse
3 -> 3
  field required (type=value_error.missing)

[MartinHjelmare]

The firmware version is missing in the device info response. That's unexpected. It should have an item with key "3".

{"0":"IKEA of Sweden","1":"TRADFRI on/off switch","2":"","6":3,"7":4549,"8":0,"9":50}

[lellky]

My gateway is running 1.21.31 fyi.

[MartinHjelmare]

I'd try running the script again, restarting the gateway and running the script again, removing and adding the on/off switch and running the script again.

[lellky]

This is really weird. In the Ikea app some entities are missing the firmware information as well!

[lellky]

So, after switching batteries in some devices and power toggle other devices, the ikea app finally reported a firmware version on all entities except one. That was a motion sensor. Removed it from the gateway and now it seems to work (no exceptions).

The issue could come back. Why is firmware version needed when getting device info, and why do we raise an exception when it's missing?

[ggravlingen]

When we designed handling the responses using pydantic, the firmware version was in the response from the gateway and I believe it should be as per IKEA's design. The exception is raised by pydantic as a part of its validation of the response.

We could set the firmware version as optional but it's probably better to make sure the devices are properly linked to the gateway.

[lellky]

How would one make sure the devices are properly linked to the gateway? I did nothing and suddenly it stopped working.

If it doesn't mess up the home assistant integration I would suggest adding the firmware as optional. Also, when some devices lack the firmware version the Ikea app doesn't crash.

[MartinHjelmare]

To be clear, this version of pytradfri that uses pydantic isn't used in Home Assistant yet. So any issues in Home Assistant is not because the pydantic validation we have implemented.

[lellky]

Thank you for the clarification.

My understanding is that the behaviour is the same in the two versions? I.e. if the problem occurs in the gateway, both versions (9 and 13) will raise an exception (KeyError: '3' in version 9).

[MartinHjelmare]

I think we can make the device firmware version optional. That information isn't required to communicate with the device or use it.

Note though that missing information that is expected for a device is a sign that something about the gateway or device in question isn't working correctly. Other things that are required for proper operation may be affected.

[lellky]

I think that is for the maintainers of this repo to decide. I'm just a regular user of trådfri and home assistant :)