Loading group members fail with validation error on `ATTR_DEVICE_INFO`

[Nosskirneh]

Where are you using pytradfri (eg stand-alone, Home Assistant etc)

stand-alone

Version of pytradfri

13.0.0

Backend used (aiocoap, libcoap)

aiocoap

What type of environment are you in, eg Dockerized, Raspberry PI, VirtualBox, WiFi network (give as much details as possible)

Raspberry Pi 2

Expected behaviour

Get all group members using these two functions: https://github.com/Nosskirneh/SmartRemoteControl/blob/master/IKEA.py#L91-L109.

Actual behaviour

Crashes with this traceback:

  [...]
  File "/srv/smartRemote/IKEA.py", line 93, in load_group_members
    self.group_members[group.id] = self.api(group.members())
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pytradfri/api/libcoap_api.py", line 147, in request
    result = self._execute(api_command, timeout=timeout)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pytradfri/api/libcoap_api.py", line 124, in _execute
    api_command.process_result(_process_output(return_value, parse_json))
  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pytradfri/command.py", line 62, in process_result
    self._result = self._process_result(result)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pytradfri/gateway.py", line 144, in process_result
    return Device(result)
           ^^^^^^^^^^^^^^
  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pytradfri/resource.py", line 40, in __init__
    self.raw = self._model_class(**raw)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pydantic/main.py", line 341, in __init__
    raise validation_error
pydantic.error_wrappers.ValidationError: 1 validation error for DeviceResponse
3 -> 3
  field required (type=value_error.missing)

As far as I can see, 3 is ATTR_DEVICE_INFO, but I don't know why my group member doesn't have this.

This started happening after replacing the batteries in my Trådfri remotes. I had to remove the lights and the groups and pair and add them again when doing so. Any ideas?

[Nosskirneh]

Upgrading pydantic to the latest version results in another error instead 🤔

  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pydantic/main.py", line 164, in __init__
    __pydantic_self__.__pydantic_validator__.validate_python(data, self_instance=__pydantic_self__)
pydantic_core._pydantic_core.ValidationError: 3 validation errors for GroupResponse
9054
  Field required [type=missing, input_value={'9001': 'Taklampa Vardag...[65561, 65562, 65563]}}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
5706
  Field required [type=missing, input_value={'9001': 'Taklampa Vardag...[65561, 65562, 65563]}}}, input_type=dict]
    For further information visit https://errors.pydantic.dev/2.5/v/missing
9039
  Input should be a valid string [type=string_type, input_value=0, input_type=int]
    For further information visit https://errors.pydantic.dev/2.5/v/string_type

[Nosskirneh]

Using these versions, it works perfectly fine (which I'll be using for the meantime, but I wouldn't want to fall behind in updates). Let me know if I can help in anyway.

pytradfri==9.0.0
pydantic==1.9.2

[ggravlingen]

Does it work if you set 'device_info: DeviceInfoResponse | None' in the device/init file?

It won't work under pedantic 2, it has to be v1.

[Nosskirneh]

Yup yup, I'm aware that pedantic needs to be v1. But now I'm starting to question my sanity... it just started working as expected again – could the gateway give a half empty response for deactivated devices some for some time period? Anyway, I'm closing this as it's no longer an issue for me. Thanks for the response though!

Edit: ah, didn't realize it upgraded to v2 in my second comment... I remember having to downgrade a few months back, but thought pytradfri just needed to pin the version. Maybe I was using a pytradfri version that didn't have the pinned version of pydantic? 🤔 I'm not too experienced with pip dependencies. Oh well, it works 🤷

[Nosskirneh]

Sorry for reopening this after quite some time... but it just happened out of the blue again. I'm not sure if it is a single device acting up (I can still control everything through HomeKit) or if all of them suddenly miss that device info field. I think the IKEA gateway and the lights will update their software automatically, so maybe that could have changed something...

Does it work if you set 'device_info: DeviceInfoResponse | None' in the device/init file?

The same thing happens. If I comment out the line, it moves on but instead crashes when it tries to do something as the device_info property is None:

  File "/srv/smartRemote/.venv/lib/python3.11/site-packages/pytradfri/device/__init__.py", line 193, in manufacturer
    return self.raw.manufacturer
           ^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'manufacturer'

I'm not familiar with the code base, so any hints would be greatly appreciated!

[ggravlingen]

Sounds like the response from the gateway is incomplete, which is not something this library handles well. We expect the response to have a certain shape, else there will be errors.

Are you experiencing this with all devices or just some? Would it be possible to remove and add the faulty device and see if that makes a difference?

[Nosskirneh]

And just when I was about to debug this some more it started working as expected again. Some days ago I found an unknown device in the the IKEA Home app, which I removed, so it might have been that one (it didn't start working directly afterwards though, but maybe the gateway has some delay in updating devices or something). Anyway, when this happens again I'll just wrap the call with a try catch and see if there's any device not working. Thanks for the response!