[bug] core simulator web ui not working with media_player sound_mode_list or source_list - real remote web ui working fine

[nikos1988]

Is there an existing issue for this?

• [X] I have searched the existing issues

Description

It looks like the core simulator web ui does not support sound_mode_list and source_list of the media_player entity. The real remote with current firmware does support it.

This has been discovered using the python integration library and a minimal example.

How to Reproduce

1. Use python integration library to create a media_player entity with source_list and sound_mode_list (see attached file) hello_integration.py.txt
2. Add integration to core simulator and try to find source list command via web ui

Expected behavior

• See source and sound_mode commands and the values from entity attributes.
• Works as expected with the real remote.

System version

No response

What part of the system affected by the problem?

No response

Additional context

No response

[zehnm]

Which versions are you using? Did you configure the media-player entity in the Simulator before adding the additional features? Added features in the integration after configuring the entity, will not show the newly added features.

I can't reproduce this and the simulator uses the same code base for handling entities as the remote. Your example in the Simulator within an activity shows the input source and sound mode commands:

Why it doesn't show the dropdown list with the predefined values is another issue :-)

[nikos1988]

Well, sorry for wasting your time. I only updated the images with "docker-compose pull" and forgot to recreate the containers. So my image was 5 month old. It now works fine :)

Also, i had the same issue with the predefined list values not showing on my real remote. A reboot somehow made them available on my real remote.

Restarting the core-simulator container after adding the entity also brings up the list with the predefined source and sound mode values.

Should i create another ticket for this minor bug?

[zehnm]

No worries. I've looked into the empty list issue, and it's multiple things: The lists not showing is due to a missing get_entity_states request from the remote. This request is currently only triggered after waking up from standby (Simulator is never in standby), or after the integration sends a device_state event with state: CONNECTED.

Another issue is the sequence of request messages the remote is supposed to send. That's how it should work, first a subscribe_events then get_entity_states: https://github.com/unfoldedcircle/core-api/blob/main/doc/integration-driver/write-integration-driver.md#normal-operation

The Python integration library moves the entities from available to configured if it receives a subscribe_events request. The get_entity_states operates only on the configured entities.

Looking at the Python example logs, the subscribe_events is received after get_entity_states:

DEBUG:ucapi.api:[('10.0.0.107', 57733)] ->: {"kind": "resp", "req_id": 0, "code": 200, "msg": "authentication", "msg_data": {}}

--> TRIGGERED BY authentication event above (sync send #2) to check for updated driver.
DEBUG:ucapi.api:[('10.0.0.107', 57733)] <-: {"id":11,"kind":"req","msg":"get_driver_version"}
DEBUG:ucapi.api:[('10.0.0.107', 57733)] ->: {"kind": "resp", "req_id": 11, "code": 200, "msg": "driver_version", "msg_data": {"name": "Hello Python integration", "version": {"api": "0.20.0", "driver": "0.0.1"}}}

--> TRIGGERED BY authentication event above (sync send #3)
DEBUG:ucapi.api:[('10.0.0.107', 57733)] <-: {"id":12,"kind":"event","msg":"connect"}
DEBUG:ucapi.api:[('10.0.0.107', 57733)] ->: {"kind": "event", "msg": "device_state", "msg_data": {"state": "CONNECTED"}, "cat": "DEVICE"}

--> TRIGGERED BY get_driver_version response IF version doesn't match to retrieve new metadata
DEBUG:ucapi.api:[('10.0.0.107', 57733)] <-: {"id":13,"kind":"req","msg":"get_driver_metadata"}
DEBUG:ucapi.api:[('10.0.0.107', 57733)] ->: {"kind": "resp", "req_id": 13, "code": 200, "msg": "driver_metadata", "msg_data": {"driver_id": "hello_integration", "version": "0.0.1", "min_core_api": "0.20.0", "name": {"en": "Hello Python integration"}, "icon": "uc:integration", "description": {"en": "Minimal Python integration driver example."}, "port": 9080, "developer": {"name": "Unfolded Circle ApS", "email": "hello@unfoldedcircle.com", "url": "https://www.unfoldedcircle.com"}, "home_page": "https://www.unfoldedcircle.com", "release_date": "2023-10-30"}}

--> TRIGGERED BY device_state event with state: CONNECTED 
DEBUG:ucapi.api:[('10.0.0.107', 57733)] <-: {"id":14,"kind":"req","msg":"get_entity_states"}
DEBUG:ucapi.api:[('10.0.0.107', 57733)] ->: {"kind": "resp", "req_id": 14, "code": 200, "msg": "entity_states", "msg_data": []}

--> TRIGGERED BY authentication event above (async send #1)
DEBUG:ucapi.api:[('10.0.0.107', 57733)] <-: {"kind":"req","id":15,"msg":"subscribe_events","msg_data":{"entity_ids":["test1"]}}
DEBUG:ucapi.entities:[configured] entity added: 'test1'
DEBUG:ucapi.api:[('10.0.0.107', 57733)] ->: {"kind": "resp", "req_id": 15, "code": 200, "msg": "result", "msg_data": {}}

Issue: the subscribe_events msg should be sent first, then get_driver_version and connect. (Reason: the first message is sent asynchronously, ends up in a queue and is overtaken by the the other messages sent synchronously...)

Since the messages are swapped, the get_entity_states response is empty and the remote / simulator doesn't get the entity attributes with the list values.

I'll fix the message sequence and retest all integrations. We can keep this issue open until it's fixed in the Simulator.

We might also change the get_available_entities behaviour to send all entity attributes back. They are currently excluded, but it could make sense to already include them.

[zehnm]

Message sequence order is fixed in core-simulator v0.39.10, which resulted in empty drop-down lists.

[zehnm]

Fixed message sequence order is included in the just released version 1.6.4