Platform rest does not generate unique IDs

[Mariusthvdb]

The problem

upon reloading Rest entities via the yaml menu,(eg after a config change, or adding a new sensor) below error is returned.

All rest sensor entities in my config have a unique_id. Somehow the error seems to indicate the system wants to re-create these.

What version of Home Assistant Core has the issue?

2023.3.6

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Sensor , sensor rest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/sensor.rest https://www.home-assistant.io/integrations/sensor

Diagnostics information

No response

Example YAML snippet

rest:

  - resource: !secret solar_forecast_api
    scan_interval: 86400
    sensor:

      - unique_id: solar_forecast_estimate_watt_hours_day
        name: Solar Forecast estimate watt hours day
        icon: mdi:calendar-today
        unit_of_measurement: Wh
        value_template: >
          {% set today = now().strftime('%Y-%m-%d') %}
          {{value_json.result.watt_hours_day[today]}}
        json_attributes_path: "$.result"
        json_attributes: watt_hours_day

      - unique_id: solar_forecast_estimate_message
        name: Solar Forecast estimate message
        icon: mdi:message-text-outline
        value_template: >
          {% set type = value_json.message.type %}
          {{type if type == 'success' else value_json.message.text}}
        json_attributes_path: "$.message"
        json_attributes:
          - code
          - type
          - text
          - info
          - ratelimit

Anything in the logs that might be useful for us?

Logger: homeassistant.components.sensor
Source: helpers/entity_platform.py:540 
Integration: Sensor (documentation, issues) 
First occurred: 09:03:25 (11 occurrences) 
Last logged: 09:03:27

Platform rest does not generate unique IDs. ID solar_forecast_estimate_watt_hours_day already exists - ignoring sensor.solar_forecast_estimate_watt_hours_day
Platform rest does not generate unique IDs. ID solar_forecast_estimate_message already exists - ignoring sensor.solar_forecast_estimate_message
Platform rest does not generate unique IDs. ID maan_astronomy already exists - ignoring sensor.maan_astronomy
Platform rest does not generate unique IDs. ID knmi_waarschuwingen_nb already exists - ignoring sensor.knmi_waarschuwingen_nb
Platform rest does not generate unique IDs. ID knmi_waarschuwingen_basis already exists - ignoring sensor.knmi_waarschuwingen_basis

Additional information

No response

[home-assistant[bot]]

rest documentation rest source

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[tribut]

Seeing the same issue.

[derMicro]

same issue here Version 2023.9.1

Logger: homeassistant.components.sensor Source: helpers/entity_platform.py:610 Integration: Sensor (documentation, issues) First occurred: 22:36:07 (8 occurrences) Last logged: 22:36:10

Platform rest does not generate unique IDs. ID sbbtimetableall already exists - ignoring sensor.sbbtimetableall Platform rest does not generate unique IDs. ID bustimetableall already exists - ignoring sensor.bustimetableall Platform rest does not generate unique IDs. ID weatherdatafrom8805wae already exists - ignoring sensor.weatherdatafrom8805wae Platform rest does not generate unique IDs. ID weatherdatafrom8805waeforecast already exists - ignoring sensor.weatherdatafrom8805waeforecast Platform rest does not generate unique IDs. ID weatherdatafrom8004sma already exists - ignoring sensor.weatherdatafrom8004sma

` - platform: rest name: WeatherDatafrom8805WAE unique_id: weatherdatafrom8805wae json_attributes_path: $.. scan_interval: 120 json_attributes:

• Current
• Forecast

• Condition resource: http://harestapi.xxx.xx/api/swissmeteo/8805/WAE value_template: "{{ value_json.Condition }}" icon: mdi:weather-sunny`

  Also not all REST sensors are listed, every day there are different ones, but all of them have a unique_id

[coserotondo]

Same for me. Version 2023.10.1

[eyalpost]

Same here. Any new about this issue?

Logger: homeassistant.components.sensor
Source: helpers/entity_platform.py:610
Integration: Sensor 
First occurred: 21:39:01 (6 occurrences)
Last logged: 21:47:51

Platform rest does not generate unique IDs. ID xxx already exists - ignoring sensor.xxx
Platform rest does not generate unique IDs. ID xxx2 already exists - ignoring sensor.xxx2
Platform rest does not generate unique IDs. ID xxx_test already exists - ignoring sensor.xxx_test
Platform rest does not generate unique IDs. ID xxx2_test already exists - ignoring sensor.xxx2_test

[kernel610]

Same for me. Version 2023.10.1

Also experiencing this on 2023.10.1. Pulling my hair out trying to figure out why my REST sensors and then saw this ignored message in the log! Removing the unique_id creates duplicates each time the REST sensors reload.

[kernel610]

Is there another way to pull REST integrations without creating duplicate ids for the sensor data coming back? I'm surprised this isn't a bigger problem so I assume there may be a different way to do this?

[PacmanForever]

Same here.

It doesn't cause me any problems, but the traces are rough.

[jesseuitto]

I had this same also. And it did cause me problems because these sensors were unavailabe about 6 hours today. But then suddenly they just started to work. I did try to reload yaml earlier, it didn't help. When these started to work, I didn't do anything at that time, it happened itself.

My yaml:

rest:
- resource: http://xxx/api/status
    scan_interval: 5
    method: GET
    sensor:
      - name: Kiinteistön Kokonaiskulutus
        unique_id: goe1_Kulutus_All
        value_template: "{{ (value_json.cec.0.0 / 1000) | round(2) }}"
        device_class: "energy"
        unit_of_measurement: "kWh"
      - name: Auton lataus yht.
        unique_id: goe1_Kulutus_Autolaturi
        value_template: "{{ (value_json.cec.2.0 / 1000) | round(2) }}"
        device_class: "energy"
        unit_of_measurement: "kWh"
      - name: Teho yhteensä
        unique_id: goe1_Teho_yht
        value_template: "{{ (value_json.ccp.0 / 1000) | round(2) }}"
        device_class: "power"
        unit_of_measurement: "kW"
      - name: Teho Autonlataus
        unique_id: goe1_Teho_Autolaturi
        value_template: "{{ (value_json.ccp.2 / 1000) | round(2) }}"
        device_class: "power"
        unit_of_measurement: "kW"
      - name: Virta L1
        unique_id: goe1_Virta_L1
        value_template: "{{ value_json.isv.0.i | round(2) }}"
        device_class: "current"
        unit_of_measurement: "A"
      - name: Virta L2
        unique_id: goe1_Virta_L2
        value_template: "{{ value_json.isv.1.i | round(2) }}"
        device_class: "current"
        unit_of_measurement: "A"
      - name: Virta L3
        unique_id: goe1_Virta_L3
        value_template: "{{ value_json.isv.2.i | round(2) }}"
        device_class: "current"
        unit_of_measurement: "A"
      - name: Teho L1
        unique_id: goe1_Teho_L1
        value_template: "{{ (value_json.isv.0.p / 1000) | round(2) }}"
        device_class: "power"
        unit_of_measurement: "kW"
      - name: Teho L2
        unique_id: goe1_Teho_L2
        value_template: "{{ (value_json.isv.1.p / 1000) | round(2) }}"
        device_class: "power"
        unit_of_measurement: "kW"
      - name: Teho L3
        unique_id: goe1_Teho_L3
        value_template: "{{ (value_json.isv.2.p / 1000) | round(2) }}"
        device_class: "power"
        unit_of_measurement: "kW"

This last logged @ 1:46 PM is the time I tried to manually reload my yaml configs:

Logger: homeassistant.components.sensor Source: helpers/entity_platform.py:610 Integration: Sensor (documentation, issues) First occurred: November 24, 2023 at 11:10:35 PM (60 occurrences) Last logged: 1:46:16 PM

Platform rest does not generate unique IDs. ID goe1_Virta_L2 already exists - ignoring sensor.virta_l2 Platform rest does not generate unique IDs. ID goe1_Virta_L3 already exists - ignoring sensor.virta_l3 Platform rest does not generate unique IDs. ID goe1_Teho_L1 already exists - ignoring sensor.teho_l1 Platform rest does not generate unique IDs. ID goe1_Teho_L2 already exists - ignoring sensor.teho_l2 Platform rest does not generate unique IDs. ID goe1_Teho_L3 already exists - ignoring sensor.teho_l3

Then it started to work around 4:41 PM itself.

Also you might notice it doesn't tell this from ALL my sensors. For some reason only about half of them are warned, others are working fine. Example my sensor Teho Yhteensä, unique id: _goe1_Tehoyht has worked perfectly entire day:

[sstidl]

still here in 2023.12.0

[BenDavidson90]

Same here: 2023-12-19 00:35:22.245 ERROR (MainThread) [homeassistant.components.sensor] Platform rest does not generate unique IDs. ID 7f1bec6b-d31f-4a62-b86d-0d8d3686e7b8 already exists - ignoring sensor.###

Here is my YAML:

- platform: rest
  name: "###"
  unique_id: "7f1bec6b-d31f-4a62-b86d-0d8d3686e7b8"
  resource_template: "https://#####"
  headers:
    X-Api-Token: !secret ###_api_key
  value_template: "{{ value_json.status }}"
  json_attributes:
    - pump
    - status
    - changed_at
  scan_interval: 5

This ID is unique and not used elsewhere.

HA 2023.12.3

[jesmak]

Happens to me also, HA 2023.12.0

[bonifazkaufmann]

Getting the same error. Platform unifiprotect does not generate unique IDs. ID 68D79AE131A9_0 already exists - ignoring camera.cam_m_high As a result, the camera live image does not work. Deleting the integration and readding it, has not solved the issue.

HA Core 2023.12.4

[purcell-lab]

I have found these error message go away if I don't specify a unique_id for REST sensors.

[jurw2201]

Same for me. Version 2023.10.1

Also experiencing this on 2023.10.1. Pulling my hair out trying to figure out why my REST sensors and then saw this ignored message in the log! Removing the unique_id creates duplicates each time the REST sensors reload.

And each duplicate calls the REST API it seems.

[issue-triage-workflows[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates. Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[PacmanForever]

keep alive...

[jesseuitto]

I do confirm this problems still exists. I have documented problems earlier in this thread.

[timr49]

Still occurs with today's (3 May 2024) versions, namely:

Core 2024.5.0 Supervisor 2024.04.4 Operating System 12.2 Frontend 20240501.0

From my log. Note that all sensors' referred to do have unique_id's configured in their templates.

Logger: homeassistant.components.sensor Source: helpers/entity_platform.py:744 integration: Sensor (documentation, issues) First occurred: 9:24:32 PM (43 occurrences) Last logged: 9:25:02 PM

Platform rest does not generate unique IDs. ID solax1_sn already exists - ignoring sensor.solax1_sn Platform rest does not generate unique IDs. ID solax1_status already exists - ignoring sensor.solax1_status Platform rest does not generate unique IDs. ID solax2_type already exists - ignoring sensor.solax2_type Platform rest does not generate unique IDs. ID solax2_sn already exists - ignoring sensor.solax2_sn Platform rest does not generate unique IDs. ID solax2_status already exists - ignoring sensor.solax2_status

[markwoodward86]

doing my part to keep this thread alive, issue persists

[TTLucian]

Same issue here! HA ver 2024.6.0

[alekslegkih]

Nobody was able to fix the problem? HA 2024.6.2

[kobraevents]

had this issue since 2023 and still in 2024.6.0. in my case its a big problem cause rest api does a lot of thing in our house and so by reloading yaml or restarting the hass it creates the same sensor with x+1 id like: sensorXY sensorXY_1 sensorXY_2

[timr49]

I have found one cause of this. Consider the YAML configuration:

sensor:
  - platform: rest
    scan_interval: 200
    unique_id: rest_test2
    resource: "http://date.jsontest.com/"
    name: test2
    value_template: >
      {{ value_json.date }}
  - platform: rest
    scan_interval: 400
    unique_id: rest_test4
    resource: "http://date.jsontest.com/"
    name: test4
    value_template: >
      {{ value_json.time }}
  - platform: rest
    scan_interval: 200
    unique_id: rest_test6
    resource: "http://ip.jsontest.com/"
    name: test6
    value_template: >
      {{ value_json.ip }}

Debug tracing shows that this results in two (2) instances of EntityPlatform (helpers/entity_platform.py). It appears this is because helpers/entity_component.py uses scan_interval as part of a key and there are two (2) distinct scan intervals in the above three (3) sensors. From async_setup_platform ...

key = (platform_type, scan_interval, entity_namespace)

 if key not in self._platforms:
     self._platforms[key] = self._async_init_entity_platform(
         platform_type, platform, scan_interval, entity_namespace
      )

Reloading this configuration results in the error in question only for the entity with unique id rest_test4: ERROR (MainThread) [homeassistant.components.sensor] Platform rest does not generate unique IDs. ID rest_test4 already exists - ignoring sensor.test4 Debug tracing shows that the reload code in helpers/reload.py resets the entities for only one of the two instances of PlatformEntity - in this case for the one containing the entities with unique ids rest_test2 and rest_test6 but not the one containing rest_test4. This is because _resetup_platform in helpers/reload.py calls _async_reconfig_platform and hence async_reset with only one instance of PlatformEntity, namely the first one found by async_get_platform_without_config_entry, and this happens to be the instance containing rest_test2 and rest_test6. Then _async_reconfig_platform calls platform.async_setup with the config for all (both) instances of EntityPlatform, which proceeds to set them all up even though platform only relates to one of them. Because the instance containing rest_test4 was never passed to async_reset, the tests for it already existing by _async_add_entity (helpers/entity_platform.py) fail and the error results.

If you delete the scan intervals from the above config, or make them all the same, there are no "Platform rest does not generate unique IDs" error.

On the other hand, the YAML configuration:

rest:
  - resource: "http://date.jsontest.com/"
    scan_interval: 100
    sensor:
      - unique_id: rest_test1
        name: test1
        value_template: >
          {{ value_json.date }}
      - unique_id: rest_test3
        name: test3
        value_template: >
          {{ value_json.time }}
  - resource: "http://ip.jsontest.com/"
    scan_interval: 500
    sensor:
      - unique_id: rest_test5
        name: test5
        value_template: >
          {{ value_json.ip }}

does not result in a "Platform rest does not generate unique IDs" error. Debug tracing shows that there is only one instance of PlatformEntity created (not sure where), to which all three entities are attached. In this case, polling of the resource is driven by DataUpdateCoordinator, so presumably the different scan intervals do not cause additional instances of PlatformEntity to be created.

Thirdly, if both of the above YAML configurations are used together, it does result in "Platform rest does not generate unique IDs" for entities from both configurations. The SERVICE_RELOAD handler reload_service_handler in components/rest/init.py relies on async_reload_integration_platforms to reset the entities rest_test1/3/5. For the same reason as above, only rest_test2/6 gets reset resulting in errors for rest_test4/1/3/5.

It seems to me that the solution lies in changing helpers/reload.py to loop through all relevant instances of PlatformEntity in the manner of async_get_platform_without_config_entry but to subject them all to platform.async_reset() and to call platform.async_setup(p_config) with a p_config that is the config for that instance only.

[timr49]

had this issue since 2023 and still in 2024.6.0. in my case its a big problem cause rest api does a lot of thing in our house and so by reloading yaml or restarting the hass it creates the same sensor with x+1 id like: sensorXY sensorXY_1 sensorXY_2

Hi @kobraevents, my investigations so far suggest that if you only use the form:

rest:
  - resource: ...
    sensor:
       - ...

and never use the form:

sensor:
  - platform: rest
    ...

then you should not have this problem (provided your unique_ids are unique, of course). Does that work for you?

[jesseuitto]

had this issue since 2023 and still in 2024.6.0. in my case its a big problem cause rest api does a lot of thing in our house and so by reloading yaml or restarting the hass it creates the same sensor with x+1 id like: sensorXY sensorXY_1 sensorXY_2

Hi @kobraevents, my investigations so far suggest that if you only use the form:

rest:
  - resource: ...
    sensor:
       - ...

and never use the form:

sensor:
  - platform: rest
    ...

then you should not have this problem (provided your unique_ids are unique, of course). Does that work for you?

I have this as you suggested, as stated in this comment https://github.com/home-assistant/core/issues/90394#issuecomment-1828071120

In short, the code is:

rest:
  - resource: xxx
    scan_interval: 5
    method: GET
    sensor:

I still have this unique id problem.

[timr49]

I have this as you suggested, as stated in this comment #90394 (comment)

In short, the code is:

rest:
  - resource: xxx
    scan_interval: 5
    method: GET
    sensor:

I still have this unique id problem.

Hi @jesseuitto, here are a few suggestions:

1. make sure that you do not have the:

   sensor:
   - platform: rest
   ...

   format anywhere in any of your YAML configuration as it will trigger the problem;

2. make sure that you Restart Home Assistant (not just Quick reload) after removing all occurences of the above format;
3. if that doesn't fix it, provide all of the YAML under rest: just as you have it (except any passwords).

[jesseuitto]

1. make sure that you do not have the:

sensor:
  - platform: rest
    ...

format anywhere in any of your YAML configuration as it will trigger the problem;

Actually I just started to go through the code and found

sensor:
  - platform: rest

from one of my custom components. I am just going through to change that, update the HA and then see what happens. I think I have to wait for couple days, because this problem is occasional.

[Ekomak]

Thanks for the tips.

I check suggestion 1 and after removing the old rest method, the unique_id issue disapear.

[timr49]

@Ekomak - that's good news!

Because it depends on the values of scan_interval, it can appear random if you happen to change them at some time prior to a reload. At first I thought it was a race condition because of this apparent randomness, but with controlled tests it is 100% reproducible.

[alekslegkih]

and never use the form:

sensor:
  - platform: rest
    ...

then you should not have this problem (provided your unique_ids are unique, of course). Does that work for you?

I confirm. If you remove the:

sensor:
  - platform: rest

the error does not occur.

@timr49 excellent research, thanks