RESTful sensor fails and does not recover

[tomlut]

The problem

I have the following rest sensors:

- resource: https://api.solcast.com.au/rooftop_sites/<MY_SITE_ID_HERE>/forecasts?format=json&api_key=<MY_API_KEY_HERE>
  headers:
    User-Agent: Home Assistant # recently added
    Content-Type: application/json # recently added
  scan_interval: '00:30:00' # 50 calls allowed per UTC day
  sensor:
  - name: "Solcast Forecast Data"
    unique_id: 7390082f-b19d-4301-b386-36823458bac1
    force_update: true
    value_template: "{{ value_json.forecasts[0].pv_estimate|round(1) }}"
    unit_of_measurement: kW
    device_class: power
    json_attributes:
    - forecasts 

  - name: "Solcast Forecast 10"
    unique_id: 829f84af-93dc-4c11-bfe3-75363a76c310
    force_update: true
    value_template: "{{ value_json.forecasts[0].pv_estimate10|round(1) }}"
    unit_of_measurement: kW
    device_class: power
    state_class: measurement

  - name: "Solcast Forecast 90"
    unique_id: a2a5acd4-4d46-4c12-896e-8d96389c58b9
    force_update: true
    value_template: "{{ value_json.forecasts[0].pv_estimate90|round(1) }}"
    unit_of_measurement: kW
    device_class: power
    state_class: measurement

I was seeing sporadic failed requests that would sometimes only work again after a HA restart. However during these times I could access the API via a web browser without issue. These failure would be reported as empty json data. I also checked I was not exceeding my API call limit. I was nowhere close (4 of 50 used).

After adding the user agent option I thought I had fixed the issue but in fact it was just that the failed updates no longer log errors. The sensor still fails to update until after a HA restart. Before the restart I checked I could access the API via a web browser without issue.

I have now removed the user agent option again and will update this issue with the error when it re-occurs.

What version of Home Assistant Core has the issue?

core-2024.7.2

What was the last working version of Home Assistant Core?

No response

What type of installation are you running?

Home Assistant OS

Integration causing the issue

Rest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/rest/

Diagnostics information

No response

Example YAML snippet

No response

Anything in the logs that might be useful for us?

Logger: homeassistant.helpers.template Source: helpers/template.py:2652 First occurred: 02:08:44 (3 occurrences) Last logged: 02:08:44

Template variable error: 'value_json' is undefined when rendering '{{ value_json.forecasts[0].pv_estimate|round(1) }}' Template variable error: 'value_json' is undefined when rendering '{{ value_json.forecasts[0].pv_estimate10|round(1) }}' Template variable error: 'value_json' is undefined when rendering '{{ value_json.forecasts[0].pv_estimate90|round(1) }}'

Additional information

No response

[home-assistant[bot]]

rest documentation rest source

[tomlut]

I have implemented a workaround to reload the restful integration when this happens:

- id: 2801a42c-9d0c-4c54-ae6e-e5cfb32e866c
  alias: 'Fix solar forecast rest sensor'
  trigger:
  - platform: state
    entity_id: sensor.solcast_forecast_data
    to: unknown
    for: 
      minutes: 5
  action:
  - service: rest.reload
  - service: notify.telegram_alert
    data:
      title: '☀️ <b>Solar Forecast Update</b>'
      message: "Rest integration reloaded"

[cconnoruk]

Same problem here. Only recently though, mid Aug for me. Will look at your work around.