Combined renewables sensor?

[ando2040]

Hi, I've had a play with custom sensors, but I think I'm failing because this integration provides arrays of data, and all the tutorials on how to do this are A + B + C single values.

I should probably be able to combine sensors in Node-RED to make a new one but I lack the skill (needs JsonATA?).

What I'm trying to achieve is a new renewables sensor consisting of sensor.national_grid_embedded_solar_forecast_three_day + sensor.national_grid_wind_forecast_now_to_three_day (I perceive as the most reliable of the wind forecasts but happy to be told if there's a better one.)

The purpose of this is to be able to see more clearly when renewables are approaching the current demand (which is a good indicator of plunge in wholesale prices)

Does anyone have an idea how I should best achieve this, or whether it could be added to the integration please?

[JRascagneres]

Hey. Looks like this does what you want:

https://community.home-assistant.io/t/need-help-with-template-sum-of-the-elements-of-2-lists/466171/2

Its a bit more complicated as its objects rather than single items but its doable. Give it a stab and if not I can try later. Rather than the arrays he has statically set though you can do like {{ state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast') }}

[ando2040]

I'd got as far as state_attr and got stuck (I hadn't seen the code you linked). I think the below might be close to what you intended but there's still a big error in how it iterates through the arrays. ('list object' has no attribute 'generation' in template editor)

{% set a = state_attr('sensor.national_grid_wind_forecast_now_to_three_day', 'forecast')[1:-1].generation(',') | map('int') | list %}
{% set b = state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast')[1:-1].generation(',') | map('int') | list %}
{% set ns = namespace(items = []) %}
{% for i in range(a | length) %}
  {% set ns.items = ns.items + [ a[i]  + b[i]  ] %}
{% endfor %}
{{ ns.items }}

I'm not sure this will achieve what I want anyway, since I'd like to maintain start_time in an array but see a new value of generation which is the sum of the two input sensors for each timestamp

forecast:

• start_time: "2024-03-26T21:00:00+00:00" generation: 8878
• start_time: "2024-03-26T21:30:00+00:00" generation: 9220
• start_time: "2024-03-26T22:00:00+00:00" generation: 9515

[JRascagneres]

{% set a = state_attr('sensor.national_grid_wind_forecast_now_to_three_day', 'forecast') -%}
{% set b = state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast') -%}

{% set ns = namespace(items=[]) -%}

{% for i in range (a | length) -%}
{% set start_time = a[i].start_time -%}
{% set generation_total = a[i].generation + b[i].generation -%}

{% set ns.items = ns.items + [{'start_time': start_time, 'generation': generation_total}] %}

{% endfor %}

{{ns.items}}

That should do it. That maintains the time too.

[ando2040]

Thanks - it looks sensible in template editor, but when I add it as a template sensor I get state 'unknown'

I'm feeling very dense, what have I got wrong now?

template:
  - sensor:
      - name: "National Grid Renewables 3 day forecast"
        unit_of_measurement: "MW"
        state: >
          {% set a = state_attr('sensor.national_grid_wind_forecast_now_to_three_day', 'forecast') -%}
          {% set b = state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast') -%}
          {% set ns = namespace(items=[]) -%}
          {% for i in range(a | length) -%}
            {% set start_time = a[i].start_time -%}
            {% set generation_total = a[i].generation + b[i].generation -%}
            {% set ns.items = ns.items + [{'start_time': start_time, 'generation': generation_total}] %}
          {% endfor %}
          {{ns.items}}
        state_class: measurement

[JRascagneres]

@ando2040 Apologies. Missed the reply.

The reason is because a basic sensor can only show one value, not an array like this. You need to specify it in an attribute field.

I'll look tonight for you if you can't figure it out.

[ando2040]

As mentioned, I've only manipulated array sensors with Node-RED, and that was with someone else writing the initial code.

Thanks for the pointer, I feel like this is now close but not quite there... FYI the now_to_three_day sensor seemed to stop producing a result in template editor for me, maybe a length issue? So I switched to the shorter array which does work in template editor but not in the form below. I'm not sure the significance of state:...

      - name: "National Grid Renewables 3day Forecast"
        device_class: power
        state: standby
        attribute_templates:
          forecast: >-
            {% set a = state_attr('sensor.national_grid_wind_forecast', 'forecast') -%}
            {% set b = state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast') -%}
            {% set ns = namespace(items=[]) -%}
            {% for i in range(a | length) -%}
              {% set start_time = a[i].start_time -%}
              {% set generation_total = a[i].generation + b[i].generation -%}
              {% set ns.items = ns.items + [{'start_time': start_time, 'generation': generation_total}] %}
            {% endfor %}
            {{ ns.items }}

[JRascagneres]

  - sensor:
    - name: "National Grid Renewables 3 Day Forecast"
      state: "Forecast..."
      attributes:
        forecast: >
          {% set a = state_attr('sensor.national_grid_wind_forecast_now_to_three_day', 'forecast') -%}
          {% set b = state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast') -%}
          {% set ns = namespace(items=[]) -%}
          {% for i in range (a | length) -%}
          {% set start_time = a[i].start_time.isoformat() -%}
          {% set generation_total = a[i].generation + b[i].generation -%}
          {% set ns.items = ns.items + [{'start_time': start_time, 'generation': generation_total}] %}
          {% endfor %}
          {{ns.items | list}}

That does it

[ando2040]

Thank you so much! That was beyond my current debugging skill but I think I see what you've done with formatting. Hopefully I can learn from you.

I went on to add a third sensor as a constant of the current nuclear generation - there's no forecast but it's not very dynamic and forms the base load:

  template:
    sensor:
      - name: "National Grid Renewables 3 Day Forecast"
        state: "Forecast"
        attributes:
          forecast: >
            {% set a = state_attr('sensor.national_grid_wind_forecast_now_to_three_day', 'forecast') -%}
            {% set b = state_attr('sensor.national_grid_embedded_solar_forecast_three_day', 'forecast') -%}
            {% set c = states('sensor.national_grid_grid_generation_nuclear_mw') -%}
            {% set ns = namespace(items=[]) -%}
            {% for i in range(a | length) -%}
              {% set start_time = a[i].start_time.isoformat() -%}
              {% set generation_total = a[i].generation | int + b[i].generation | int + c | int -%}
              {% set ns.items = ns.items + [{'start_time': start_time, 'generation': generation_total}] %}
            {% endfor %}
            {{ ns.items | list }}

The result of which is the realisation of my vision - a chart which predicts wholesale price plunges (relevant for Octopus Agile tariff) when the combined renewables forecast (in purple) approaches/crosses the demand (red) line. Happy to contribute it to a community wiki if you do create one! The Octopus forum has an increasing number of people interested in this integration.

[JRascagneres]

Np looks good. I'm also on Agile. Looks good. What forum? 😉

[JRascagneres]

Another note I'm not sure whats up but it appears the forecast demand is always a little lower than the actual demand. I've yet to figure it out but I think its down to transmission losses so you may need to add something else to account for a 10% buffer.

[ando2040]

I had noticed that mismatch - I don't think I will put in a guess programmatically just yet though. If I'm right, this sensor is predicting Friday will herald the second full morning of negative prices this year.

I got invited to the forum by one of the moderators, it is exactly for people like you! @bottlecapdave is active on there, you've probably seen his Octopus integration. Along with a number of industry people like the Hildebrand folks. You might have to wait a few days for a response on agile email. Smart tariff forum

[ando2040]

For the record, and the relatively little it is worth, here is the long range counterpart sensor.

FYI, I had to delete and re-add the integration (including Elexon API key, the full works) both times I created a new custom sensor. No idea why - but I kept losing access to sensors that had been working for weeks, and I was running out of working ones to test with!

  template:
    sensor:
      - name: "National Grid Renewables 14 Day Forecast"
        state: "Forecast"
        attributes:
          forecast: >
            {% set a = state_attr('sensor.national_grid_wind_forecast_fourteen_day', 'forecast') -%}
            {% set b = state_attr('sensor.national_grid_embedded_solar_forecast_fourteen_day', 'forecast') -%}
            {% set c = states('sensor.national_grid_grid_generation_nuclear_mw') -%}
            {% set ns = namespace(items=[]) -%}
            {% for i in range(a | length) -%}
              {% set start_time = a[i].start_time.isoformat() -%}
              {% set generation_total = a[i].generation | int + b[i].generation | int + c | int -%}
              {% set ns.items = ns.items + [{'start_time': start_time, 'generation': generation_total}] %}
            {% endfor %}
            {{ ns.items | list }}

[JRascagneres]

Cool. I've tried with mine and definitely didn't require restarting the integration?

[JRascagneres]

I also notice you're only adding 'wind' and embedded solar. There should be regular solar and embedded wind too.

Regular is grid level for big projects, embedded is the type you put on your house

[ando2040]

I wondered about this - do you mean eg. sensor.national_grid_solar_forecast?

If so, it seems to match exactly onto embedded, but cover a shorter time series, hence I omitted it as a probable duplicate.

yeah, darkyellow doesn't exist but the default does show up so I left it for illustration rather than look up a valid colour name! Just about to click cancel anyway....

[JRascagneres]

@ando2040 Hrm... Might be a bug, I'll have to check. Can defo do both wind ones though.

[ando2040]

I've ignored embedded wind previously as it was fairly insignificant when I checked - however you're right that this weekend it becomes significant at 5GW.

However, I don't see an embedded fourteen day wind forecast?

[JRascagneres]

sensor.national_grid_embedded_wind_forecast_fourteen_day

I did just try to find solar and can't find a data set for long term grid solar... Hrm