Solcast integration question / suggestion

[BJReplay]

Describe the bug It's not super clear from the documentation that Solcast forecast retrieval is supported by EMHASS.

I'm happy to update the documentation, but I'm not sure how often EMHASS calls Solcast to retrieve forecasts. A comment on this issue clarifying that will be enough for me to fork the repo to clarify the documentation.

Additional context Hi, I'm somewhat reluctantly opening an issue (as there's no discussion option on this repo) regarding Solcast forecasts.

I'm guessing based on what I saw in https://github.com/davidusb-geek/emhass-add-on/pull/69 that if you add

   solcast_api_key:
   solcast_rooftop_id:

to secrets_emhass.yaml and specify weather_forecast_method: solcast in config_emhass.yaml and the changelog, and given the code at https://github.com/davidusb-geek/emhass/blob/c9686cd657844cf193b1101c45f0fc7496af0221/src/emhass/forecast.py#L235, that there's support for EMHASS directly calling Solcast for insolation forecasts.

But there really isn't any documentation other than the example of setting up your own Solcast sensor (https://emhass.readthedocs.io/en/latest/forecasts.html#example-using-solcast-forecast-amber-prices).

_As an aside, given that there is a HACS add in Solcast Integration (that integrates nicely with HA and the energy dashboard) - https://github.com/oziee/ha-solcast-solar - what would be ideal would be an option to use this just like var_PV and var_load (so that you don't use limited API calls) - so add in varForecastSolcast.

[davidusb-geek]

Hi the Solcast method is well documented in the second paragraph here: https://emhass.readthedocs.io/en/latest/forecasts.html#pv-power-production-forecast

You probably oversaw that. But yes any improvement to this documentation is welcomed.

This method is directly called in the code, on the line that you showed. And as with the other endpoints of the EMHASS API it is you that control the number of calls to the Solcast API from your automation on the HomeAssistant side. If you setup this method then the Solcast API is called each time that you launch an optimization task.

I'm aware that there are other integrations for Solcast in Home Assistant. The REST sensor is another option. So if someone is already using Solcast, then the preferred method is to pass the forecast to EMHASS using lists of data passed at runtime on your shell commands or REST commands.

[BJReplay]

Thanks, that's really helpful (and yes, I did miss that information - though I did try to read through the documentation before raising the issue).

Thanks for confirmation that improvements to documentation are welcomed; I'll take the time to read it again properly before submitting any suggested changes, so that I can both be sure of all of the contents, as well as suggest changes that make it easier to find things for first time users setting it up.

I know you and other contributors have put a lot of effort into the add on; for those who are running in docker, it's a bit harder to get going and if I can contribute to make that easier, then, when I'm sure that I'm adding value, I will attempt to.

I'll have to look into passing in the forecast, as I'd ideally like to re-optimise regularly (with a five-minute market here in Australia, once a day won't really cut it!).

[Octofinger]

Passing the forecast data as runtime parameters to the optimizations have several advantages. Solcast has a limited amount of API calls on their free plan, so you can have a better control of API call usage from HA. I have an automation running that only updates Solcast during the day (when the sun is up), then pass the values as a runtime parameter as part of the REST call:

I create the REST call in a HA script. Create a bunch of value parameters and then set them all together to the runtime call. Here's the code for the solcast value parameter:

solcast=((state_attr('sensor.solcast_pv_forecast_forecast_today',
    'detailedHourly') | map(attribute='pv_estimate') | map('multiply',1000) |
    map('int') | list
      + state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedHourly') | map(attribute='pv_estimate') | map('multiply',1000) | map('int') | list))
      [now().hour:][:24] %}

This creates a list that can be sent in the REST JSON.

[BJReplay]

This creates a list that can be sent in the REST JSON.

Thanks.

I'm slowly figuring this out, but, for the moment, have put any further work on hold: primarily as my training data for the ML optimisation is in kW not W, so I can't utilise the ML forecast of load with any appreciable history.

The second reason is that, unless you use sensors and configuration, it becomes more complex again for what is already complex; that's not a complaint, just an observation.

If I could get around the kW issue, I'd pursue this, but I don't think I can; I've loaded W history data into a new sensor using https://github.com/klausj1/homeassistant-statistics but that doesn't seem to be adequate; whatever this does, it isn't sufficient to populate the history / statistics tables in such a way that EMHASS can then retrieve the data. Given that it is unsupported, I'm stuck; run ML with a few days of history or just use a naive prediction (which won't be any better than sum of load).

[davidusb-geek]

Yes I did not found the time to answer to that second issue. I don't think that adding more configurations parameters will help on getting this easier to set-up. So we will keep units as they are. However you can still set-up your configuration and use the ML forecaster with that sensor in kW. Here is how: set a new custom ML forecaster for that sensor in kW. With the long history data that you have you'll certainly obtain some nice results. This custom ML object should provide you with forecast of that sensor in kW. Then on HA create a template sensor that prepares this forecast into a list of values multiplied by 1000 to change from W to kW, then feed that list of values at runtime to the optimization. This totally feasible with the current code as it is without adding more complexity.

[BJReplay]

Here is how: set a new custom ML forecaster for that sensor in kW

Thanks. I'll see if I can figure out how to do this.