search

davidusb-geek / emhass

emhass: Energy Management for Home Assistant, is a Python module designed to optimize your home energy interfacing with Home Assistant.
MIT License
260 stars 51 forks source link

Data posted via home assistant is different to what appears in the EMHASS logs #304

Closed TETZUO closed 3 weeks ago

TETZUO commented 3 weeks ago

Describe the bug I am using EMHASS in Home Assistant to post to the MPC optimiser, The issue i am seeing is that the data in the curl command is somehow different in EMHASS compared to what the template render shows in the dev tools, this is causing an infeasble result.

An example is the soc_init, in the HA template render it shows 0.38 remaning, but in the logs for EMHASS it shows 1.0

To Reproduce The following command is run from Home Assistant service: shell_command.post_mpc_optim_solcast data: command: > curl -i -H "Content-Type: application/json" -X POST -d '{ "load_cost_forecast": {{ ( [states('sensor.general_price')|float(0)] + state_attr('sensor.general_forecast', 'forecasts')|map(attribute='per_kwh')|list )|tojson }}, "prod_price_forecast": {{ ( [states('sensor.feed_in_price')|float(0)] + state_attr('sensor.feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list )|tojson }}, "pv_power_forecast": {{ (state_attr('sensor.solcast_24hrs_forecast', 'forecasts')|map(attribute='pv_estimate')|list)|tojson }}, "prediction_horizon": 48, "soc_init": {{ (states('sensor.powerwall_charge')|float(0)) / 100 }}, "soc_final": 0.05, "def_total_hours": [2], "def_load_start": [0], "def_load_end": [95] # Adjusted to match the prediction horizon }' http://localhost:5000/action/naive-mpc-optim

Expected behavior The Sensor Data that appears in home assistant should appear in the EMHASS logs, data does appear but its not the data from the curl command

Screenshots 1234

Home Assistant installation type

Your hardware

EMHASS installation type

Additional context My configuration for EMHASS `logging_level: DEBUG data_path: /data/ costfun: profit sensor_power_photovoltaics: sensor.solar_energy_v2 sensor_power_load_no_var_loads: sensor.home_load_power set_total_pv_sell: false set_nocharge_from_grid: false set_nodischarge_to_grid: false maximum_power_from_grid: 35000 maximum_power_to_grid: 15000 number_of_deferrable_loads: 1 list_nominal_power_of_deferrable_loads:

davidusb-geek commented 3 weeks ago

These are very tricky. Did you tried putting a |tojson at the end of your soc_init template. Otherwise I think that another problem is your setting of SOC final which is lower than the SOC min defined in your configuration file

TETZUO commented 3 weeks ago

Yep same issue,

I set the MPC call to "def_load_end": [90] but the log still shows def_load_end': [96]

Also the load forcasts are different from whats in home assistant, for example the template render shows the correct cost forecast where EMHASS does not

curl -i -H "Content-Type: application/json" -X POST -d '{ "load_cost_forecast": [0.06, 0.1, 0.1, 0.13, 0.15, 0.29, 0.29, 0.55, 0.59, 0.55, 0.55, 0.48, 0.48, 0.39, 0.39, 0.16, 0.15, 0.12, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.1, 0.11, 0.03, 0.03, 0.09, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11, 0.07, 0.07, 0.07, 0.08, 0.36, 0.08, 0.08, 0.06],

EMHASS Web Log `2024-06-10 13:45:13,793 - web_server - INFO - Status: Infeasible 2024-06-10 13:45:13,793 - web_server - INFO - Total value of the Cost function = 42.40 2024-06-10 13:45:22,618 - web_server - INFO - Passed runtime parameters: {'load_cost_forecast': [0.15, 0.19, 0.22, 0.43, 0.37, 0.69, 0.68, 0.73, 0.73, 0.68, 0.71, 0.8, 0.75, 0.5, 0.57, 0.48, 0.36, 0.36, 0.36, 0.36, 0.31, 0.27, 0.44, 0.39, 0.36, 0.32, 0.27, 0.26, 0.26, 0.26, 0.29, 0.3, 0.36, 0.59, 0.82, 0.46, 0.4, 0.23, 0.22, 0.13, 0.12, 0.14, 0.09, 0.09, 0.09, 0.09, 0.09, 0.14], 'prod_price_forecast': [0.05, 0.09, 0.11, 0.26, 0.2, 0.28, 0.27, 0.32, 0.32, 0.27, 0.3, 0.39, 0.34, 0.32, 0.38, 0.3, 0.19, 0.19, 0.19, 0.19, 0.15, 0.11, 0.26, 0.22, 0.19, 0.15, 0.11, 0.1, 0.1, 0.1, 0.13, 0.14, 0.19, 0.4, 0.61, 0.28, 0.23, 0.08, 0.06, 0.03, 0.03, 0.04, -0.0, -0.0, -0.0, -0.0, -0.0, 0.05], 'pv_power_forecast': [0.9197, 0.3969, 0.7298, 0.5452, 0.2939, 0.0066, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0492, 1.6617, 3.1556, 4.4012, 5.5342, 6.3928, 7.0159, 7.6574, 7.9615, 8.0619, 7.7789, 7.1398, 6.4503, 5.6678, 4.9008, 4.0576], 'prediction_horizon': 24, 'soc_init': 1.0, 'soc_final': 0.2, 'def_total_hours': [24], 'def_load_start': [0], 'def_load_end': [96]} 2024-06-10 13:45:22,619 - web_server - INFO - >> Setting input data dict 2024-06-10 13:45:22,619 - web_server - INFO - Setting up needed data 2024-06-10 13:45:22,623 - web_server - INFO - Retrieve hass get data method initiated... 2024-06-10 13:45:23,064 - web_server - INFO - Retrieving weather forecast data using method = list 2024-06-10 13:45:23,065 - web_server - INFO - Retrieving data from hass for load forecast using method = naive 2024-06-10 13:45:23,066 - web_server - INFO - Retrieve hass get data method initiated... 2024-06-10 13:45:23,385 - web_server - INFO - >> Performing naive MPC optimization... 2024-06-10 13:45:23,385 - web_server - INFO - Performing naive MPC optimization 2024-06-10 13:45:23,396 - web_server - INFO - Perform an iteration of a naive MPC controller 2024-06-10 13:45:23,461 - web_server - DEBUG - Deferrable load 0: Proposed optimization window: 0 --> 0 2024-06-10 13:45:23,461 - web_server - DEBUG - Deferrable load 0: Validated optimization window: 0 --> 0 2024-06-10 13:45:23,471 - web_server - WARNING - Solver default unknown, using default Welcome to the CBC MILP Solver Version: 2.10.3 Build Date: Dec 15 2019

command line - /usr/local/lib/python3.11/dist-packages/pulp/solverdir/cbc/linux/64/cbc /tmp/d64115d077194a7582bb81edc5a47375-pulp.mps -max -timeMode elapsed -branch -printingOptions all -solution /tmp/d64115d077194a7582bb81edc5a47375-pulp.sol (default strategy 1) At line 2 NAME MODEL At line 3 ROWS At line 435 COLUMNS At line 2778 RHS At line 3209 BOUNDS At line 3498 ENDATA Problem MODEL has 430 rows, 264 columns and 2030 elements Coin0008I MODEL read with 0 errors Option for timeMode changed from cpu to elapsed Problem is infeasible - 0.00 seconds Option for printingOptions changed from normal to all Total time (CPU seconds): 0.00 (Wallclock seconds): 0.00

2024-06-10 13:45:23,484 - web_server - INFO - Status: Infeasible 2024-06-10 13:45:23,484 - web_server - INFO - Total value of the Cost function = 42.40`

MPC Curl

service: shell_command.post_mpc_optim_solcast data: command: > curl -i -H "Content-Type: application/json" -X POST -d '{ "load_cost_forecast": {{ ( [states('sensor.general_price')|float(0)] + state_attr('sensor.general_forecast', 'forecasts')|map(attribute='per_kwh')|list )|tojson }}, "prod_price_forecast": {{ ( [states('sensor.feed_in_price')|float(0)] + state_attr('sensor.feed_in_forecast', 'forecasts')|map(attribute='per_kwh')|list )|tojson }}, "pv_power_forecast": {{ (state_attr('sensor.solcast_24hrs_forecast', 'forecasts')|map(attribute='pv_estimate')|list)|tojson }}, "prediction_horizon": 48, "soc_init": {{ ((states('sensor.powerwall_charge')|float(0)) / 100) | tojson }}, "soc_final": 0.2, # Adjusted to match the minimum SOC from your configuration "def_total_hours": [2], "def_load_start": [0], "def_load_end": [90] # Adjusted to match the prediction horizon }' http://localhost:5000/action/naive-mpc-optim

purcell-lab commented 3 weeks ago

Something else is injecting those values into EMHASS, which normally should only be your automations.

How often are you calling the MPC command ( every minute/ five minutes) and does that correspond to you to logs?

Can you call the curl shell command from the developer tools: services menu and see what happens in your logs?

In your EMHASS configuration file, please add the following to the end. 

continual_publish: false

Are you comfortable to manually call curl from a command line? You may need to change the escape `` characters to call from the command line. Then call from the command line and see what is happening in the logs.

The other thing to do is to switch from the curl command line to a REST POST service and then call that from developer tools.

TETZUO commented 3 weeks ago

@purcell-lab

I may of resovled the issue there seems to be an issue with the solcast forecast sensor being an intger and not a float. It seems to be publishing results now and indicates at 5:30am it wants the battery to charge (Amber price is 0.04 for 30 mins)

does look like the numbers are around the wrong way

image

purcell-lab commented 3 weeks ago

Odd,

I run my solcast forecasts as int, but as whole W.


        "pv_power_forecast": {{
          ([states('sensor.solaredge_nopowerlimit')|int(0)] +
          state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list +
          state_attr('sensor.solcast_pv_forecast_forecast_tomorrow', 'detailedForecast')|selectattr('period_start','gt',utcnow()) | map(attribute='pv_estimate')|map('multiply',1000)|map('int')|list
          )| tojson
        }},
TETZUO commented 3 weeks ago

odd but it seems like its working im in SA with Amber and the battery charge events look to be optimal

image

I will close this one out as take the conversion back to the HA forums, thanks for your help @davidusb-geek @purcell-lab