davidusb-geek / emhass-add-on

The Home Assistant Add-on for EMHASS: Energy Management Optimization for Home Assistant
MIT License
88 stars 26 forks source link

Standard Config not running on first time run #76

Closed bf8392 closed 7 months ago

bf8392 commented 7 months ago

Hi, I'm new to EMHASS, and basically just downloaded it, and readed the docs for some days... Sadly I was not able to get it to run, as the standard config does throw an error...The only thing I changed, was the variable: I have no photovoltaic...I wanted to use the forecast for energy usage...

logging_level: DEBUG
costfun: profit
sensor_power_photovoltaics: sensor.power_photovoltaics
sensor_power_load_no_var_loads: sensor.gesamtenergieverbrauch_in_watt
set_total_pv_sell: false
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
maximum_power_from_grid: 9000
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 3000
  - nominal_power_of_deferrable_loads: 750
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 5
  - operating_hours_of_each_deferrable_load: 8
list_start_timesteps_of_each_deferrable_load:
  - start_timesteps_of_each_deferrable_load: 0
  - start_timesteps_of_each_deferrable_load: 0
list_end_timesteps_of_each_deferrable_load:
  - end_timesteps_of_each_deferrable_load: 0
  - end_timesteps_of_each_deferrable_load: 0
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "05:54"
  - peak_hours_periods_start_hours: "10:24"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "09:24"
  - peak_hours_periods_end_hours: "11:54"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: true
  - treat_deferrable_load_as_semi_cont: true
list_set_deferrable_load_single_constant:
  - set_deferrable_load_single_constant: false
  - set_deferrable_load_single_constant: false
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
list_pv_module_model:
  - pv_module_model: CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M
list_pv_inverter_model:
  - pv_inverter_model: Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_
list_surface_tilt:
  - surface_tilt: 30
list_surface_azimuth:
  - surface_azimuth: 205
list_modules_per_string:
  - modules_per_string: 16
list_strings_per_inverter:
  - strings_per_inverter: 1
set_use_battery: false
battery_nominal_energy_capacity: 5000

Logs:

s6-rc: info: service s6rc-oneshot-runner: starting
s6-rc: info: service s6rc-oneshot-runner successfully started
s6-rc: info: service fix-attrs: starting
s6-rc: info: service fix-attrs successfully started
s6-rc: info: service legacy-cont-init: starting
s6-rc: info: service legacy-cont-init successfully started
s6-rc: info: service legacy-services: starting
services-up: info: copying legacy longrun emhass (no readiness notification)
s6-rc: info: service legacy-services successfully started
2024-02-14 11:00:30,613 - web_server - INFO - Launching the emhass webserver at: http://0.0.0.0:5000
2024-02-14 11:00:30,613 - web_server - INFO - Home Assistant data fetch will be performed using url: http://supervisor/core/api
2024-02-14 11:00:30,613 - web_server - INFO - The data path is: /share
2024-02-14 11:00:30,615 - web_server - INFO - Using core emhass version: 0.7.7
waitress   INFO  Serving on http://0.0.0.0:5000
2024-02-14 11:03:39,626 - web_server - INFO - EMHASS server online, serving index.html...
2024-02-14 11:03:39,637 - web_server - WARNING - The data container dictionary is empty... Please launch an optimization task
2024-02-14 11:03:40,715 - web_server - INFO - Setting up needed data
2024-02-14 11:03:40,768 - web_server - INFO - Retrieving weather forecast data using method = scrapper
2024-02-14 11:03:42,154 - web_server - INFO - Retrieving data from hass for load forecast using method = naive
2024-02-14 11:03:42,155 - web_server - INFO - Retrieve hass get data method initiated...
2024-02-14 11:03:42,171 - web_server - ERROR - The retrieved JSON is empty, check that correct day or variable names are passed
2024-02-14 11:03:42,171 - web_server - ERROR - Either the names of the passed variables are not correct or days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)
2024-02-14 11:03:42,171 - web_server - ERROR - Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 49, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 91, in set_input_data_dict
    P_load_forecast = fcst.get_load_forecast(method=optim_conf['load_forecast_method'])
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/forecast.py", line 589, in get_load_forecast
    rh.get_data(days_list, var_list)
  File "/usr/local/lib/python3.11/dist-packages/emhass/retrieve_hass.py", line 150, in get_data
    self.df_final = pd.concat([self.df_final, df_day], axis=0)
                                              ^^^^^^
UnboundLocalError: cannot access local variable 'df_day' where it is not associated with a value
2024-02-14 11:03:47,351 - web_server - INFO - Setting up needed data
2024-02-14 11:03:47,355 - web_server - INFO - Retrieve hass get data method initiated...
2024-02-14 11:03:47,376 - web_server - ERROR - The retrieved JSON is empty, check that correct day or variable names are passed
2024-02-14 11:03:47,377 - web_server - ERROR - Either the names of the passed variables are not correct or days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)
2024-02-14 11:03:47,377 - web_server - ERROR - Exception on /action/perfect-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 1463, in wsgi_app
    response = self.full_dispatch_request()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 872, in full_dispatch_request
    rv = self.handle_user_exception(e)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 870, in full_dispatch_request
    rv = self.dispatch_request()
         ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/flask/app.py", line 855, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)  # type: ignore[no-any-return]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/web_server.py", line 49, in action_call
    input_data_dict = set_input_data_dict(config_path, str(data_path), costfun,
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/dist-packages/emhass/command_line.py", line 78, in set_input_data_dict
    rh.get_data(days_list, var_list,
  File "/usr/local/lib/python3.11/dist-packages/emhass/retrieve_hass.py", line 150, in get_data
    self.df_final = pd.concat([self.df_final, df_day], axis=0)
                                              ^^^^^^
UnboundLocalError: cannot access local variable 'df_day' where it is not associated with a value

I should have enough history...eve trying defining the days to two didn't wor out...maybe I'm doing something wrong? Please help =)

davidusb-geek commented 7 months ago

What are your settings for these parameters?

hass_url: empty
long_lived_token: empty
bf8392 commented 7 months ago

I have not put it in... should I put it? Do I need to create an access token?

davidusb-geek commented 7 months ago

Just check that they have and empty value if you are using the add-on.

Do I need to create an access token?

No (if using the add-on)

bf8392 commented 7 months ago

Okay I'm using the add-on and altered the config...still same error;


logging_level: DEBUG
costfun: profit
sensor_power_photovoltaics: sensor.power_photovoltaics
sensor_power_load_no_var_loads: sensor.gesamtenergieverbrauch_in_watt
set_total_pv_sell: false
set_nocharge_from_grid: false
set_nodischarge_to_grid: false
maximum_power_from_grid: 9000
number_of_deferrable_loads: 2
list_nominal_power_of_deferrable_loads:
  - nominal_power_of_deferrable_loads: 3000
  - nominal_power_of_deferrable_loads: 750
list_operating_hours_of_each_deferrable_load:
  - operating_hours_of_each_deferrable_load: 5
  - operating_hours_of_each_deferrable_load: 8
list_start_timesteps_of_each_deferrable_load:
  - start_timesteps_of_each_deferrable_load: 0
  - start_timesteps_of_each_deferrable_load: 0
list_end_timesteps_of_each_deferrable_load:
  - end_timesteps_of_each_deferrable_load: 0
  - end_timesteps_of_each_deferrable_load: 0
list_peak_hours_periods_start_hours:
  - peak_hours_periods_start_hours: "05:54"
  - peak_hours_periods_start_hours: "10:24"
list_peak_hours_periods_end_hours:
  - peak_hours_periods_end_hours: "09:24"
  - peak_hours_periods_end_hours: "11:54"
list_treat_deferrable_load_as_semi_cont:
  - treat_deferrable_load_as_semi_cont: true
  - treat_deferrable_load_as_semi_cont: true
list_set_deferrable_load_single_constant:
  - set_deferrable_load_single_constant: false
  - set_deferrable_load_single_constant: false
load_peak_hours_cost: 0.1907
load_offpeak_hours_cost: 0.1419
photovoltaic_production_sell_price: 0.065
list_pv_module_model:
  - pv_module_model: CSUN_Eurasia_Energy_Systems_Industry_and_Trade_CSUN295_60M
list_pv_inverter_model:
  - pv_inverter_model: Fronius_International_GmbH__Fronius_Primo_5_0_1_208_240__240V_
list_surface_tilt:
  - surface_tilt: 30
list_surface_azimuth:
  - surface_azimuth: 205
list_modules_per_string:
  - modules_per_string: 16
list_strings_per_inverter:
  - strings_per_inverter: 1
set_use_battery: false
battery_nominal_energy_capacity: 5000
days_to_retrieve: 2
hass_url: empty
long_lived_token: empty
davidusb-geek commented 7 months ago

Are you sure that you have enough history data on sensor.gesamtenergieverbrauch_in_watt?

bf8392 commented 7 months ago

Oh I'm sorry my bad...the parent sensor would have enough history, but the template only has one and a half day... But even if I set "days_to_retrieve to one, I get the same error...Screenshot_20240214_145101_Home Assistant.jpg

davidusb-geek commented 7 months ago

It is hard coded to 2 days minimum. So the logged message of your first post is correct: days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)

GeoDerp commented 7 months ago

@davidusb-geek , maybe we can set it to auto set to two days if any numbers under 2, and warn the user with a logger.error ? May reduce the python error in some workflows yet still letting the user know there is an issue.

davidusb-geek commented 7 months ago

Yes that could be an option. But for example in the case with this issue the result will be the same right? The real solution will be finding a way to let the user just enter their normal load power sensor and then automatically identify the deferrable loads on that data to subtract them from the total load. An idea could be that for a first ever run with just the load data as it is but for a seond run we use the saved results from a previous optimization load schedule run to try to identify at which time the deferrable loads are and then subtract them. That could work

purcell-lab commented 7 months ago

try to identify at which time the deferrable loads are and then subtract them.

I do something quite different to subtract my deferrable loads and have been meaning to discuss this online as they would greatly simplify setup for new users:

    - name: power_load_no_var_loads
      unit_of_measurement: W
      device_class: power
      state_class: measurement 
      unique_id: 9e2d02cb-2ac4-4eb2-9f57-c4c158c145e7
      state: >-
        {{states('sensor.gateway_load_power')|int(0)
        - states('sensor.p_deferrable0')|int(0) 
        - states('sensor.p_deferrable1')|int(0)
        - states('sensor.p_deferrable2')|int(0) 
        - states('sensor.p_deferrable3')|int(0)
        - states('sensor.p_deferrable4')|int(0)
        - states('sensor.p_deferrable5')|int(0)
        }}

My original motivation was that my HVAC load is quite variable (dependent on the setpoint difference, but that is another conversation).

Anyway I noticed that EMHASS couldn't allocate additional power if it was going to be consumed by the load, sometimes upto 5000W difference and I observed how power_load_no_var_loads was being utilised to manage the overs and unders with the power allocation, so i tried this alternative calculation and it works really well.

Giving an example my p_deferrable3 is my HVAC which has a p_nom (average) of 3000W, but in reality the power draw could be anywhere between 0W - 11,000W, depending on the set point. So using the modified load calculation above I have gateway_load_power = 8000W (which includes the actual HVAC maybe 6000W) and p_deferrable3 = 3000W (p_nom). So the power_load_no_vars comes out at 8000 - 3000 = 5000W. If p_pv is 10000W then EMHASS allocates 3000 to deferrable_3, 5000W to load_no_vars (which includes the missing 3000W from HVAC) and has 2000 W remaining to allocate to other loads.

vs the traditional method of calculation would be power_load_no_vars = 8000 - 6000 = 2000 W. If p_pv is 10000W then EMHASS allocates 3000 W to deferrable_3, 2000W to load_no_vars and has 5000W to allocate. But the HVAC is already consuming 3000W, so in reality EMHASS only should be allocating 2000W to the other loads.

Now where this can really simply EMHASS optimisation is these values are all internal (except gateway_load) to EMHASS, so the user doesn't need to supply a subtraction calculation to come up with load_no_vars, it can be calculated internally within EMHASS.

davidusb-geek commented 7 months ago

@purcell-lab I don't think that I completely understand well. How is that template a modified version of what is needed to compute the power_load_no_var_loads sensor. It is actually what is prescribed and it is also what I use everyday. Or states('sensor.p_deferrable3')|int(0) is just retrieving the constant nominal value of your deferrable load 3? The main goal for power_load_no_var_loads is to be used for the load power forecast, either naive of the more advanced ML method. So I don't see how can we do what you say internally without knowing for each deferrable both the power value AND when these loads where scheduled in the past by EMHASS. Maybe an example on a time series curve could help

purcell-lab commented 7 months ago

Yes, I agree I haven't explained the connect too well.

I shall try and connect since examples to demonstrate the concept.

bf8392 commented 7 months ago

It is hard coded to 2 days minimum. So the logged message of your first post is correct: days_to_retrieve is larger than the recorded history of your sensor (check your recorder settings)

I see graphs 😍 it works great thanks :-). Should I close the issue, or do you want to leave it open?

davidusb-geek commented 7 months ago

Great! we will close it. Anyways it stays here for future reference to others.