search

davidusb-geek / emhass

emhass: Energy Management for Home Assistant, is a Python module designed to optimize your home energy interfacing with Home Assistant.
MIT License
278 stars 54 forks source link

Error: KeyError: 'list_hp_periods' when running optimization with CSV option #24

Closed sschimmel closed 1 year ago

sschimmel commented 1 year ago

When I trigger a optimization I’m getting the following error logged in the docker container's output:

2022-10-01 11:25:58,714] INFO in command_line: Setting up needed data
[2022-10-01 11:25:58,715] ERROR in app: Exception on /action/dayahead-optim [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.8/site-packages/flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "src/emhass/web_server.py", line 134, in action_call
    input_data_dict = set_input_data_dict(config_path, str(config_path.parent), costfun,
  File "/usr/local/lib/python3.8/site-packages/emhass-0.3.19-py3.8.egg/emhass/command_line.py", line 42, in set_input_data_dict
    retrieve_hass_conf, optim_conf, plant_conf = utils.get_yaml_parse(config_path, params=params)
  File "/usr/local/lib/python3.8/site-packages/emhass-0.3.19-py3.8.egg/emhass/utils.py", line 239, in get_yaml_parse
    optim_conf['list_hp_periods'] = dict((key,d[key]) for d in optim_conf['list_hp_periods'] for key in d)
KeyError: 'list_hp_periods'

I'm running Emhass in a standalone docker container. I’m loading the solar forecast from Solcast manually using a script and storing that forecast in a csv file in the /data folder. I have two sites setup that need to be combined and I could only add one Solcast site id. I have confirmed that there is a CSV file in the /data folder with the forecast data.

There is also a scipt running that downloads the electricity prices from my electricity provider and stores that in two CSV files in the same /data folder. One for the production and one for the load costs.

In the /data folder I have these CSV files:

Some content from data_pv_power_forecast.csv

2022-10-02 06:00:00+02:00,0
2022-10-02 06:30:00+02:00,0
2022-10-02 07:00:00+02:00,0
2022-10-02 07:30:00+02:00,0.048100000000000004
2022-10-02 08:00:00+02:00,0.31379999999999997
2022-10-02 08:30:00+02:00,0.8818999999999999
2022-10-02 09:00:00+02:00,1.6953
2022-10-02 09:30:00+02:00,2.6088
2022-10-02 10:00:00+02:00,3.5980999999999996

Some content from data_prod_price_forecast.csv

2022-10-01 20:00:00+02:00,1.0199
2022-10-01 21:00:00+02:00,0.9753
2022-10-01 22:00:00+02:00,1.0066
2022-10-01 23:00:00+02:00,0.998
2022-10-02 00:00:00+02:00,0.9714

Based on the comment # list of different tariff periods (only needed if load_cost_forecast_method='hp_hc_periods') after list_hp_periods this config option should not be needed. I’ve tried commenting that line and still I get the KeyError about list_hp_periods.

I haven’t changed what’s in list_hp_periods, it’s still the same from the default config that came with the download of Emhass.

My full config_emhass.yaml file:

# Configuration file for EMHASS

retrieve_hass_conf:
  - freq: 5 # The time step to resample retrieved data from hass in minutes
  - days_to_retrieve: 8 # We will retrieve data from now and up to days_to_retrieve days
  - var_PV: 'sensor.total_current_solar_production_watts' # Photovoltaic produced power sensor in Watts
  - var_load: 'sensor.total_household_consumption' # Household power consumption sensor in Watts (deferrable loads should be substracted)
  - load_negative: False # Set to True if the retrived load variable is negative by convention
  - set_zero_min: True # A special treatment for a minimum value saturation to zero. Values below zero are replaced by nans
  - var_replace_zero: # A list of retrived variables that we would want  to replace nans with zeros
    - 'sensor.total_current_solar_production_watts'
  - var_interp: # A list of retrived variables that we would want to interpolate nan values using linear interpolation
    - 'sensor.total_current_solar_production_watts'
    - 'sensor.total_household_consumption'
  - method_ts_round: 'nearest' # Set the method for timestamp rounding, options are: first, last and nearest

optim_conf:
  - set_use_battery: True # consider a battery storage
  - delta_forecast: 1 # days
  - num_def_loads: 0
  - P_deferrable_nom: # Watts
    - 3000.0
    - 750.0
  - def_total_hours: # hours
    - 5
    - 8
  - treat_def_as_semi_cont: # treat this variable as semi continuous 
    - True
    - True
  - set_def_constant: # set as a constant fixed value variable with just one startup for each 24h
    - False
    - False
  - weather_forecast_method: 'scrapper' # options are 'scrapper' and 'csv'
  - load_forecast_method: 'naive' # options are 'csv' to load a custom load forecast from a CSV file or 'naive' for a persistance model
  - load_cost_forecast_method: 'csv' # options are 'hp_hc_periods' for peak and non-peak hours contracts and 'csv' to load custom cost from CSV file 
  - list_hp_periods: # list of different tariff periods (only needed if load_cost_forecast_method='hp_hc_periods')
    - period_hp_1:
      - start: '02:54'
      - end: '15:24'
    - period_hp_2:
      - start: '17:24'
      - end: '20:24'
  - load_cost_hp: 0.1907 # peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
  - load_cost_hc: 0.1419 # non-peak hours load cost in €/kWh (only needed if load_cost_forecast_method='hp_hc_periods')
  - prod_price_forecast_method: 'csv' # options are 'constant' for constant fixed value or 'csv' to load custom price forecast from a CSV file
  - prod_sell_price: 0.065 # power production selling price in €/kWh (only needed if prod_price_forecast_method='constant')
  - set_total_pv_sell: False # consider that all PV power is injected to the grid (self-consumption with total sell)
  - lp_solver: 'PULP_CBC_CMD' # set the name of the linear programming solver that will be used
  - lp_solver_path: 'empty' # set the path to the LP solver

plant_conf:
#   - P_grid_max: 17000 # The maximum power that can be supplied by the utility grid in Watts
#   - Pd_max: 1000 # If your system has a battery (set_use_battery=True), the maximum discharge power in Watts
#   - Pc_max: 1000 # If your system has a battery (set_use_battery=True), the maximum charge power in Watts
#   - eta_disch: 0.95 # If your system has a battery (set_use_battery=True), the discharge efficiency
#   - eta_ch: 0.95 # If your system has a battery (set_use_battery=True), the charge efficiency
#   - Enom: 5000 # If your system has a battery (set_use_battery=True), the total capacity of the battery stack in Wh
#   - SOCmin: 0.3 # If your system has a battery (set_use_battery=True), the minimun allowable battery state of charge
#   - SOCmax: 0.9 # If your system has a battery (set_use_battery=True), the minimun allowable battery state of charge
#   - SOCtarget: 0.6 # If your system has a battery (set_use_battery=True), the desired battery state of charge at the end of each optimization cycle
davidusb-geek commented 1 year ago

Ok, could you please post the commands that you are using for:

Also commenting all the items on plant_conf might be a problem

davidusb-geek commented 1 year ago

Hi, did you make this work?

Octofinger commented 1 year ago

I run into the same issue with emhass docker stand-alone. I get the exact same error as above, even if I set CSV as method and send in cost estimate directly. Or maybe just because...

Struggling with the config file and the data to emhass just to get it started. PVLib is extremely outdated and I see no near match to my panel/inverter setup. (JA Solar JAM66S30 500W panels and Sungrow SH10RT hybrid inverter). Gonna try Solcast or make my own data generator to predict my PV powers.

Config file: (Probably not optimal. )

retrieve_hass_conf:
  - freq: 30
  - days_to_retrieve: 2
  - var_PV: sensor.pv_power
  - var_load: sensor.template_emhass_no_var_load_2
  - load_negative: False
  - set_zero_min: True
  - var_replace_zero: []
  - var_interp: []
  - method_ts_round: nearest

optim_conf:
  - set_use_battery: False
  - delta_forecast: 1
  - num_def_loads: 0
  - P_deferrable_nom: []
  - def_total_hours: []
  - treat_def_as_semi_cont: []
  - set_def_constant: []
  - weather_forecast_method: scrapper
  - load_forecast_method: naive
  - load_cost_forecast_method: csv
  - prod_price_forecast_method: csv
  - set_total_pv_sell: False
  - lp_solver: COIN_CMD
  - lp_solver_path: /usr/bin/cbc
  - set_nocharge_from_grid: True

plant_conf:
  - P_grid_max: 15000
  - Pd_max: 4400
  - Pc_max: 5200
  - Enom: 9600
  - SOCmin: 0
  - SOCmax: 1

Docker run command: (in bash script)

docker run -it --name emhass \
  --restart always \
  -p 5000:5000 \
  -e "LOCAL_COSTFUN=profit" \
  -v /srv/docker/emhass/config_emhass.yaml:/app/config_emhass.yaml \
  -v /srv/docker/emhass/secrets_emhass.yaml:/app/secrets_emhass.yaml \
  davidusb/emhass-docker-standalone

curl query: (Hard coded nordpool values as I got some weird bad request issues if I added jinja templates for the sensor.) curl -i -H 'Content-Type:application/json' -X POST -d '{"load_cost_forecast":[96.0, 98.0, 28.0, 4.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 97.0, 109.0, 123.0, 108.0, 98.0, 6.0, -3.0, -4.0, -4.0, -4.0, -3.0, -1.0, -0.0, -0.0], "prod_price_forecast":[96.0, 98.0, 28.0, 4.0, 3.0, 2.0, 2.0, 2.0, 2.0, 2.0, 97.0, 109.0, 123.0, 108.0, 98.0, 6.0, -3.0, -4.0, -4.0, -4.0, -3.0, -1.0, -0.0, -0.0]}' http://localhost:5000/action/dayahead-optim