Fix Preset validation errors

antw commented 10 years ago

This is a summary of Preset validation errors in ETSource. These fall into two categories: a preset specifies an input which does not exist (probably deleted since creation of the preset), or it contains values for inputs belonging to a share group, and the group does not sum to 100.

@ChaelKruip There's no useful Git history (I expect these changes predate Atlas and ActiveDocuments); is there any chance you are able to tell me which of the inputs should be removed, and which are renamed? If that information is available, I expect I can do much of the grunt work myself.

Summary:

Missing inputs:

industry_number_of_biomass_chp - 43 times
number_of_micro_chp_fixed - 3 times
number_of_small_chp_fixed - 3 times
co_firing_wood_pellets_share - 2 times
households_useful_demand_cooking - 2 times
industry_useful_demand_useable_heat_efficiencty - 2 times
number_of_biomass_chp_fixed - 2 times
number_of_biomass_fired_heater_fixed - 2 times
number_of_electric_heat_pump_fixed - 2 times
number_of_geothermal_fixed - 2 times
number_of_solar_pv_roofs_fixed - 2 times
number_of_solar_water_heater_fixed - 2 times
buildings_cooling_airconditioning_electricity_share - 1 times
buildings_cooling_collective_cooling_network_electricity_share - 1 times
buildings_lighting_led_electricity_tube_share - 1 times
households_heating_solar_thermal_panels_share - 1 times
number_of_backup_burner_fixed - 1 times
number_of_co_firing_coal - 1 times
number_of_coal_fired_heaterv - 1 times
number_of_gas_fired_heat_pump_fixed - 1 times
number_of_gas_fired_heater_fixed - 1 times
number_of_lignite_chp_fixed - 1 times
number_of_oil_fired_heater_fixed - 1 times
number_of_small_gas_chp_fixed - 1 times

Imbalanced share groups:

transport_car_tech - 19 times
transport_marine - 16 times
agri_heat - 11 times
cooking - 7 times
lighting_buildings - 4 times
aluminium_production - 2 times
household_light - 2 times
energy_sector_gas - 1 times
steel_production - 1 times

(removed lengthy error list; see latest post for latest results)

ChaelKruip commented 10 years ago

To be removed:

industry_number_of_biomass_chp - 43 times
number_of_micro_chp_fixed - 3 times
number_of_small_chp_fixed - 3 times
co_firing_wood_pellets_share - 2 times
number_of_biomass_chp_fixed - 2 times
number_of_biomass_fired_heater_fixed - 2 times
number_of_electric_heat_pump_fixed - 2 times
number_of_geothermal_fixed - 2 times
number_of_solar_pv_roofs_fixed - 2 times
number_of_solar_water_heater_fixed - 2 times
buildings_cooling_collective_cooling_network_electricity_share - 1 times
number_of_backup_burner_fixed - 1 times
number_of_co_firing_coal - 1 times
number_of_gas_fired_heat_pump_fixed - 1 times
number_of_gas_fired_heater_fixed - 1 times
number_of_lignite_chp_fixed - 1 times
number_of_oil_fired_heater_fixed - 1 times
number_of_small_gas_chp_fixed - 1 times

Renamed:

households_useful_demand_cooking => households_useful_demand_cooking_per_person
industry_useful_demand_useable_heat_efficiencty => industry_useful_demand_useable_heat_efficiency
buildings_cooling_airconditioning_electricity_share => buildings_cooling_airconditioning_share
buildings_lighting_led_electricity_tube_share => buildings_lighting_led_electricity_share
households_heating_solar_thermal_panels_share => households_water_heater_solar_thermal_share
number_of_coal_fired_heaterv => number_of_energy_heater_for_heat_network_coal

ChaelKruip commented 10 years ago

@antw thanks for this :smile: I'll look at the rest tomorrow

antw commented 10 years ago

thanks for this

Sorry. :disappointed:

Removing / renaming the inputs fixed all of the "does not exist" messages. Unfortunately all the share group errors remain. In many cases, the preset omits a value for some inputs in the group; it's possible that filling in the omitted inputs with the default value form ETEngine will resolve some errors. I'll try that tomorrow.

Summary:

Imbalanced share groups:

transport_car_tech - 19 times
transport_marine - 16 times
agri_heat - 11 times
cooking - 7 times
lighting_buildings - 4 times
aluminium_production - 2 times
household_light - 2 times
energy_sector_gas - 1 times
steel_production - 1 times

(removed lengthy error list; see latest post for latest results)

antw commented 10 years ago

it's possible that filling in the omitted inputs with the default value form ETEngine will resolve some errors. I'll try that tomorrow.

Did that; the imbalances are looking a lot better now, and many that remain are quite small:

Summary:

Imbalanced share groups:

transport_car_tech - 11 times
agri_heat - 7 times
lighting_buildings - 4 times
cooking - 3 times

anton_broenink (presets/watt_nu/2013/anton_broenink.ad)

user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (5.0)
- transport_car_using_diesel_mix_share (30.6)
- transport_car_using_electricity_share (25.0)
- transport_car_using_gasoline_mix_share (34.8)
- transport_car_using_lpg_share (4.459272277233526)

bernard_fortuyn (presets/watt_nu/2012/bernard_fortuyn.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 93.09033373131207, not 100

specified:
- agriculture_burner_crude_oil_share (0.0)
- agriculture_burner_network_gas_share (0.0)
- agriculture_burner_wood_pellets_share (2.2)
- agriculture_final_demand_steam_hot_water_share (50.48784757625757)
- agriculture_geothermal_share (40.0)
- agriculture_heatpump_water_water_ts_electricity_share (0.4024861550544924)

gertjan_lankhorst (presets/watt_nu/2012/gertjan_lankhorst.ad)

user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (5.0)
- transport_car_using_diesel_mix_share (30.6)
- transport_car_using_electricity_share (25.0)
- transport_car_using_gasoline_mix_share (34.8)
- transport_car_using_lpg_share (4.459272277233526)

harm_post (presets/watt_nu/2012/harm_post.ad)

user_values contains inputs belonging to the lighting_buildings share group, but the values sum to 91.69999999999999, not 100

specified:
- buildings_lighting_efficient_fluorescent_electricity_share (15.699999999999987)
- buildings_lighting_led_electricity_share (5.0)
- buildings_lighting_standard_fluorescent_electricity_share (71.0)

jan_borghuis (presets/watt_nu/2012/jan_borghuis.ad)

user_values contains inputs belonging to the lighting_buildings share group, but the values sum to 91.69999999999999, not 100

specified:
- buildings_lighting_efficient_fluorescent_electricity_share (15.699999999999987)
- buildings_lighting_led_electricity_share (50.0)
- buildings_lighting_standard_fluorescent_electricity_share (26.0)
user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (15.0)
- transport_car_using_diesel_mix_share (5.0)
- transport_car_using_electricity_share (25.0)
- transport_car_using_gasoline_mix_share (50.4)
- transport_car_using_lpg_share (4.459272277233526)

jolande_sap (presets/watt_nu/2012/jolande_sap.ad)

user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (10.0)
- transport_car_using_diesel_mix_share (0.0)
- transport_car_using_electricity_share (80.0)
- transport_car_using_gasoline_mix_share (5.4)
- transport_car_using_lpg_share (4.459272277233526)

liesbeth_van_tongeren (presets/watt_nu/2013/liesbeth_van_tongeren.ad)

user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (10.0)
- transport_car_using_diesel_mix_share (0.0)
- transport_car_using_electricity_share (80.0)
- transport_car_using_gasoline_mix_share (5.4)
- transport_car_using_lpg_share (4.459272277233526)

marco_witschge (presets/watt_nu/2012/marco_witschge.ad)

user_values contains inputs belonging to the cooking share group, but the values sum to 99.77591213995785, not 100

specified:
- households_cooker_halogen_electricity_share (7.928599391370579)
- households_cooker_induction_electricity_share (67.6)
- households_cooker_network_gas_share (8.4)
- households_cooker_resistive_electricity_share (15.847312748587278)
- households_cooker_wood_pellets_share (0.0)

paulus_jansen (presets/watt_nu/2012/paulus_jansen.ad)

user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (10.0)
- transport_car_using_diesel_mix_share (35.0)
- transport_car_using_electricity_share (30.0)
- transport_car_using_gasoline_mix_share (20.4)
- transport_car_using_lpg_share (4.459272277233526)

peter_molengraaf (presets/watt_nu/2012/peter_molengraaf.ad)

user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723352, not 100

specified:
- transport_car_using_compressed_natural_gas_share (15.7)
- transport_car_using_diesel_mix_share (22.2)
- transport_car_using_electricity_share (33.3)
- transport_car_using_gasoline_mix_share (24.2)
- transport_car_using_lpg_share (4.459272277233526)

pieter_verberne (presets/watt_nu/2012/pieter_verberne.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 93.08784757625757, not 100

specified:
- agriculture_burner_crude_oil_share (0.0)
- agriculture_burner_network_gas_share (27.6)
- agriculture_burner_wood_pellets_share (5.0)
- agriculture_final_demand_steam_hot_water_share (50.48784757625757)
- agriculture_geothermal_share (5.0)
- agriculture_heatpump_water_water_ts_electricity_share (5.0)

rene_leegte (presets/watt_nu/2012/rene_leegte.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 93.28386232714178, not 100

specified:
- agriculture_burner_crude_oil_share (0.0)
- agriculture_burner_network_gas_share (20.3)
- agriculture_burner_wood_pellets_share (0.7935285958297138)
- agriculture_final_demand_steam_hot_water_share (50.48784757625757)
- agriculture_geothermal_share (21.3)
- agriculture_heatpump_water_water_ts_electricity_share (0.4024861550544924)
user_values contains inputs belonging to the lighting_buildings share group, but the values sum to 91.69999999999999, not 100

specified:
- buildings_lighting_efficient_fluorescent_electricity_share (15.699999999999987)
- buildings_lighting_led_electricity_share (67.6)
- buildings_lighting_standard_fluorescent_electricity_share (8.4)

richard_kooloos (presets/watt_nu/2012/richard_kooloos.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 93.33220189587564, not 100

specified:
- agriculture_burner_crude_oil_share (6.5)
- agriculture_burner_network_gas_share (11.1)
- agriculture_burner_wood_pellets_share (24.5)
- agriculture_final_demand_steam_hot_water_share (50.48784757625757)
- agriculture_geothermal_share (0.34186816456358055)
- agriculture_heatpump_water_water_ts_electricity_share (0.4024861550544924)
user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (17.6)
- transport_car_using_diesel_mix_share (23.2)
- transport_car_using_electricity_share (15.7)
- transport_car_using_gasoline_mix_share (38.9)
- transport_car_using_lpg_share (4.459272277233526)

romeo_malizia (presets/watt_nu/2012/romeo_malizia.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 93.08784757625756, not 100

specified:
- agriculture_burner_crude_oil_share (0.0)
- agriculture_burner_network_gas_share (0.0)
- agriculture_burner_wood_pellets_share (9.7)
- agriculture_final_demand_steam_hot_water_share (50.48784757625757)
- agriculture_geothermal_share (24.8)
- agriculture_heatpump_water_water_ts_electricity_share (8.1)
user_values contains inputs belonging to the lighting_buildings share group, but the values sum to 91.69999999999999, not 100

specified:
- buildings_lighting_efficient_fluorescent_electricity_share (15.699999999999987)
- buildings_lighting_led_electricity_share (55.7)
- buildings_lighting_standard_fluorescent_electricity_share (20.3)
user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (17.7)
- transport_car_using_diesel_mix_share (44.5)
- transport_car_using_electricity_share (5.0)
- transport_car_using_gasoline_mix_share (28.2)
- transport_car_using_lpg_share (4.459272277233526)

ron_wit (presets/watt_nu/2013/ron_wit.ad)

user_values contains inputs belonging to the cooking share group, but the values sum to 99.77591213995791, not 100

specified:
- households_cooker_halogen_electricity_share (7.928599391370579)
- households_cooker_induction_electricity_share (15.847312748587328)
- households_cooker_network_gas_share (17.5)
- households_cooker_resistive_electricity_share (58.5)
- households_cooker_wood_pellets_share (0.0)

tjerk_wagenaar (presets/watt_nu/2012/tjerk_wagenaar.ad)

user_values contains inputs belonging to the cooking share group, but the values sum to 99.77591213995791, not 100

specified:
- households_cooker_halogen_electricity_share (7.928599391370579)
- households_cooker_induction_electricity_share (15.847312748587328)
- households_cooker_network_gas_share (17.5)
- households_cooker_resistive_electricity_share (58.5)
- households_cooker_wood_pellets_share (0.0)

vera_dalm (presets/watt_nu/2013/vera_dalm.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 100.43788291544779, not 100

specified:
- agriculture_burner_crude_oil_share (8.2)
- agriculture_burner_network_gas_share (0.0)
- agriculture_burner_wood_pellets_share (0.7935285958297138)
- agriculture_final_demand_steam_hot_water_share (90.7)
- agriculture_geothermal_share (0.34186816456358055)
- agriculture_heatpump_water_water_ts_electricity_share (0.4024861550544924)
user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (30.0)
- transport_car_using_diesel_mix_share (0.0)
- transport_car_using_electricity_share (30.0)
- transport_car_using_gasoline_mix_share (35.4)
- transport_car_using_lpg_share (4.459272277233526)

vera_dalm_2012 (presets/watt_nu/2012/vera_dalm_2012.ad)

user_values contains inputs belonging to the agri_heat share group, but the values sum to 100.43788291544779, not 100

specified:
- agriculture_burner_crude_oil_share (8.2)
- agriculture_burner_network_gas_share (0.0)
- agriculture_burner_wood_pellets_share (0.7935285958297138)
- agriculture_final_demand_steam_hot_water_share (90.7)
- agriculture_geothermal_share (0.34186816456358055)
- agriculture_heatpump_water_water_ts_electricity_share (0.4024861550544924)
user_values contains inputs belonging to the transport_car_tech share group, but the values sum to 99.85927227723353, not 100

specified:
- transport_car_using_compressed_natural_gas_share (30.0)
- transport_car_using_diesel_mix_share (0.0)
- transport_car_using_electricity_share (30.0)
- transport_car_using_gasoline_mix_share (35.4)
- transport_car_using_lpg_share (4.459272277233526)

ChaelKruip commented 10 years ago

Sorry. :disappointed:

I wasn't being sarcastic! I really like the fact that you are tackling this issue :smiley:

My approach for the share groups that add up to 100% is to

Take the default value for missing inputs (as you suggest)
Proportionally distribute the discrepancy over the members of the group (in all other cases)

ChaelKruip commented 10 years ago

Closed with https://github.com/quintel/etsource/commit/b8d65d510f3f8c030beb4edbd8b268aaaadb600e

quintel / etsource