Features/#1001 desaggregate industry demands

nesnoj commented 1 year ago

Fixes #1001

Before merging into `dev`-branch, please make sure that

[x] the CHANGELOG.rst was updated.
[x] new and adjusted code is formated using black and isort.
[x] the Dataset-version is updated when existing datasets are adjusted.
[x] the branch was merged into the continuous-integration/run-everything-over-the-weekend-branch.
[x] the workflow is running successful in test mode.
[x] the workflow is running successful in Everything mode.

nesnoj commented 1 year ago

@IlkaCu I compared the results (new server):

DR

SELECT scenario, sum(demand)/1e6 as demand_total
FROM demand.egon_demandregio_cts_ind
WHERE wz in (SELECT wz FROM demand.egon_demandregio_wz WHERE sector='industry')
GROUP BY scenario;

scenario	demand_total
"eGon2035"	257.1342536669533
"eGon2021"	276.4958651938644
"eGon100RE"	359.5375072681997

New tables

SELECT d1.scn_name, d1.demand+d2.demand as demand_total
FROM
    (
        SELECT scn_name, sum(demand)/1e6 as demand
        FROM demand.egon_sites_ind_load_curves_individual
        GROUP BY scn_name
    ) AS d1,
    (
        SELECT scn_name, sum(demand)/1e6 as demand
        FROM demand.egon_osm_ind_load_curves_individual
        GROUP BY scn_name
    ) AS d2
WHERE d1.scn_name = d2.scn_name
;

scenario	demand_total
"eGon2035"	234.38934296805377
"eGon100RE"	329.11333557404123

Any ideas where the difference may origin from?

khelfen commented 1 year ago

The new table egon_sites_ind_load_curves_individual is missing information about the Wirtschaftszweig wz which is needed for the DSM tasks. Could you please add this column to the table?

IlkaCu commented 1 year ago

I pushed some code changes which, in my opinion, solve the problem - some ehv loads were neglected so far. I also added a column 'wz' for the loads assigned to industrial sites. This wasn't merged and tested on the CI branch yet. Do you have a chance to test the feature on one of your servers before merging it?

nesnoj commented 1 year ago

I pushed some code changes which, in my opinion, solve the problem - some ehv loads were neglected so far. I also added a column 'wz' for the loads assigned to industrial sites. This wasn't merged and tested on the CI branch yet. Do you have a chance to test the feature on one of your servers before merging it?

Yes, I will test it

nesnoj commented 1 year ago

I pushed some code changes which, in my opinion, solve the problem - some ehv loads were neglected so far. I also added a column 'wz' for the loads assigned to industrial sites. This wasn't merged and tested on the CI branch yet. Do you have a chance to test the feature on one of your servers before merging it?

Yes, I will test it

Results look good for SH, thank you @IlkaCu!

nesnoj commented 1 year ago

@IlkaCu L253 in https://github.com/openego/eGon-data/blob/effe60dfacb56f47469f0754a391b13b7e9b9781/src/egon/data/datasets/industry/__init__.py#L251-L257 is not used, can it be removed or shall landuse_nuts3 be used in L254? (it worked until now so I guess we can drop it)

nesnoj commented 1 year ago

@IlkaCu In 8078ef54a31d681e44c5ed13fef45ddabd07fe8d I introduced a new scenario "eGon2021", I hope it won't cause any problems at your end? (SH run is ongoing..)

nesnoj commented 1 year ago

@IlkaCu In 8078ef5 I introduced a new scenario "eGon2021", I hope it won't cause any problems at your end? (SH run is ongoing..)

I only see the dataset ElectricalLoadEtrago using the mentioned tables demand.egon_osm_ind_load_curves and demand.egon_sites_ind_load_curves. In there you explicitly use the scenario names.

Please let me know whether you found other dependent data or not.

IlkaCu commented 1 year ago

@IlkaCu In 8078ef5 I introduced a new scenario "eGon2021", I hope it won't cause any problems at your end? (SH run is ongoing..)

I only see the dataset ElectricalLoadEtrago using the mentioned tables demand.egon_osm_ind_load_curves and demand.egon_sites_ind_load_curves. In there you explicitly use the scenario names.

Please let me know whether you found other dependent data or not.

I agree. The code should be able to handle additional scenarios as it is.

nesnoj commented 1 year ago

@IlkaCu In 8078ef5 I introduced a new scenario "eGon2021", I hope it won't cause any problems at your end? (SH run is ongoing..)

I only see the dataset ElectricalLoadEtrago using the mentioned tables demand.egon_osm_ind_load_curves and demand.egon_sites_ind_load_curves. In there you explicitly use the scenario names. Please let me know whether you found other dependent data or not.

I agree. The code should be able to handle additional scenarios as it is.

Great, thanks for checking! :) I'll merge this tonight.

openego / eGon-data