This is the first pull request I propose for the merge of pypsa-earth-sec into pypsa-earth. The changes are listed below:
Changes in _helpers.py
_two_digits_2_namecountry : I add a new function argument (called _namestring) to specify whether we wish to return the full name of the country or its full name. For example when _name_string=nameshort CD -> DR Congo, whereas when _name_string=nameofficial CD -> Democratic Republic of the Congo
_preparecosts: I add this method from the helpers.py scripts of pypsa-earth-sec
_create_networktopology: I add this method from the helpers.py script of pypsa-earth-sec
_cyclingshift: I add this method from the helpers.py script of pypsa-earth-sec
_downloadgadm: I add this method from the helpers.py script of pypsa-earth-sec
_get_gadmlayer: I add this method from the helpers.py script of pypsa-earth-sec
_locatebus: I add this method from the helpers.py script of pypsa-earth-sec
_safedivide: I add this method from the helpers.py script of pypsa-earth-sec
I created the issue https://github.com/pypsa-meets-earth/pypsa-earth-sec/issues/310 in pypsa-earth-sec. In a nutshell, one of the inputs used in pypsa-earth-sec for _rule build_base_energytotals contains a few typos (e.g. "Hrad coal") and many duplicated entries (e.g. "Coke-oven coke", "Coke oven coke" and "Coke Oven Coke"). Such typos and duplicated entries have been propagated in the method _get_convfactors and in the method _aggregatefuels. Moreover, _aggregatefuels contains two mutually exclusive definitions for the list _coalfuels. Finally, the method _aggregatefuels does not use the function argument sector.
The above-mentioned input is composed of several text files, which are read into a pandas dataframe. The processing happens in _scripts/build_base_energytotals.py in pypsa-earth-sec. The entries "Hrad coal", "Coke-oven coke", "Coke oven coke" and "Coke Oven Coke" etc. populate the column "Commodity" of the pandas dataframe.
My proposed solution is to leave the input files untouched and to modify the pandas dataframe, replacing the typos and lower-casing all entries of column "Commodity". I implement the solution with the following methods:
_modifycommodity: I create the method to replace the typos and lower-case the entries of the "Commodity" column
_get_convfactors: I add this method from the helpers.py script of pypsa-earth-sec. I lower case all keys of the dictionary and correct the typos.
_aggregatefuels: I add this method from the helpers.py script of pypsa-earth-sec. I lower case all entries of the lists, I unify the mutually exclusive definitions of list _coalfuels into one. I modify the method such that it returns the defined lists when sector==industry. This is because _get_convfactors and _aggregatefuels are used in conjunction. Hence it makes sense that they both have the same behavior with respect to the value of sector.
Unit tests for the _helpers.py methods
I have added dedicated unit tests for some of the methods present in the _helpers.py script. In particular I test that the _modifycommodity method yields the requested changes to the "Commodity" column of the pandas dataframe.
Further notes
_create_dummydata(n, sector, carriers): I did not copy this method from pypsa-earth-sec/scripts/helpers.py because it is not used in pypsa-earth-sec
_getcountry(target, **keys): this method from pypsa-earth-sec/scripts/helpers.py is not needed
Checklist
[x] I consent to the release of this PR's code under the AGPLv3 license and non-code contributions under CC0-1.0 and CC-BY-4.0.
[x] I tested my contribution locally and it seems to work fine.
[x] Code and workflow changes are sufficiently documented.
[ ] Newly introduced dependencies are added to envs/environment.yaml and doc/requirements.txt.
[ ] Changes in configuration options are added in all of config.default.yaml and config.tutorial.yaml.
[ ] Add a test config or line additions to test/ (note tests are changing the config.tutorial.yaml)
[ ] Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
[x] A note for the release notes doc/release_notes.rst is amended in the format of previous release notes, including reference to the requested PR.
Closes # (if applicable).
Together with the merge of the methods from the helpers.py script of pypsa-earth-sec into the _helpers.py script of pypsa-earth, the pull request proposes also a possible solution for https://github.com/pypsa-meets-earth/pypsa-earth-sec/issues/310.
Changes proposed in this Pull Request
This is the first pull request I propose for the merge of pypsa-earth-sec into pypsa-earth. The changes are listed below:
Changes in _helpers.py
Proposed resolution of issue https://github.com/pypsa-meets-earth/pypsa-earth-sec/issues/310
I created the issue https://github.com/pypsa-meets-earth/pypsa-earth-sec/issues/310 in pypsa-earth-sec. In a nutshell, one of the inputs used in pypsa-earth-sec for _rule build_base_energytotals contains a few typos (e.g. "Hrad coal") and many duplicated entries (e.g. "Coke-oven coke", "Coke oven coke" and "Coke Oven Coke"). Such typos and duplicated entries have been propagated in the method _get_convfactors and in the method _aggregatefuels. Moreover, _aggregatefuels contains two mutually exclusive definitions for the list _coalfuels. Finally, the method _aggregatefuels does not use the function argument sector.
The above-mentioned input is composed of several text files, which are read into a pandas dataframe. The processing happens in _scripts/build_base_energytotals.py in pypsa-earth-sec. The entries "Hrad coal", "Coke-oven coke", "Coke oven coke" and "Coke Oven Coke" etc. populate the column "Commodity" of the pandas dataframe.
My proposed solution is to leave the input files untouched and to modify the pandas dataframe, replacing the typos and lower-casing all entries of column "Commodity". I implement the solution with the following methods:
Unit tests for the _helpers.py methods
I have added dedicated unit tests for some of the methods present in the _helpers.py script. In particular I test that the _modifycommodity method yields the requested changes to the "Commodity" column of the pandas dataframe.
Further notes
Checklist
envs/environment.yamlanddoc/requirements.txt.config.default.yamlandconfig.tutorial.yaml.test/(note tests are changing the config.tutorial.yaml)doc/configtables/*.csvand line references are adjusted indoc/configuration.rstanddoc/tutorial.rst.doc/release_notes.rstis amended in the format of previous release notes, including reference to the requested PR.