Monte-Carlo improvement

[pz-max]

Below I am tracking a PR that we merged in an incomplete state. Because the cleaning process drops all PRs we decided to merge this PR anyways while keeping track of the necessary tasks/possible improvements. Once the rebranding/cleaning of pypsa-meets-earth is done, the issues/parts of it can be tackled in new PR's

Changes proposed in this Pull Request

Only a draft and concept PR which adds a flexible monte-carlo optimization. The implementation works already on a playground repository: https://github.com/pz-max/uncertainty-esm. Remaining TODO's for this PR:

• [x] implement input/output requirements in Snakefile
• [x] make sure that parallel computation works
• [x] add documentation
• [x] add separate test
• [ ] improve the test

Future PR:

• [ ] Make sure the problem scales well on HPC's
• [ ] Improve the recommended sampling number according to the paper
• [ ] The current monte-carlo approaches samples from uniform distributions. Some options for other probability distributions could be added e.g [l_bound, u_bound, zeta, gamma, distribution_type]
• [ ] Support inputs to simulate outages of biggest lines "lines.p_nom_opt.max(): [-3000MW 0MW]"" or biggest power plant "generators.p_nom.max()": [-1000MW 0MW]
• [ ] Generalize the near-optimal(MGA)/ broad ranges part (https://github.com/fneum/broad-ranges)
• [x] Avoid tests/config_test2

Checklist

• [x] I tested my contribution locally and it seems to work fine.
• [x] Code and workflow changes are sufficiently documented.
• [x] Newly introduced dependencies are added to envs/environment.yaml and envs/environment.docs.yaml.
• [x] Changes in configuration options are added in all of config.default.yaml, config.tutorial.yaml, and test/config.test1.yaml.
• [x] Changes in configuration options are also documented in doc/configtables/*.csv and line references are adjusted in doc/configuration.rst and doc/tutorial.rst.
• [ ] A note for the release notes doc/release_notes.rst is amended in the format of previous release notes, including reference to the requested PR.

  maybe we can open yaml default turn on Monte Carlo & create copy called config.yaml that will be executed

  This was done here: https://github.com/pypsa-meets-africa/pypsa-earth-sec/blob/163648f3657135739251a93ad9c848275a228c8e/Snakefile#L490 as mentioned by Davide

[davide-f]

@pz-max can we close this?

[pz-max]

@pz-max can we close this?

Let's keep it as feature request @davide-f . The monte-carlo part still needs improvements. Few of them that we track here