search

OSeMOSYS / osemosys_global

A global power system model generator for OSeMOSYS
https://osemosys-global.readthedocs.io/
GNU Affero General Public License v3.0
27 stars 17 forks source link

Directory Restructure #96

Closed trevorb1 closed 2 years ago

trevorb1 commented 2 years ago

I restructured the directory to follow snakemake's recommendation. The biggest changes involved splitting up the rules into smaller files, and including a workflow, resources, and results folders at the root directory to hold all files. The workflow can now be run straight from the root directory using the command snakemake -c1.

Also included in the update are better structured conda environments to avoid dependency issues. I have included two requirement files, one for conda environments (conda_requirements.txt) and one for pip environment (pip_requirements.txt). Moreover, the workflow can also install the environments for the user automatically using the --use-conda flag (ie. snakemake -c1 --use-conda)

Finally, this update addresses issue #87 which resolved itself after breaking up the workflow. And I think it also addresses the issue of having to delete the scenario results folder if you want to rerun the workflow. The file_check rule rule isn't quite as general as I think it can be, but for an initial release I think it will be fine.

I have checked that functionality of the workflow has not changed. The OsemosysGlobal.txt data file that currently gets created by the workflow in the master branch matches the data file that is created in this update.

We can discuss more in our next meeting, but I think this structure positions the workflow to be more readable and modifiable in the future (easier to run add extra config files) ! :)

trevorb1 commented 2 years ago

Have updated the dependency files and checked that they all work on my computer. In the root folder (osemosys_global/) and base conda environment, you should be able to run the workflow with the following commands....

  1. conda env create -f workflow/envs/osemosys-global.yaml
  2. conda activate osemosys-global
  3. snakemake -c

Creating the environment took a few minutes, but everything did work! If that doesnt work though, try installing snakemake then running through the --use-conda flag

  1. conda create -c conda-forge -c bioconda -n osemosys-global snakemake
  2. conda activate osemosys-global
  3. snakemake -c --use-conda

If you are finding you are waiting a long time for package installations during this troubleshooting phase, you can try installing mamba which is a faster drop-in replacement for conda and recommended by snakemake.

trevorb1 commented 2 years ago

Sina was able to execute the workflow and got the same results as me. I have also updated the installation instructions in the README.md file to reflect the directory restructuring