search

OSeMOSYS / osemosys_global

A global power system model generator for OSeMOSYS
https://osemosys-global.readthedocs.io/
GNU Affero General Public License v3.0
27 stars 17 forks source link

Work out where to host the data, how to deploy it, and declare licenses etc. #39

Open willu47 opened 3 years ago

willu47 commented 3 years ago

At the moment, data is stored in the data folder in the repository.

When installing as a Python library, this data is not included, and so all local references to data break. There are a few workarounds:

  1. include all the data inside the Python library so it is installed with the package
  2. host the data on a web-server, or provide links to online sources for all the data
  3. distribute the data in an zip archive, and get users to place somewhere manually

In terms of pros/cons:

  1. a bit of a hack, users cannot see the data (e.g. identify data issues), bulky codebase, and an unhealthy mix of code and data;
  2. potentially brittle, as broken links to other sources would break all installed versions of the package; potential licensing issues if data is not open? However, all users would benefit from central updates to data;
  3. simplest, need to clear data licences, messy install;
maartenbrinkerink commented 2 months ago

Similar to #34 and #35, i've moved this to the 'Longer term developments' milestone as turning OG into a useable python library has a bit lower priority at the moment due to the constrained development capacity. If there is bandwidth within one of the collaborating institutions it would be very useful though! If there is anything to be added to this issue @trevorb1 feel free.

willu47 commented 2 months ago

Another option not mentioned above is:

  1. Add a command line interface argument which downloads the data and configures the Snakemake workflow