search

calliope-project / euro-calliope

A workflow to build models of the European electricity system for Calliope.
https://euro-calliope.readthedocs.io
MIT License
31 stars 18 forks source link

Publish euro-calliope-lib as an individual package #207

Open timtroendle opened 2 years ago

timtroendle commented 2 years ago

Packaging euro-calliope-lib within the Euro-Calliope workflow is causing issues. First, it increases the complexity of the repository as the distinction of scipts and lib may not always be obvious. Second, it increases the complexity of the test infrastructure as the library code requires a different way of testing as other code. Third, it complicates the environment creation as we need relative file paths for installing the lib (an unfortunate decision of Snakemake to copy the env file makes this problem worse).

Instead, we should publish euro-calliope-lib as an individual package. The library would move to its own Git repository. On release, we publish the library to Zenodo from which we can install through pip in the env files (pip install https://zenodo.org/record/xxxxxx/files/calliope-project/euro-calliope-lib-v1.2.0.zip). During development, one can either point to a local path again, or push to Github and install from there.

There are three disadvantages of this approach: First, technical maintenance requires slightly more effort. Second, the larger structural break makes development slightly more complicated ("Which functions exist in the lib?", "What is the name of fucntion x?"). And third, documentation efforts are higher -- as the lib will require better documentation once shipped individually.

Version

1.2.0.dev

timtroendle commented 2 years ago

This is causing issues in balancing-act, so I am pushing this up.

brynpickering commented 2 years ago

to mitigate the issue of a structural break, could we have a sort of 'libification' at every major release? That is, if there are functions that we need to duplicate in multiple euro-calliope scripts then we do so during a development round, building up a list of TODOs for them to move to euro-calliope-lib and then one of the steps for version release is to move those functions to euro-calliope-lib in one go. That way, we minimise the confusion on what is and isn't in euro-calliope-lib at any given time?