calliope-project / calliope

A multi-scale energy systems modelling framework
https://www.callio.pe
Apache License 2.0
287 stars 93 forks source link

New model definition structure #518

Closed brynpickering closed 9 months ago

brynpickering commented 11 months ago

This is a biggie.

Fixes issue #324 Fixes issue #441 Fixes issue #337 Fixes issue #362 Fixes issue #263 Fixes issue #230

Adresses discussions #483, #484,

Summary of changes in this pull request:

Summary of things still to do:

EDIT: done some

* Fix one_way parameter so it actually does something (I know how, just need to do it) * Add docstrings (sorry sorry) Add test for new helper function which handles referencing a variable/expression that may not be built for a given model (e.g., referencing `storage_cap cost_storage_cap` when the model includes no techs with storage) * Add check for referencing transmission techs in nodes - this should not be allowed. Add check for referencing a different parent in different nodes. Although it's technically OK to do this, having to infer the number of dimensions on the resulting array is a pain (ideally they are only* indexed over techs).

Things that I won't do in this PR

Reviewer checklist:

brynpickering commented 11 months ago

The new global expressions (particularly the flow_in_inc_eff/flow_out_inc_eff ones) increase the memory footprint of models since they keep more unique pyomo objects in memory. For the example models (1yr):

urban scale: 14% increase national scale: 2% increase

^ I would guess that this is the effect of the extra carriers in the urban scale model.

brynpickering commented 11 months ago

The linux py3.9 failure is due to type hinting (I'm surprised it took this long for it to have a problem with my use of pipes for unions). See #519 for proposal to kill py3.9 in favour of py3.12 support.

codecov[bot] commented 11 months ago

Codecov Report

Attention: 33 lines in your changes are missing coverage. Please review.

Comparison is base (3b84476) 93.42% compared to head (7592368) 94.98%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #518 +/- ## ========================================== + Coverage 93.42% 94.98% +1.55% ========================================== Files 28 24 -4 Lines 3804 3290 -514 Branches 873 648 -225 ========================================== - Hits 3554 3125 -429 + Misses 162 96 -66 + Partials 88 69 -19 ``` | [Files](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project) | Coverage Δ | | |---|---|---| | [src/calliope/\_\_init\_\_.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL19faW5pdF9fLnB5) | `100.00% <100.00%> (ø)` | | | [src/calliope/attrdict.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2F0dHJkaWN0LnB5) | `96.44% <100.00%> (ø)` | | | [src/calliope/backend/expression\_parser.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2JhY2tlbmQvZXhwcmVzc2lvbl9wYXJzZXIucHk=) | `97.14% <100.00%> (-0.86%)` | :arrow_down: | | [src/calliope/backend/helper\_functions.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2JhY2tlbmQvaGVscGVyX2Z1bmN0aW9ucy5weQ==) | `97.75% <100.00%> (-0.06%)` | :arrow_down: | | [src/calliope/backend/latex\_backend\_model.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2JhY2tlbmQvbGF0ZXhfYmFja2VuZF9tb2RlbC5weQ==) | `96.00% <100.00%> (ø)` | | | [src/calliope/backend/parsing.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2JhY2tlbmQvcGFyc2luZy5weQ==) | `98.66% <ø> (ø)` | | | [src/calliope/backend/pyomo\_backend\_model.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2JhY2tlbmQvcHlvbW9fYmFja2VuZF9tb2RlbC5weQ==) | `94.52% <100.00%> (-0.29%)` | :arrow_down: | | [src/calliope/backend/where\_parser.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2JhY2tlbmQvd2hlcmVfcGFyc2VyLnB5) | `97.70% <100.00%> (+0.03%)` | :arrow_up: | | [src/calliope/cli.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2NsaS5weQ==) | `81.67% <100.00%> (ø)` | | | [src/calliope/core/\_\_init\_\_.py](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL2NvcmUvX19pbml0X18ucHk=) | `100.00% <100.00%> (ø)` | | | ... and [11 more](https://app.codecov.io/gh/calliope-project/calliope/pull/518?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project) | | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/calliope-project/calliope/pull/518/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project)