calliope-project / calliope

A multi-scale energy systems modelling framework
https://www.callio.pe
Apache License 2.0
276 stars 89 forks source link

Force minimum of one header row; Add to docs #596

Closed brynpickering closed 3 days ago

brynpickering commented 1 month ago

Fixes #573

Based on the decision in #573, I have forced the need for a header row by setting header=[0] if a user does not provide columns data in their data source config.

It can lead to a difficult to understand failure. When a user forgets a header row, data ends up as the column / index level names. This gets picked up later when Calliope tries to match index/column levels to what has been defined in the config. That seems to be the only place we can catch it, but it leads to an error like this:

calliope.exceptions.ModelError: (data_sources, ds_name) | Trying to set names for index but names in the file do no match names provided | in file: ['bar1', 'bar2'] | defined: ['test_row1', 'test_row2'].

Maybe just adding to that error with something like "Make sure you have a header row in your CSV file"?

Summary of changes in this pull request

Reviewer checklist

codecov[bot] commented 1 month ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 95.35%. Comparing base (872978d) to head (132b282). Report is 20 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #596 +/- ## ========================================== - Coverage 95.85% 95.35% -0.51% ========================================== Files 24 24 Lines 3619 3656 +37 Branches 788 733 -55 ========================================== + Hits 3469 3486 +17 + Misses 86 82 -4 - Partials 64 88 +24 ``` | [Files](https://app.codecov.io/gh/calliope-project/calliope/pull/596?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project) | Coverage Δ | | |---|---|---| | [src/calliope/preprocess/data\_sources.py](https://app.codecov.io/gh/calliope-project/calliope/pull/596?src=pr&el=tree&filepath=src%2Fcalliope%2Fpreprocess%2Fdata_sources.py&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project#diff-c3JjL2NhbGxpb3BlL3ByZXByb2Nlc3MvZGF0YV9zb3VyY2VzLnB5) | `100.00% <100.00%> (ø)` | | ... and [3 files with indirect coverage changes](https://app.codecov.io/gh/calliope-project/calliope/pull/596/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=calliope-project)