This is a small PR that just does the minimum updates to download 2023 data and run the pipeline, without actually fixing any data quality issues yet. The pipeline did successfully complete but there are a number or errors raised, including a large amount of negative fuel in MISO. It is possible that these will be fixed once we properly go through the entire update process.
Advances CAR-4691
What the code is doing
How is the purpose executed?
Testing
Ran the pipeline
Where to look
It's helpful to clarify where your new code lives if you moved files around or there could be confusion/
What files are most important?
Usage Example/Visuals
How the code can be used and/or images of any graphs, tables or other visuals (not always applicable).
Review estimate
5 minutes
Future work
The data pipeline log file was over 300MB because it output a large amount of hourly data in the warning. We should limit these warnings in the future to just print a sample of the data
Checklist
[x] Update the documentation to reflect changes made in this PR
[x] Format all updated python files using black
[x] Clear outputs from all notebooks modified
[x] Add docstrings and type hints to any new functions created
Purpose
This is a small PR that just does the minimum updates to download 2023 data and run the pipeline, without actually fixing any data quality issues yet. The pipeline did successfully complete but there are a number or errors raised, including a large amount of negative fuel in MISO. It is possible that these will be fixed once we properly go through the entire update process. Advances CAR-4691
What the code is doing
How is the purpose executed?
Testing
Ran the pipeline
Where to look
It's helpful to clarify where your new code lives if you moved files around or there could be confusion/
What files are most important?
Usage Example/Visuals
How the code can be used and/or images of any graphs, tables or other visuals (not always applicable).
Review estimate
5 minutes
Future work
The data pipeline log file was over 300MB because it output a large amount of hourly data in the warning. We should limit these warnings in the future to just print a sample of the data
Checklist
black