Allow to run pipeline for years 2005, 2006 and 2007. Closes CAR-4211.
Add missing typehints and add/format docstrings in all functions located in the load_data and download_data modules.
What the code is doing
Print a warning message and return download_data.download_raw_eia923 instead of raising an error for years < 2008
Updates reference tables to add the SC energy source code that is used for years < 2008 in place of SGC
Testing
Run 2005, 2006 and 2007 pipelines successfully.
Where to look
the download_data.download_raw_eia923 where we warn the file is not available and return
the load_data.load_pudl_table where a warning is added if the table is empty
the 2 reference tables (CSV files) where we added information for the SC energy code.
Usage Example/Visuals
From log file:
2024-05-17 11:21:37,194 [WARNING] oge.oge.download_data:313 EIA-923 data is unavailable for 2006.
...
2024-05-17 11:22:32,070 [INFO] oge.data_pipeline:169 3. Cleaning EIA-923 data
2024-05-17 11:22:33,969 [WARNING] oge.oge.load_data:670 denorm_boiler_fuel_monthly_eia923 is empty
2024-05-17 11:22:34,136 [WARNING] oge.oge.load_data:670 denorm_generation_monthly_eia923 is empty
2024-05-17 11:22:34,660 [WARNING] oge.oge.load_data:670 boiler_generator_assn_eia860 is empty
Review estimate
2 min if you don't pay attention to added/formatted docstrings. 20min otherwise
Future work
Add information in the plant attributes file.
Checklist
[ ] Update the documentation to reflect changes made in this PR
The README will be added in a separate PR before merging the historical_coverage_feature branch into development
[x] Format all updated python files using black
[x] Clear outputs from all notebooks modified
[x] Add docstrings and type hints to any new functions created
Purpose
Allow to run pipeline for years 2005, 2006 and 2007. Closes CAR-4211.
Add missing typehints and add/format docstrings in all functions located in the
load_data
anddownload_data
modules.What the code is doing
download_data.download_raw_eia923
instead of raising an error for years < 2008SC
energy source code that is used for years < 2008 in place ofSGC
Testing
Run 2005, 2006 and 2007 pipelines successfully.
Where to look
download_data.download_raw_eia923
where we warn the file is not available and returnload_data.load_pudl_table
where a warning is added if the table is emptySC
energy code.Usage Example/Visuals
From log file:
Review estimate
2 min if you don't pay attention to added/formatted docstrings. 20min otherwise
Future work
Add information in the plant attributes file.
Checklist
historical_coverage_feature
branch intodevelopment
black