singularity-energy / open-grid-emissions

Tools for producing high-quality hourly generation and emissions data for U.S. electric grids
MIT License
67 stars 4 forks source link

Small improvements to command line arguments #288

Closed miloknowles closed 1 year ago

miloknowles commented 1 year ago

I was originally debugging the --small option for the data pipeline, which crashes at the EIA-930 cleaning step. During that process, I made a couple improvements to how command line arguments are handled in data_pipeline.py:

EDIT: Added one more change to this PR:

miloknowles commented 1 year ago

FWIW, the --small option only works right now if you also do --flat so that EIA-930 cleaning is skipped.

grgmiller commented 1 year ago

Thanks, Milo. I'll take a look. I think that maybe we may get rid of the -small argument altogether as part of our upcoming infrastructure improvements, so not sure if it is worth our time to update it further.

miloknowles commented 1 year ago

Thanks, Milo. I'll take a look. I think that maybe we may get rid of the -small argument altogether as part of our upcoming infrastructure improvements, so not sure if it is worth our time to update it further.

I think having a --small option is still useful for testing and debugging. Currently, I'm using it as a faster way to test out a CI/CD pipeline for OGE.