search

catalyst-cooperative / pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
https://catalyst.coop/pudl
MIT License
456 stars 105 forks source link

Serialize import of FERC1 data into PUDL DB #67

Closed zaneselvans closed 7 years ago

zaneselvans commented 7 years ago

With more than a decade's worth of data potentially available from the FERC DB, we will probably need to start working with it in smaller chunks as we migrate it into the PUDL DB (rather than trying to pull an entire table over all at once -- at least for the larger tables). Need to research how to do this effectively.

zaneselvans commented 7 years ago

It turns out that the amount of data in the FERC1 DB is small enough that this isn't really a problem.