catalyst-cooperative / pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
https://catalyst.coop/pudl
MIT License
458 stars 105 forks source link

Archive allocated FERC-714 hourly demand and historical EIA-861 service territories #1136

Closed zaneselvans closed 2 years ago

zaneselvans commented 2 years ago

We still haven't archived the outputs of the hourly state level demand allocation project, and the historical balancing authority and utility service territories. Might as well use the fresh v0.4.0 software to generate those outputs and throw them up on Zenodo so they can be cited and accessed by anyone, without needing to run the scripts.

zaneselvans commented 2 years ago

Sadly, running the pudl_territories script with or without dissolving the territories is blowing up memory usage, consuming all of my RAM and even all of my swap, which are together almost 100GB, so something is terribly wrong. So once again we learn that if it's not tested automatically, it is broken. See #1174

zaneselvans commented 2 years ago

I output a version of the territories that just includes the FIPS codes and included that, rather than the geospatial data, so others can join to spatial data. The archive has been published on Zenodo.