search

catalyst-cooperative / pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
https://catalyst.coop/pudl
MIT License
476 stars 110 forks source link

Investigate less zealous removal of NA values during f1_fuel import #39

Closed zaneselvans closed 1 year ago

zaneselvans commented 7 years ago

After the fuel and fuel_unit strings are cleaned up in the f1_fuel import, any record which contains any NA values is dropped, before the DataFrame is pulled into the PUDL DB. In some cases, this means we lose some data. E.g. some utilities only report their mmbtu/kWh numbers on a separate "Total" line. We should check and see whether there's a less destructive way we can deal with these leftover records.

swinter2011 commented 5 years ago

test comment sync 3

zaneselvans commented 2 years ago

@katie-lamb Ah, we've had this issue for a whiiiiiiile.

zaneselvans commented 1 year ago

I think this was finally addressed in the XBRL / FERC 1 refactor. #1574