The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
Some of our output tables are mostly just grab x table and denormalize it (merge entity x table into it), but sometimes we do imputations or filling in of data.
There are also two main ways we access the output tables, through the cached table stored within pudl_out._dfs or by re-regenerating the output table directly through the function in pudl.output. The plants_eia860 is being accessed in a few different places via the second method and for a while has had some imputations happening via pudl.output.eia860.fill_in_missing_ba_codes. Because of this, this imputation is being run over and over again.
I've noticed this mostly from re-running the net-gen/fuel allocation process in a notebook over and over and over again (a new pudl_out object will re-run this missing ba codes process 7 times for the mcoe output when the net generation/fuel is allocated).
e-belfer
commented
11 months ago
Some of our output tables are mostly just grab x table and denormalize it (merge entity x table into it), but sometimes we do imputations or filling in of data.
There are also two main ways we access the output tables, through the cached table stored within
pudl_out._dfsor by re-regenerating the output table directly through the function inpudl.output. Theplants_eia860is being accessed in a few different places via the second method and for a while has had some imputations happening viapudl.output.eia860.fill_in_missing_ba_codes. Because of this, this imputation is being run over and over again.I've noticed this mostly from re-running the net-gen/fuel allocation process in a notebook over and over and over again (a new
pudl_outobject will re-run this missing ba codes process 7 times for the mcoe output when the net generation/fuel is allocated).