The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
Functions that produce valuable interim tables will be converted software defined assets. These steps include:
EIA extraction: pudl.extract.eia923.Extractor(ds).extract(), pudl.extract.eia860.Extractor(ds).extract(), pudl.extract.eia861.Extractor(ds).extract() methods will be called in a single configurable multi_asset. This way we can ensure downstream assets are working with the same subset of years.
Table transformations: table clean functions in pudl.transform.eia[form_number] will be come assets that can depend on multiple assets and return a single asset with the exception of the generation_fuel_eia923 which creates the generation_fuel_eia923 and generation_fuel_nuclear_eia923 tables.
Harvesting: Converting the harvesting function is more complicated because it depends on all clean EIA data and returns harvested versions of the input tables. We will likely be harvesting more tables in the future so this function needs to accommodate for an arbitrary number of inputs and outputs. Luckily, we can programmatically specify the inputs by loading upstream assets:
eia_assets = load_assets_from_modules([eia860, eia923])
@multi_asset(
ins={
asset_key.to_python_identifier(): AssetIn()
for eia_asset in eia_assets
for asset_key in eia_asset.asset_keys
}
)
def eia_transform(**eia_transformed_dfs):
...
Functions that produce valuable interim tables will be converted software defined assets. These steps include:
pudl.extract.eia923.Extractor(ds).extract()
,pudl.extract.eia860.Extractor(ds).extract()
,pudl.extract.eia861.Extractor(ds).extract()
methods will be called in a single configurable multi_asset. This way we can ensure downstream assets are working with the same subset of years.pudl.transform.eia[form_number]
will be come assets that can depend on multiple assets and return a single asset with the exception of the generation_fuel_eia923 which creates thegeneration_fuel_eia923
andgeneration_fuel_nuclear_eia923
tables.