catalyst-cooperative / pudl

The Public Utility Data Liberation Project provides analysis-ready energy system data to climate advocates, researchers, policymakers, and journalists.
https://catalyst.coop/pudl
MIT License
471 stars 108 forks source link

Consolidate EIA fuel type codes in metadata #1269

Closed zaneselvans closed 2 years ago

zaneselvans commented 3 years ago

We've got veritable 🤡 🚗 of EIA fuel type codes categorized in the current pudl.constants module, with a different dictionary for each table in which they appear, so we can simplify them when we create fuel_type_code_pudl. These dictionaries need to be consolidated and turned into a static labeling table under pudl.metadata.dfs in conjunction with #1253

FUEL_TYPE_EIA923_GEN_FUEL_SIMPLE_MAP: Dict[str, Tuple[str, ...]] = {
    'coal': ('ant', 'bit', 'cbl', 'lig', 'pc', 'rc', 'sc', 'sub', 'wc'),
    'oil': ('dfo', 'rfo', 'wo', 'jf', 'ker'),
    'gas': ('bfg', 'lfg', 'ng', 'og', 'obg', 'pg', 'sgc', 'sgp'),
    'solar': ('sun', ),
    'wind': ('wnd', ),
    'hydro': ('wat', ),
    'nuclear': ('nuc', ),
    'waste': ('ab', 'blq', 'msb', 'msn', 'msw', 'obl', 'obs', 'slw', 'tdf', 'wdl', 'wds'),
    'other': ('geo', 'mwh', 'oth', 'pur', 'wh'),
}
"""Simplified grouping of fuel codes found in the generation_fuel_eia923 table."""

FUEL_TYPE_EIA923_BOILER_FUEL_SIMPLE_MAP: Dict[str, Tuple[str, ...]] = {
    'coal': ('ant', 'bit', 'lig', 'pc', 'rc', 'sc', 'sub', 'wc'),
    'oil': ('dfo', 'rfo', 'wo', 'jf', 'ker'),
    'gas': ('bfg', 'lfg', 'ng', 'og', 'obg', 'pg', 'sgc', 'sgp'),
    'waste': ('ab', 'blq', 'msb', 'msn', 'obl', 'obs', 'slw', 'tdf', 'wdl', 'wds'),
    'other': ('oth', 'pur', 'wh'),
}
"""Simplified grouping of fuel codes found in the boiler_fuel_eia923 table."""

AER_FUEL_TYPE_STRINGS: Dict[str, Tuple[str, ...]] = {
    'coal': ('col', 'woc', 'pc'),
    'gas': ('mlg', 'ng', 'oog'),
    'oil': ('dfo', 'rfo', 'woo'),
    'solar': ('sun', ),
    'wind': ('wnd', ),
    'hydro': ('hps', 'hyc'),
    'nuclear': ('nuc', ),
    'waste': ('www', ),
    'other': ('geo', 'orw', 'oth'),
}
"""
Consolidation of AER fuel types into energy_sources_eia' categories.

These classifications are not currently used, as the EIA fuel type and energy
source designations provide more detailed information.
"""

FUEL_TYPE_EIA860_SIMPLE_MAP: Dict[str, Tuple[str, ...]] = {
    'coal': ('ant', 'bit', 'cbl', 'lig', 'pc', 'rc', 'sc', 'sub', 'wc', 'coal', 'petroleum coke', 'col', 'woc'),
    'oil': ('dfo', 'jf', 'ker', 'rfo', 'wo', 'woo', 'petroleum'),
    'gas': ('bfg', 'lfg', 'mlg', 'ng', 'obg', 'og', 'pg', 'sgc', 'sgp', 'natural gas', 'other gas', 'oog', 'sg'),
    'solar': ('sun', 'solar'),
    'wind': ('wnd', 'wind', 'wt'),
    'hydro': ('wat', 'hyc', 'hps', 'hydro'),
    'nuclear': ('nuc', 'nuclear'),
    'waste': ('ab', 'blq', 'bm', 'msb', 'msn', 'obl', 'obs', 'slw', 'tdf', 'wdl', 'wds', 'biomass', 'msw', 'www'),
    'other': ('mwh', 'oth', 'pur', 'wh', 'geo', 'none', 'orw', 'other')
}
"""Simplified grouping of fuel codes found in the generators_eia860 table."""
zaneselvans commented 2 years ago

This is essentially done, and just waiting to merge.