Some generators are missing from the EIA-923 primary fuel table

[miloknowles]

Context: Currently, the clean_eia923 function returns (1) gen_fuel_allocated and (2) primary_fuel_table. For each plant/generator/month row in gen_fuel_allocated, I've been using primary_fuel_table to look up the energy source code for that plant and generator.

Problem: For a small number of generators in gen_fuel_allocated, there is no corresponding data in primary_fuel_table. Usually, that generator "appears" in the primary_fuel_table in a later year, which has allowed me to manually fix those generators.

Any idea why this might be @grgmiller ? Maybe this is a problem you've already found? I can either create a manual data file to assign fuel types to these generators, or we can dig deeper into why this is happening.

For reference, here is my manual workaround:

MANUAL_PLANT_GENERATOR_ENERGY_SOURCE_CODE = {
  '6058': {
    '2': 'NG'
  },
  '54224': {
    'GEN6': 'BIT'
  },
  '6190': {
    '3': 'PC'
  },
  '7790': {
    '2': 'BIT' # Default to the plant primary fuel type.
  },
  '10612': {
    'GEN2': 'NG'
  },
  '54690': {
    '6000': 'SUB'
  },
  '55821': {
    'BCT': 'NG',
    'BST': 'NG'
  },
  '645': {
    'GT4': 'NG'
  },
  '7652': {
    '1': 'BIT'
  },
  '54408': {
    '2': 'WDS'
  },
  '1904': {
    '6': 'NG'
  },
  '10562': {
    'GEN5': 'WDS',
  },
  '54851': {
    'SOL1': 'LFG',
    'SOL2': 'LFG',
    'SOL3': 'LFG',
  },
  # IC1/IC2 generators are both DFO, so assume the same for the rest.
  '676': {
    'IC3': 'DFO',
    'IC4': 'DFO',
    'IC5': 'DFO',
    'IC6': 'DFO',
    'IC7': 'DFO'
  }
}

[grgmiller]

Potential causes to check:

• Did you update your pudl.sqlite file by running load_data.download_updated_pudl_database(download=True)?
• Do these plants have an operating status of proposed in the years that you are looking at?
• Do the generators that are missing from primary_fuel_table have a non-missing energy_source_code assigned in gen_fuel_allocated

[grgmiller]

If this is something you want to dig into, feel free to use the branch primary_fuel_table

[grgmiller]

It seems like this issue is now fixed - performing an outer merge on gen_fuel_allocated and primary_fuel_table results in no records with a "left_only" indicator.