validate emissions data is correct

[derekeder]

According to the Draft Inventory of U.S. Greenhouse Gas Emissions and Sinks: 1990-2020 by the EPA (discussed in https://github.com/chihacknight/state-by-state-decarbonization/issues/10#issuecomment-1043182534), the total CO2e emissions for 2018 were 6,689.5 MM CO2 Eq:

Our US total for 2018 in https://raw.githubusercontent.com/chihacknight/state-by-state-decarbonization/main/data/final/emissions.json is:

"dirty_power": 1729.0 "buildings": 607.4 "transportation": 1963.6 "dumps_farms_industrial": 1845.7 = 6,145.7 (units?)

@swatland610 you mentioned that we are not including everything from our data source. Does that account for this discrepancy of 543.8 MM CO2 Eq?

[swatland610]

@derekeder Worked on this a little bit this morning. I just did some quick gut checks across this data source we are using here

I noticed that in this data source itself, the Emissions by Sub Sector (which is what we are using for the charts) doesn't match the total across any of the states - for pretty much any of the years. Thankfully, these are not generally big differences.

So yes, I can adjust the util.py file to add the one missing column, but it won't fully line up to the actual total emissions.

If there is one silver lining, it's that the total_emissions column does match year over year to the graph you shared above. So there is some consistency between the 2 sources.

I can push a branch over with the changed code either quick tonight or tomorrow.

[derekeder]

@swatland610 interesting, but good to know they're close numbers. Do you have a sense of how far off they are? 1% 5% or more?

[swatland610]

@derekeder nevermind! I found the magic combo to align it.

Here's a breakdown of the columns we are using for our emissions data from this raw data.

To equal the total_emissions_excluding_lucf columns, we need to sum the following:

• Dirty Power
• 'emission_sub_electric'

Buildings

• 'emission_sub_commercial'
• 'emission_sub_residential'

Transportation

• 'emission_sub_transportation'

Dumps, Farms, Industrial, Other

• 'emission_by_waste'
• 'emission_by_agriculture'
• 'emission_sub_industrial'
• 'emission_by_industrial' -> this is industrial processes
• 'emission_sub_fugitive'

Just a quick note to call out that we have to explicitly exclude the emission_by_energy, emission_by_forestry, & emission_by_bunker_fuel columns.

This makes sense as:

1. the energy column overlaps with electricity
2. forest or LUCF is excluded in the total anyway

The only one I'm not sure why is bunker_fuels. Could overlap with some industrial processes.

Anyway, I'll adjust the utils.py file to reflect these changes and push them to a new branch.

[derekeder]

amazing

[derekeder]

closed via #15