singularity-energy / open-grid-emissions

Tools for producing high-quality hourly generation and emissions data for U.S. electric grids
MIT License
72 stars 5 forks source link

Update the flag threshold in `Check_allocated_gf_matches_input_gf` #278

Closed grgmiller closed 1 year ago

grgmiller commented 1 year ago

validation.check_allocated_gf_matches_input_gf() checks that the allocated generation and fuel from EIA-923 matches the input totals when running data_cleaning.clean_eia923().

Changes in this PR:

Note: this validation check should be temporary until https://github.com/catalyst-cooperative/pudl/pull/2297 and https://github.com/catalyst-cooperative/pudl/pull/2235 are merged into PUDL.

This is an example of what the output of this warning looks like for 2021:

WARNING: Allocated EIA-923 data doesn't match input data for the following plants:
Percentage Difference:
              net_generation_mwh  fuel_consumed_mmbtu  fuel_consumed_for_electricity_mmbtu
plant_id_eia                                                                              
1206                   -0.001076            -0.002514                            -0.002514
1316                    0.023964             0.000000                             0.000000
54967                  -1.000000            -1.000000                            -1.000000
58256                   0.411966             0.000000                             0.000000
59260                  -0.475486            -0.475496                            -0.475496
59825                  -1.000000            -1.000000                            -1.000000
63098                  -1.000000            -1.000000                            -1.000000
EIA-923 Input Totals:
              net_generation_mwh  fuel_consumed_mmbtu  fuel_consumed_for_electricity_mmbtu
plant_id_eia                                                                              
1206                    56313.00             493763.0                             493763.0
1316                     2003.00              32704.0                              32704.0
54967                       1.96                 24.0                                 24.0
58256                     585.00               7304.0                               7304.0
59260                    1760.00              28873.0                              28873.0
59825                    9539.00              84354.0                              84354.0
63098                       1.00                  6.0                                  6.0
Allocated Totals:
              net_generation_mwh  fuel_consumed_mmbtu  fuel_consumed_for_electricity_mmbtu
plant_id_eia                                                                              
1206                56252.431817        492521.524247                        492521.524247
1316                 2051.000000         32704.000000                         32704.000000
54967                   0.000000             0.000000                             0.000000
58256                 826.000000          7304.000000                          7304.000000
59260                 923.144000         15144.000000                         15144.000000
59825                   0.000000             0.000000                             0.000000
63098                   0.000000             0.000000                             0.000000