rouille
commented
3 months ago Thanks, Ben! Looks like there's a lot of ESCs that are getting added or removed, but most of these have zero gen and fuel consumption. I think at this point we only care about whether there is a fuel missing in one dataset that has a non-zero allocation in another dataset, so for example in the first output dataframe, we'd want to print 58256 MWH and 59825 WND but not any of the other ones that are all zeros. Are all of the non-zero mismatches from your first two tables reflected in the bottom three? It appears that at least 58256 and 59285 are. If so, maybe we just keep the original three bottom dataframe outputs and not print the top two if they repeat the same information?
Yes, all non-zero mismatches in the first two tables are listed in the bottom 3 tables. More specifically, there are only 3 non-zero (either generation or fuel consumed or both) entries in the first table (the two you quoted + 59825 that was truncated), all entries in the second table are zero. The other 4 entries in the last 3 tables have the same (plant_eia_id, energy_source_code) but different value for generation, fuel consumed or both).
I will not print the first two tables but if you agree I will quote the number of non-zero entries vs zero entries for both tables.
grgmiller
Purpose
Look for difference in the input and allocated EI-923 data at the plant/fuel level. This was previously done at the plant level only. Closes CAR-3506
What the code is doing
Group the input and allocated data frames by
plant_id_eiaandenergy_source_code. Derive their symmetric difference in plant/fuel combination. It appears that among the 469 plant/fuel combinations that are in one of the data frames but not in both only 3 entries located in the input EIA-923 data frame have non-zero net generation and/or fuel consumed.Testing
Manually tested
Where to look
All the code is in the
check_allocated_gf_matches_input_gffunction of theoge.validationmoduleUsage Example/Visuals
Outputs (data frames have been truncated by hand for the PR):
Review estimate
10min
Future work
N/A
Checklist
black