USEPA / flowsa

Library that attributes resource use, waste, emissions, and loss to economic sectors
MIT License
23 stars 19 forks source link

Updates to generate 2014 CAP_HAP_Nonroad without memory errors #393

Closed bl-young closed 10 months ago

bl-young commented 10 months ago

splits activity sets into more groups to reduce memory errors

bl-young commented 10 months ago

also uses a modified FBA, aggregated to state level see script https://github.com/USEPA/FLOWSA_design/commit/f8426a16b679ff1a94deb8ed7a068ae339511170

bl-young commented 10 months ago

@WesIngwersen @catherinebirney see this minor update in flowsa and the associated script linked above in flowsa_design which creates a state level FBA for NEI_Nonroad before being used for the CAP_HAP_Nonroad FBS (2014 only). This gives us a working FBS for 2014 (state and national) which will be posted on data commons.

bl-young commented 10 months ago

@catherinebirney, I don't think there is a way to reference multiple activity sets, e.g.,:

      offroad_vehicles_equipment:
        selection_fields:
          PrimaryActivity: 
               - !from_index:NEI_Nonroad_asets.csv offroad_vehicles_equipment_a
               - !from_index:NEI_Nonroad_asets.csv offroad_vehicles_equipment_b

etc etc. So instead I duplicated the rows in the activity set file so I could use them separately for the 2014 FBS, but all together for 2017 and 2020.

bl-young commented 10 months ago

With multiple activity sets and converting to state this can work without the modification to the FBA in the script above: Successfully ran with https://github.com/USEPA/flowsa/actions/runs/6883135284

One minor downside is this FBS will end up with multiple metasources that are split. But this is not a major issue

bl-young commented 10 months ago

One minor downside is this FBS will end up with multiple metasources that are split. But this is not a major issue

Fixed with the final commit.