USEPA / EPA_MOVES_Model

Estimating emissions for mobile sources
Other
82 stars 23 forks source link

How do I access the actual opmode distributions that were used in a run? #72

Closed Hanisek closed 3 months ago

Hanisek commented 6 months ago

If I run links with default driving cycles (only average speed specified - no user inputs for opMode or driving cycles) in a project-level analysis (running-exhaust only), what would be a good way to see the actual opmode distributions that are used for each link (which options do I need to check to save these outputs)? I understand these distributions were interpolated based on the embedded cycles, and I just want to learn more about how to properly use rates operating mode distribution generator vs. lookup operating mode distribution generator vs. project-domain operating mode distribution generator - mostly just opMode distributions (no polProcess)..

danielbizercox commented 6 months ago

Hello,

This is a complicated portion of the model because the operating mode distributions on each link vary by source type, regulatory class, and model year (due to differences in vehicle mass and road load coefficients, as stored in the SourceUseTypePhysics table of the default database). Unfortunately, the tables involved in these calculations do not have fields for all these variables, so the intermediate calculated operating modes are stored in a very unintuitive way. To keep things more manageable, I recommend looking at this with a very simple RunSpec: ideally one link, one source type, and one model year (i.e., provide an age distribution that has one model year with an ageFraction of 1.0, with all other model years set to 0.0), and just running energy consumption.

However, that said, you can have MOVES save the operating mode distributions used on each link by checking the "Save Data" box for the "Project-Domain Operating Mode Distribution Generator (running exhaust)" component on the Advanced Features panel. Lower on the same panel, check "Copy Saved Generator Data" and give a database name. And please be aware that there are some issues in the GUI when it comes to this feature, so after saving the RunSpec, double-check that your selections actually made it into the .mrs file. It should have an entry in the <savedata> tag for LinkOperatingModeDistributionGenerator, and the <generatordatabase> tag should look like this: <generatordatabase shouldsave="true" servername="" databasename="dbnamehere" description=""/>

After running, look in the "Saved Generator Data" database that you specified for the opmodedistribution table. This will contain the actual operating mode distributions that were used for each link.

In summary, it is possible to review the operating mode distributions that MOVES calculates at Project Scale, but it is complicated because each link actually has multiple operating mode distributions, one for each applicable source type / regulatory class / model year range.

Hanisek commented 6 months ago

Daniel -

Thanks for the thorough answer!

I think you answer really makes sense to me (especially the GUI issues that I was not aware of - I was wondering why the generator did not save anything). I just want to follow up more on three items:

  1. How does MOVES break the source types into regulatory classes when calculating the opModes? Is it just distributions of the samplevehiclepopulation table?
  2. How does MOVES account for the entries of user-specified opMode distributions? Does it assume all regulatory classes (of the same source type and link) follow the same distributions?
  3. This might be a more newbee question - is there a guide on how to prepare the customized opmode distributions in terms of polProcesses (e.g., do all opModes of the same polProcess sum to 100%, or all opmodes for the source type and link?)

I am asking as we are preparing an emission rate lookup table. What we did for MOVES2014 was storing the outputs of MOVES project-level results by source type (default fuel and IM settings per county), model year, calendar year, temperature, and humidity. Then we use the default MOVES driving cycles (extracted using the generators) to assemble these emission rates for the fleet average based on the average speed (or use our own cycles/opmode distributions). My question about MOVES4 is how do we account for the source type physics that are broken into regulatory classes during the process..

Thanks again!

danielbizercox commented 5 months ago

Glad I could help!

  1. MOVES applies the age distributions (in sourcetypeagedistribution) to the vehicle populations (in sourcetypeyear) to calculate vehicle populations by model year. Then it applies the fuel type & engTechID distributions in the user-supplied AVFT table to get vehicle populations by model year, fuel type, and engTechID. Finally, it applies the stmyFuelEngFraction column in samplevehiclepopulation to calculate the number of vehicles by all those dimensions + regClassID.
  2. Yes, it MOVES assumes all regulatory classes have the same operating mode distribution. This is why we generally recommend using the average speed or driving schedule input instead, because then MOVES will automatically account for regulatory class differences when calculating emissions.
  3. User input operating mode distributions must sum to 1.0 over sourceTypeID, hourDayID, linkID, and polProcessID. More information about this input can be found in our training materials (in general, see Module 7 for Project Scale; specifically see slides 522-523 and 560 for the operating mode distribution input).

In general, if you are building an emission rate lookup table that varies by county, source type, model year, calendar year, temperature, and humidity, you are probably better off using MOVES at County Scale and Rates Mode, because that is the specific intended use case for that mode.

If you want to get emission rates for specific driving cycles, you can also use Project Scale in Rates Mode, as well.

Hanisek commented 4 months ago

Daniel -

Sorry for the delay on following up on this. I managed to get the runs (locally) to generate the emission rates by fuel type, source type, model year, and by operating mode. The only thing is that I had issues with OpMode #0 for quite a few pollutants (not all of them). I cannot generate any emission rates for pollutant ID smaller than 90 for this OpMode..

4/1/24, 9:07 AM DEBUG: Link 0 is using input operating mode distribution. 4/1/24, 9:07 AM DEBUG: LinkOperatingModeDistributionGenerator.calculateOpModeFractions ms=1944 4/1/24, 9:07 AM DEBUG: LinkOperatingModeDistributionGenerator.populateRatesOpModeDistribution ms=17 4/1/24, 9:07 AM INFO: LOMDG setupTime=54 bundleTime=1961 4/1/24, 9:07 AM DEBUG: newSBD Generator called for process: Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Aluminum - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Aluminum - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Ammonium (NH4) - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Ammonium (NH4) - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Atmospheric CO2 - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Atmospheric CO2 - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: CMAQ5.0 Unspeciated (PMOTHR) - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process CMAQ5.0 Unspeciated (PMOTHR) - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: CO2 Equivalent - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process CO2 Equivalent - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Calcium - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Calcium - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Carbon Monoxide (CO) - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Chloride - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Chloride - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Composite - NonECPM - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Elemental Carbon - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: H2O (aerosol) - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process H2O (aerosol) - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Iron - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Iron - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Magnesium - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Magnesium - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Manganese Compounds - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Manganese Compounds - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Methane (CH4) - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Methane (CH4) - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Nitrate (NO3) - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Nitrate (NO3) - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Nitrous Oxide (N2O) - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Non-carbon Organic Matter (NCOM) - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Non-carbon Organic Matter (NCOM) - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Organic Carbon - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Organic Carbon - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Oxides of Nitrogen (NOx) - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Potassium - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Potassium - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Primary Exhaust PM10 - Total - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Primary Exhaust PM10 - Total - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Primary Exhaust PM2.5 - Total - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Primary Exhaust PM2.5 - Total - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Silicon - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Silicon - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Sodium - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Sodium - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Sulfate Particulate - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Sulfate Particulate - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Sulfur Dioxide (SO2) - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Titanium - Running Exhaust 4/1/24, 9:07 AM DEBUG: No SourceBinDistribution for pollutant-process Titanium - Running Exhaust because there is no SourceTypePolProcess information for it. 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Total Energy Consumption - Running Exhaust 4/1/24, 9:07 AM DEBUG: Processing pollutant-process: Total Gaseous Hydrocarbons - Running Exhaust

I am attaching the files that I used to run this. Is there anything that I did wrong (I basically wanted to iterate across all pollutants for running exhaust, tirewear and brakewear) .Thanks again for your help!

csv_database.zip

xml_mm401_opfuel_6037_2021_4_67d6_70d2.zip

danielbizercox commented 4 months ago

I can't recreate your input database from these files. The importer XML file references files that are not included in the CSV database zip.

The easiest way to share a database is to zip the actual MariaDB files (.MYI, .MYD, .FRM). Alternatively, you can create a SQL dump file. If you are running an installed version of MOVES, there will be a SQLDumpUtility.bat file in the root MOVES directory that you can use. Otherwise, the command is:

mysqldump --user=moves --password=moves --port=3306 --databases db_name > db_name.sql

Hanisek commented 4 months ago

I can't recreate your input database from these files. The importer XML file references files that are not included in the CSV database zip.

The easiest way to share a database is to zip the actual MariaDB files (.MYI, .MYD, .FRM). Alternatively, you can create a SQL dump file. If you are running an installed version of MOVES, there will be a SQLDumpUtility.bat file in the root MOVES directory that you can use. Otherwise, the command is:

mysqldump --user=moves --password=moves --port=3306 --databases db_name > db_name.sql

Thanks for the prompt reply! I created it using HeidiSQL - hope it works!

mm401_opfuel_6037_2021_4_67d6_70d2_in.zip

danielbizercox commented 4 months ago

Yes, that worked to allow me to recreate your input database.

It is difficult to say for certain what is going on here, because this is a very unusual run. However, the issue may be with using linkID 0, as the linkID is manipulated and offset during the run. Try using all positive linkIDs, and see if that fixes the issue.

Hanisek commented 3 months ago

Yes, that worked to allow me to recreate your input database.

It is difficult to say for certain what is going on here, because this is a very unusual run. However, the issue may be with using linkID 0, as the linkID is manipulated and offset during the run. Try using all positive linkIDs, and see if that fixes the issue.

This works! Cannot express my gratitude! Thanks!

Did we add a warning or instruction anywhere that we have to use positive link IDs?

danielbizercox commented 3 months ago

Good to hear!

The next major version of MOVES will include an error message if you import a link table with a non-positive linkID.