Demo: swap in BAAQMD emission inventory

[dholstius]

Steps to Close

• [ ] @bkoo-git to share gridded EI (NetCDF) file(s) with @dholstius
• [ ] @bujinb to share "shell" shapefile with @dholstius (shapefile containing the columns you expect but with dummy values, can be all zeros if you like)
• [ ] @dholstius to transfer gridded emissions to @bujinb's expected format, then share
• [ ] @bujinb to share new gridded-concentration results, preserving older ones

If I have any roles wrong (@yuzhou-wang?), my apologies, just LMK and I'll fix the above. Not sure who is running what code! 😄

[bkoo-git]

@dholstius Let's clarify what you want. The raw NetCDF emissions files have hourly emissions of many emission species; area source emissions are in a gridded format but point sources are not (they use a stack-based format). So, I think some sort of processing is desired. A few questions:

• Hourly emissions, daily, or annual total?
• All emission species or total primary PM2.5 only?
• Anthropogenic emissions only or everything including biogenics?

[bujinb]

@dholstius @bkoo-git , the NEI we use are annual emissions including biogenics. Does this emission comes with the SCCs (sources?)

[bkoo-git]

@dholstius @bkoo-git , the NEI we use are annual emissions including biogenics. Does this emission comes with the SCCs (sources?)

So, what you need is annual total emissions? The NetCDF files are model-ready emissions inputs which do not retain SCC info.

[dholstius]

@bkoo-git, thanks for asking. It'd be better if the emissions were indexed by SCC, but if they're annual total emissions (per 1 km cell), they can still help us figure out what's going on with the iF for total PrimaryPM25, right?

@bujinb, my expectation is that a pattern for what you need is represented by either ISRM/ca_nei.csv or Shapefiles/ca_nei.zip in /Data/UW/2022-03-07/. Is that correct? Same format, different values? I'm offering to help reshape whatever @bkoo-git can offer into your target format. I just need clarity on what that format is.

@bujinb, if you can point us to existing detailed documentation describing the desired format of the emission inputs, that would be great. If not, let's talk about working that into the project scope?

Of course, at some point we'll want emissions indexed by SCC. Prepping those may need to be a job for someone other than @bkoo-git. Maybe Steve?. I don't know; we can ask Phil.

[bujinb]

@dholstius

Yes! I need the yearly emissions (in any unit), and other parameters such as height of the emission etc (like in the example neighbored files). We do not have the desired format documentation, we can talk about it!

Thank you so much!

[dholstius]

I found some emission-formatting [documentation] at the InMAP GitHub repo. @ctessum, is it up-to-date?

It lists three options:

1. shapefile
2. SMOKE-formatted
   • (optionally) with a SMOKE-formatted spatial surrogate description
   • (optionally) with a surrogate description for use with OpenStreeMap
3. COARDS-formatted

I'm guessing @bkoo-git and/or @stephenreid65 will like SMOKE-formatted, and @bujinb will prefer shapefile-formatted. That's fine, we just need to agree. I like COARDS, but shapefile format might be the most future-proof for downstream analyses, so my runoff vote goes to shapefile. But let's hear from @bkoo-git and/or @stephenreid65 first?

[bkoo-git]

@dholstius @bujinb If source info (e.g., SCC, stack heights) is desired, SMOKE-formatted inventory files are more sultable than model-ready emissions input files. @stephenreid65 can provide the SMOKE files. Note that the stack heights alone can't describe actual plume heights, which include smoke plume rise determined (by either SMOKE or, in our case, CMAQ) using many factors (e.g., exit velocity, exit temperature, meteorological conditions).

[EDIT] I don't have preference about the three formatting options. I just meant that "gridded EI (NetCDF) file(s)" won't provide source info such as SCC and stack parameters.

[ctessum]

@dholstius that looks like the correct information. Both the SMOKE-formatted and shapefile formats can contain stack parameters for plume rise. Note that using SMOKE formatted emissions requires extensive additional configuration information described here and here.

[dholstius]

Can a conversion from SMOKE-formatted to some more generic geodata format (e.g. a collection of shapefile layers, a raster stack, etc.) be easily effected by some off-the-shelf software?

[ctessum]

The tricky parts are applying spatial surrogates to downscale from county-level to grid cells, and to apply any variable transformations to convert inventory species to InMAP species. InMAP comes with software to do that, but getting the configuration right can be quite involved.

It might be more straightforward to convert CMAQ-formatted emissions data to a format that InMAP can use.

[bkoo-git]

What stack parameters InMAP needs for elevated sources? I'll check if the CMAQ-ready in-line point source emissions file has all the info needed.

[dholstius]

@bkoo-git and @stephenreid65, you've previously shared some netCDF gridded emissions with me, in the context of the appliance rule amendments. From what I understand, after the model is trained, InMAP is essentially just multiplying those with the S-R matrix (and doing that for each of 3 vertical layers). Of course the values in that multiplication are deltas, not baseline emissions per se, but you get my point. In the rulemaking work I just subtract the Control netCDF array from the Baseline and then I have a Delta.

Would it be too hard to use netCDF as our interchange format for emission deltas? I'm happy to transfer it to whatever polygons are needed by others.

This seems like one of those project design questions where scope, roles, and technologies are highly dependent, so it might help to clarify who is responsible for what before nailing down the how. cc @pmartien

[dholstius]

@bujinb and @ctessum: @bkoo-git would like to use netCDF to exchange emission data, if that works for you. We can make sure it adheres to COARDS format, per the InMAP documentation.

How are elevated sources to be represented in netCDF format? It seems like we would need to communicate the equivalent of height, diam, temp, and velocity for specific sources somehow.

Or does the netCDF option only work for non-stack sources?

[stephenreid65]

@dholstius there are a couple different options for representing point sources in netCDF format. For CMAQ, SMOKE can produce a 3D emission file that has emissions allocated to vertical layers as well as horizontal grid cells. SMOKE uses each point source's stack parameters to do the vertical allocation, but the resulting 3D files are quite large. The other option (the one we use internally) is to provide an "inline" emission file to CMAQ that has hourly emissions for each release point (stack) in the inventory. A second "stack groups" file is provided to CMAQ that contains stack parameters for each release point in the emission file, and CMAQ uses this information to do an internal plume rise calculation and vertical distribution of emissions. Right now, we have inline emissions and stack group files for point sources in netCDF format, not 3D files. Note that no SCC info is retained.

[ctessum]

Here's some information about stack parameters needed by InMAP: https://inmap.run/docs/emissions/#:~:text=Files%20with%20elevated%20emissions%20need%20to%20have%20attribute%20columns%20labeled%20height%2C%20diam%2C%20temp%2C%20and%20velocity%20containing%20stack%20information%20in%20units%20of%20m%2C%20m%2C%20K%2C%20and%20m/s%2C%20respectively.%20(Shapefiles%20without%20these%20attribute%20columns%20will%20be%20assumed%20to%20contain%20ground%2Dlevel%20emissions%20only.)

InMAP isn't currently set up to take elevated emissions in NetCDF format, but I guess it may not be too difficult to convert the CMAQ inline point file to shapefile format.

[stephenreid65]

Thanks @ctessum - since InMAP can read SMOKE-formatted input files, it is probably easiest to provide ORL or FF10 formatted point source files. Those would have annual emissions and all the stack parameters InMAP requires. Same for other source types, too, I suppose. Seems like we'll have to provide some info on our modeling grid and the spatial surrogates we use. to allocate area and mobile source emissions, but that's doable. If we go that route, we can avoid dealing with any netCDF files.

[dholstius]

If we want to visualize the emissions later, like on a map, what's the easiest or best way to get SMOKE data into a vector or raster format?

@ctessum, can InMAP re-export emission data in gridded form? Without stack velocity or diameter. Just the release rate for each (pollutant, cell, layer).

[stephenreid65]

We usually map the gridded SMOKE outputs in netCDF format, but those gridded files do not contain SCCs or any other source-specific info. Another possible option is to have SMOKE produce a gridded emissions report that includes the SCC/EIC codes, and then do a join between that ascii file and a shapefile of the modeling grid.

[ctessum]

If we want to visualize the emissions later, like on a map, what's the easiest or best way to get SMOKE data into a vector or raster format?

It's not super easy but there is an example for point source data here: https://inmap.run/blog/2019/04/20/sr/

@ctessum, can InMAP re-export emission data in gridded form? Without stack velocity or diameter. Just the release rate for each (pollutant, cell, layer).

It can't currently do that, but it used to be able to at one point.

[dholstius]

Thanks for the blog link, @ctessum. In sr_util.py, there's some heavy lifting done by _inmap_exe. Is the source for that executable also part of the repo?

[ctessum]

Yes. The relevant entry point in the code is here: https://github.com/spatialmodel/inmap/blob/master/inmaputil/sr.go#L118