search

openghg / openghg_inversions

University of Bristol Atmospheric Chemistry Research Group RHIME Inversion code (with openghg dependency)
MIT License
5 stars 0 forks source link

Handle multiple inlet heights #171

Open joe-pitt opened 3 weeks ago

joe-pitt commented 3 weeks ago

When Matt's new agage-archive .nc files are read into an object store, the inlet height variable is used to split the data up into separate objects. While we have always had separate objects for separate inlets, in the past we read the inlet height from a json file, and so small changes tended to be ignored. A good example is the change from 10m to 9m at MHD - this was all labelled as 10m in the past and so it caused no problem to inversions. Now if you look in "/group/chem/acrg/object_stores/paris/obs_paris_2024_07_store_zarr" you will see multiple objects for MHD with different inlet heights - we want inversions to read in MHD data associated with any of these inlets, and combine with the 10m NAME fp.

My gut instinct is that the easiest fix is to change "inlet" to a dictionary in the .ini file (similar to "filters") so we can specify multiple inlets. Alternatively, a change could be made to openghg so that inlet height is a variable not an attribute, and data with all inlet heights are in a single object. I think that would have many knock-on consequences though, so maybe we better avoid that? What do people think?

mrghg commented 3 weeks ago

A dictionary sounds sensible

brendan-m-murphy commented 2 weeks ago

There will be a fix in OpenGHG once https://github.com/openghg/openghg/pull/1064 and https://github.com/openghg/openghg/pull/1066 are done.

If you use inlet = slice(1, 15) (or inlet = slice("1m", "15m")) it will search for any inlet between 1 and 15 meters (inclusive) and try to combine them into one dataset. The combination should work if the times in the different datasets don't overlap.

It might fail if the inlet goes to 9m then back to 10m then back to 9m. I'm waiting on PR 1064 to be able to test the final PR 1066 properly.