ozone units and air pressure in OMI and OMPS O3 total columns converters

[jeromebarre]

I guess I reviewed the converters a bit too fast.

Few things:

One thing is air_pressure that is set to 0 for all nlocs. Both obs types are total columns hence providing air pressure is useless. I think this can be removed from the ioda files.

The other issue is the unit. I am trying to have a unified unit that goes in ufo to avoid having unit conversion factors that go to yaml here and there... Basically we'd like to have mol/m2 as a general unit for columns quantities, especially if we stop using AtmVertInterpLay and use the more general operator that I plan on working on soon. See our convention table (https://jointcenterforsatellitedataassimilation-jedi-docs.readthedocs-hosted.com/en/latest/inside/conventions/tbls.html). The last version need to be updated on the jedidocs page @gthompsnJCSDA is supposed to do it. DU still exist for now but this might not last.

Last thing is the variable name: is it possible to follow that https://github.com/JCSDA-internal/ioda-converters/blob/develop/src/compo/tropomi_no2_nc2ioda.py so in your case this would be 'ozone_total_column': 'ozone_in_total_column',

Thanks

[karpob]

I'm not at all sure what you want me to do here. As for getting rid of DU, well, that seems insane given that's what you get from the data stream. @kwargan

[kwargan]

Well, converting between DU and mol/m^2 is easy because 1 DU is a constant number of molecules per unit area. However, from the perspective of a scientist who will be looking at those (at least this scientist;)), mol/m^2 will lead to some crazy unreadable numbers, as opposed to the very familiar DUs. I'm not sure how much weight should be attached to this aspect.

[karpob]

If you give me a an alternative to atmVertInterpLay, I can get rid of air_pressure. Right now, I'm 99% sure it breaks without it even though it is useless.

[jeromebarre]

I meant mole/m2 not molecules/m2, sorry for the confusion. (1DU ~ 4.5E-5 mole/m2 so not so crazy). I like to joke and say that Dobson units are form an another century... More seriously only O3 total columns use DU. I think mole/m2 are more suitable for all trace gases and the generalized nadir operator I am working on will use that unit. We aim at JCSDA to set conventions and formats for all DA. Composition is not an exception to this. :)

[jeromebarre]

So things to do:

• change the unit to conform to the ioda unit conventions
• change the observable variable name
• get rid of air pressure whenever we can

[karpob]

I mean fundamentally, our customers will want Dobson Units whether or not the JCSDA likes it. If the instruments continue to report values in Dobson Units, these instrument teams are often our customers. I think it is a fatal flaw to not report Dobson Units. It's not quite like reporting W/sr/m^3 in place of brightness temperature for microwave, but the result will similar as our customers will be annoyed and probably look elsewhere.

[jeromebarre]

Straightforward unit conversion can be done in post processing at NASA for users if they want to, need to. I am sure post-processing will be done in operation anyways (e.g. providing forecast data in regular grid but also other obs-space diagnostics). So I wouldn't worry too much about this unit change that can be easily corrected in post. Plus not all satellite product are in DU, especially for other species, not even for total column. Please see: https://docs.sentinel-hub.com/api/latest/data/sentinel-5p-l2/ for example. S5P products for O3, CO, NO2, SO2, HCHO are in mol/m2. This unit choice doesn't come out of the blue.

Unit convention is important in order to keep everyone on the same page and make sure things don't deviate too much. i.e. each agency doing its own thing inside JEDI for the same scientific application. Yaml constants were being put everywhere and no one but the person that coded it understands it or remembers it's there. The purpose of conventions is to try create jointness, and have better code sanity and readability...

Hope this is clearer now. Thanks

[jeromebarre]

This has been I believe addressed in the convention sprint that took place last January. Closing this now.