convertsat with OMPROFOZ gives assert failure.

[barronh]

Using convertsat.sh with OMPROFOZ gives an assertion failure.

./convertsatomprofoz.sh
making regrid/HEMIS/OMI-Aura_L2-OMPROFOZ_2009m0101_v003.nc
**PNC:/work/ROMO/anaconda_envs/basic38/lib/python3.8/site-packages/PseudoNetCDF/pncwarn.py:24:UserWarning:
  IOAPI_ISPH is assumed to be 6370000.; consistent with WRF
Traceback (most recent call last):
  File "scripts/sat2cmaq.py", line 151, in <module>
    process()
  File "scripts/sat2cmaq.py", line 38, in process
    omf, omgf = openpaths(inpaths)
  File "scripts/sat2cmaq.py", line 34, in openpaths
    omgf = omgfs[0].stack(omgfs[1:], geotgtdim)
  File "/work/ROMO/anaconda_envs/basic38/lib/python3.8/site-packages/PseudoNetCDF/core/_files.py", line 1807, in stack
    assert(all([different.union([stackdim]) == set([stackdim])
AssertionError

The assertion is due to trying to concatenate along the wrong dimension. In netcdf v4.4.1, the phony_dim_4 (size 24) is used for both dimensions in O3AveragingKernel (nTarget, nXtrack, nLayer, nLayer). In netcdf v4.7.1, the phony_dim_4 is used for the first nLayer and phony_dim_5 is used for the second. As a results, all dimensions are incremented upward after that.

The OMPROFOZ configuration concatenates files along the phony_dim_8, which works in older netcdf versions. In newer versions, the correct dimension for concatenation is phony_dim_9.

[barronh]

cmaqsatproc requires that dimension names to be concatenated be specified, but the could also be diagnosed. The ideal solution would be to interpret the structured text-based meta-data and use named dimensions. However, that functionality is in a development branch of pseudonetcdf and is not ready.

For now, I have changed the scripts to have three approaches to selecting the concatenation dimensions.

1. specify the dimension by name (buggy with OMPROFOZ)
   • pros: using names matches results of netcdf ncdump, so it is easy to debug.
   • cons: subject to change with netcdf version
2. autoselect the dimension name if supplied value is None.
   • pros: simple
   • cons: doesn't work with OMNO2v003
3. specify the dimension by order within the group (data or geolocation).
   • pros: This works regardless of netcdf version with both OMNO2 and OMPROFOZ
   • cons: somewhat opaque

All three options are nice.

For OMPROFOZ, the configuration can be changed:

-datatgtdim = 'phony_dim_0'
-geotgtdim = 'phony_dim_8'
+datatgtdim = None # 'phony_dim_0'
+geotgtdim = None # 'phony_dim_9'

or

-datatgtdim = 'phony_dim_0'
-geotgtdim = 'phony_dim_8'
+datatgtdim = 0 # 'phony_dim_0'
+geotgtdim = 0 # 'phony_dim_9'

For OMNO2, the original configuration approach is version independent. That being said, it can be changed to either of the following.

-datatgtdim = 'phony_dim_0'
-geotgtdim = 'phony_dim_7'
+datatgtdim = None # 'phony_dim_0'
+geotgtdim = 2 # 'phony_dim_7'

or

-datatgtdim = 'phony_dim_0'
-geotgtdim = 'phony_dim_7'
+datatgtdim = 0 # 'phony_dim_0'
+geotgtdim = 2 # 'phony_dim_7'

Note that in OMNO2v003, the geotgtdim must be specified. The first dimension in the geolocation group is not the target dimension.