Issue with ingestion of some OMCLDO2 files: latitude_bounds, longitude_bounds with NaN

[StevenCompernolle]

Some OMCLDO2 files get after HARP ingestion latitude_bounds, longitude_bounds with partial NaN filling. (Note that for OMCLDO2, HARP creates itself latitude_bounds, longitude_bounds).

Example problematic orbits: OMI-Aura_L2-OMCLDO2_2018m0517t0004-o73595_v003-2018m0517t051848.he5 OMI-Aura_L2-OMCLDO2_2018m0517t0143-o73596_v003-2018m0517t071201.he5

Note that in QA4ECV OMI NO2, these orbits are simply not there, indicating that there is something peculiar with these data files. This is especially problematic when applying bin_spatial. The problematic pixels get a very high weight and therefore dominate in the spatial binning.

Perhaps a solution is that HARP throws an error when it creates latitude_bounds, longitude_bounds with NaNs in it upon ingestion?

[svniemeijer]

This type of filtering is something you can do as user. We intentionally do not try to force-filter data during an ingestion (since there are cases where you actually want to inspect the wrong data).

In your case, you should be able to simply add a valid(latitude) as filter operation to get rid of all the pixels with invalid geo-metadata. This seemed to have worked for the two files you mention.