podaac / l2ss-py

Level 2 subsetter with Harmony integration
https://podaac.github.io/l2ss-py/
Apache License 2.0
11 stars 11 forks source link

null time values in TEMPO results when spatial+temporal subsetting #226

Closed danielfromearth closed 7 months ago

danielfromearth commented 7 months ago

Challenge

When testing the subsetting procedure against TEMPO data collections, it was discovered that requests that include both spatial and temporal subsetting simultaneously produce output files with all null values (specifically, NaT values) in the 'time' variable.

Desired functionality

The 'time' variable in output files when subsetting temporally+spatially should be trimmed to the requested time range and contain real values.

Benefit

With this change, the results will be more usable to those looking to inspect or analyze time-relevant patterns.

danielfromearth commented 7 months ago

I think it's the missing_dim: 1 in this line.  When creating the conditional array (i.e., the places where the spatial and temporal conditions are True) to subset the 1D 'time' variable, the 2D conditional array (cond) needs to be trimmed into a 1D conditional array that follows the 'mirror_step' dimension — let's say the mirror_step row.  So, the missing_dim is the 'xtrack' dimension.  To do the trimming, it seems like this line will simply select the 1st column along the 'xtrack' dimension.  If this happens when both spatial and temporal subsetting are being applied, the 1st column may be all False , even though there are True values along a different column.

My first thought for a fix: instead of simply selecting the 1st column, find the column that contains the most True conditions, and use that as the new 1D conditional array to apply to the 'time' variable.