search

stcorp / harp

Data harmonization toolset for scientific earth observation data
http://stcorp.github.io/harp/doc/html/index.html
BSD 3-Clause "New" or "Revised" License
55 stars 18 forks source link

squashed month variable disappears when binning, while squashed year or orbit_index does not. #261

Closed StevenCompernolle closed 2 years ago

StevenCompernolle commented 2 years ago

I encountered the following puzzling behaviour. When deriving a month variable, then squashing to a scalar, and then binning, the month variable disappears. While this does not happen for the year variable or orbit_index variable. I tested this on a S5P PAL NO2 overpass file. It happens with bin and with bin_spatial

Operations: harpconvert -a 'derive(year {time});derive(month {time});squash(time, (year, month));bin();' 2018/05/02/S5P_PAL__L2VONO2____20180502T012325_20180502T030455_02847_01_020301_20211108T170806_belgiumbox.nc out.nc

I can of course re-derive the month variable after the binning. But it looks as unwanted behaviour.

StevenCompernolle commented 2 years ago

Perhaps it has something to do with the data type? On hdfview, I see that orbit_index is int, year is short, and month is byte

svniemeijer commented 2 years ago

We found the issue. HARP was removing any enumeration variable before checking if it was scalar or not. This is now turned around, such that scalar enumeration variables are now kept. This is fixed in: 4a155d46bf3939a247df60fe1cc91a53c3d58643