search

stcorp / harp

Data harmonization toolset for scientific earth observation data
http://stcorp.github.io/harp/doc/html/index.html
BSD 3-Clause "New" or "Revised" License
55 stars 19 forks source link

Set tolerance for squash operation? #267

Closed StevenCompernolle closed 2 years ago

StevenCompernolle commented 2 years ago

It has happened that when harpmerging two files, with squash(time, (latitude_bounds,longitude_bounds, latitude, longitude), which should have the same set of coordinates, an error occurred because there was some small difference (order 10^(-15) to 10^(-14) in the values. Perhaps because operations on both files were done using two different machines.

Perhaps it is a good idea to add a tolerance argument for the squash operation, such that values can be indicated as sufficiently close to each other? Or is there a way in HARP to round values?

svniemeijer commented 2 years ago

We have been playing around with that approach as well, but this didn't work well. A better approach that we ended up with was to perform an explicit rebin() of the data in both lat and lon with the final (accurate for your current system) lat/lon grid that you want before you merge the data. This should be a pretty fast operation and should also hardly change the grid values.