Closed mdtanker closed 1 year ago
It looks like the rounding error is occurring somewhere in the call to hm.filters._fft.ifft()
, which is the function that appears to call xrft
, so it might be an issue with xrft
.
I think a simple fix is to add the following lines to the end of the apply_filter()
function to copy the original coordinate values onto the new filtered grid:
def apply_filter(grid, fft_filter, **kwargs):
...
rest of function
...
# use original coordinates on the filtered grid (rounding errors likely occurred)
filtered_grid = filtered_grid.assign_coords(
{
dims[1]:grid[dims[1]].values,
dims[0]:grid[dims[0]].values
})
# Check that coordinates exatctly match the original coordinates
np.testing.assert_equal(filtered_grid.easting.values, grid.easting.values)
np.testing.assert_equal(filtered_grid.northing.values, grid.northing.values)
return filtered_grid
Happy to open this in a PR if people think it's the best fix for now!
Hey @mdtanker i came across the same issue when using these. The problem is that the inverse transform converts the frequencies back to spatial coordinates and there is inevitably some rounds off.
Since we do the round trip in a single function, the best solution would be to assign the original coordinates to the transformed grid instead of trying to approximate.
What do you think?
And now I actually read the full code you provided doing exactly that 🤦♂️
If that's the case, then we don't need the checks since it's impossible to fail.
Yup, that comes from the xrft
side, where the function _ifreq controls the coordinates for ifft.
def _ifreq(N, delta_x, real, shift):
# calculate frequencies from coordinates
# coordinates are always loaded eagerly, so we use numpy
if real is None:
fftfreq = [np.fft.fftfreq] * len(N)
else:
irfftfreq = lambda Nx, dx: np.fft.fftfreq(
2 * (Nx - 1), dx
) # Not in standard numpy !
fftfreq = [np.fft.fftfreq] * (len(N) - 1)
fftfreq.append(irfftfreq)
k = [fftfreq(Nx, dx) for (fftfreq, Nx, dx) in zip(fftfreq, N, delta_x)]
if shift:
k = [np.fft.fftshift(l) for l in k]
return
The delta_x
controls the interval and can cause issues. It is calculated from np.diff(coord)
.
Should we fix this on our side? Or, alternatively, should we force the coordinates to be similar on the xrft
side?
It would be hard to do on the xrft side without storing the original coordinates in the Fourier transform somehow.
We could start with a fix on our side as @mdtanker suggested and then report this as an issue to them and see what they want to do.
It would be hard to do on the xrft side without storing the original coordinates in the Fourier transform somehow.
We could start with a fix on our side as @mdtanker suggested and then report this as an issue to them and see what they want to do.
Yup, I agree! It's more easy to fix on our side. 👍
Thanks for reporting this! I agree that we should fix it on our side, but I also think this might be relevant to xrft as well.
cc @roxyboy @lanougue @rabernat
It would be hard to do on the xrft side without storing the original coordinates in the Fourier transform somehow.
I'm not sure how this would work without it being stored as an attribute to the xarray dataset..?
Ok sounds good, I'll open a PR soon with the fix I outlined above! Thanks for the feedback.
Description of the problem:
When using the new grid transformations, I noticed that several (at least
derivative_upward
andgaussian_lowpass
) functions are slightly changing the easting coordinates of the dataarrays. I was attempting to add the resulting grids as a new dataset variable with:Plotting the new grid shows:
Compared to the normal method of creating a new array shows an issue with the high and low easting values.
Numpy testing confirms that the easting values of the produced grid don't match the original past the 10th decimal place.
The northing coordinates are exactly equal for this dataset, but northing and easting both fail the test on a dataset of min.
I have a workaround, which is to round the coordinate values to the same number of decimal places before merging, which gives the desired results.
I need to add the resulting data back to the original dataset since my grid has NaN's, which I've filled with -999 and need to mask the results with the original data extent.
Any ideas what's causing this issue?
Otherwise, these new functions are great!
System information
conda list
below:Output of conda list