metno / NWPdocs

Description of the numerical weather prediction products at MET Norway
GNU General Public License v3.0
14 stars 4 forks source link

Spatial grid values (x, y) have changed since `meps_det_2_5km_20230130T09Z.nc` #20

Closed carlos-rpg closed 1 year ago

carlos-rpg commented 1 year ago

Hello metno,

Since 2023-01-30 09:00, the MEPS files have a slightly different spatial grid values. They no longer are exactly spaced in increments of 2500 m.

import xarray as xr
import pandas as pd

T06 = xr.open_dataset(
    'https://thredds.met.no/thredds/dodsC/meps25epsarchive/2023/01/30/meps_det_2_5km_20230130T06Z.nc'
)
T09 = xr.open_dataset(
    'https://thredds.met.no/thredds/dodsC/meps25epsarchive/2023/01/30/meps_det_2_5km_20230130T09Z.nc'
)
diff_stats = pd.concat(
    objs=[
        T06['x'].diff(dim='x').to_series().rename('T06_x').describe(),
        T06['y'].diff(dim='y').to_series().rename('T06_y').describe(),
        T09['x'].diff(dim='x').to_series().rename('T09_x').describe(),
        T09['y'].diff(dim='y').to_series().rename('T09_y').describe(),
    ],
    axis='columns'
)
print(diff_stats)

Is this intended or an error? Thanks in advance

eivindsMET commented 1 year ago

Hello, We have recently moved to new infrastructure, and thus the handling of roundoff errors have changed slightly. As you will see from the output of your code snippet, the variance is in order of cm which of course is completely insignificant in a geophysical sense, but might be annoying if you hard-code x,y values in your scripts. This is anyway a projected plane imperfectly representing an elliptical earth, so this difference cannot be regarded as an error.

Eivind

carlos-rpg commented 1 year ago

I see, thanks for the quick reply.

In my particular case I concatenate files in the time dimension to build a weather history. The change has made the process more difficult since now it's necessary to make the grid the same for past and future files, or face much longer concatenation times and higher memory footprint.

eivindsMET commented 1 year ago

Oh, I see. It is unfortunate this gives you complexions, hope you find a workable solution.

All the best, Eivind

carlos-rpg commented 1 year ago

Workaround to transform the new grid into the old one:

import numpy as np
import xarray as xr

new_grid_file= xr.open_dataset(
    'https://thredds.met.no/thredds/dodsC/meps25epsarchive/2023/01/30/meps_det_2_5km_20230130T09Z.nc'
)
old_grid_file = new_grid_file.assign_coords(
    x=np.round(new_grid_file.indexes['x'] + 84.0) - 84.0,
    y=np.round(new_grid_file.indexes['y'] + 17.875) - 17.875
)
eivindsMET commented 1 year ago

oh, nice! Do you mind if I use this fix in the FAQ for the documentation in case others have similar issue?

carlos-rpg commented 1 year ago

No problem! Glad to help.