dbekaert / RAiDER

Raytracing Atmospheric Delay Estimation for RADAR
Apache License 2.0
70 stars 39 forks source link

[BUG] checkContainment fails on global zenith HRES #684

Open bbuzz31 opened 2 months ago

bbuzz31 commented 2 months ago

Describe the bug when running on a global HRES file (e.g., bounding box: -90 90 -180 179.9296785) raider checkContainment fails on zenith. I suspect if fails on slants as well.

run_global.yaml.txt

Full error message

Invalid extension NetCDF for cube. Defaulting to .nc
Output cube spacing: 0.09 degrees
WARNING: Bounds extend past +/- 180. Results may be incorrect.
Output SNWE: [-90.0, 90.0, -180.18, 180.09]
Starting to run the weather model c
alculation
Requested date,time: 20200101, 00:00
Beginning weather model pre-processing
WARNING: Processed weather model already exists, please remove it ("weather_files/HRES_2020_01_01_T00_00_00_90S_90N_180W_180E.nc") if you want to download a new one.
WARNING: The processed weather model file already exists, so I will use that.
Extent of the weather model is (xmin, ymin, xmax, ymax):-179.52, -89.52, 179.51, 89.51
Extent of the input is (xmin, ymin, xmax, ymax): -180.18, -90.00, 180.09, 90.00
Weather model point bounds are -90.00/90.00/-180.18/180.09
Query datetime: 2020-01-01 00:00:00
ERROR: The weather model passed does not cover all of the input points; you may need to download a larger area.

I am able to change line 92 in processWM.py to make it run by adding HRES to the exception (in addition to HRRR). There are nans in the result that need to be checked though.

I also forced the interpolated cube datatype to float32 to conserve memory (line 200 in delay.py: np.zeros(..., dtype=np.float32))

Note that the giant processed weather model can be compressed as follows with xarray:


src = 'HRES_2020_01_01_T00_00_00_90S_90N_180W_180E.nc'
dst = 'HRES_2020_01_01_T00_00_00_90S_90N_180W_180E_compressed.nc'
ds = xr.open_dataset(src)
comp = dict(zlib=True, complevel=9)
encoding = {var: comp for var in ds.data_vars}
ds.to_netcdf(dst, encoding=encoding)
jlmaurer commented 2 months ago

@bbuzz31 is this the checkContainment function starting at line 775 or the checkContainment method that is part of the WeatherModel class?

jlmaurer commented 2 months ago

I think the general fix for this is to just check whether the model bbox contains the world bbox and, if so, set the bbox to be the world and return True. That should work in any case since the only thing checkContainment does is pass back a True or False. I don't know if that will impact anything further down the line (e.g. how will the DEM download be affected if you download a whole-earth DEM? Setting the weather model bounds to be the world bbox could work for that I think

bbuzz31 commented 2 months ago

the latter; the method of the WeatherModel class (sorry for confusion)

Also, I am concerned that because the final result has NaN values. I think these may be in a narrow regions of cell padded beyond the global extent. I think setting the bbox as you proposed should fix that but we'll need to check.