search

NasjonaltBakkeSegment / safe_to_netcdf

Python scripts for converting specific Copernicus Sentinel products in Standard Archive Format for Europe (SAFE) to NetCDF/CF
GNU General Public License v3.0
6 stars 5 forks source link

Production fails for some Sentinel1 products #5

Closed ElodieFZ closed 3 years ago

ElodieFZ commented 3 years ago

For some S1 files, the netcdf creation ends up with the following error:

Traceback (most recent call last): File "/home/nbs/file_conversion/wrapper_backfill_elf.py", line 446, in main() File "/home/nbs/file_conversion/wrapper_backfill_elf.py", line 229, in main conversion_OK = conversion_object.write_to_NetCDF(nc_outpath=destinaiton_complete, compression_level=7) File "/home/nbs/file_conversion/safeTOnetcdf/S1toNetCDF/s1_reader_and_NetCDF_converter.py", line 441, in write_to_NetCDF resampled_calibration = self.getCalLayer(pixels, lines, calibration_LUT) File "/home/nbs/file_conversion/safeTOnetcdf/S1toNetCDF/s1_reader_and_NetCDF_converter.py", line 807, in getCalLayer tck = interpolate.RectBivariateSpline(lines[0::nb_lines],pixels[0:nb_lines],calibration_table) File "/usr/lib/python2.7/dist-packages/scipy/interpolate/fitpack2.py", line 1146, in init raise TypeError('x must be strictly ascending') TypeError: x must be strictly ascending

See all the EW files in /lustre/storeB/project/NBS2/sentinel/production/NorwAREA/netCDFNBS_work/working_files.

The product S1A_EW_GRDH_1SDH_20201023T180210_20201023T180420_034927_0412AD_4F16 is used as a the test file. When trying to run SAFE2nc on just that file, the same error is found.

Can happen both to EW and IW products. See for example S1A_IW_GRDH_1SDV_20201022T042202_20201022T042246_034904_0411D4_4375

Unfortunately, running on CentOs (with Python3) does not magically solve the problem.

Loading gdal/3.2.0-devel Loading requirement: hdf5/1.10.5-gcc netcdf/4.7.3-gcc4.8+benchmark Traceback (most recent call last): File "/home/nbs/file_conversion_test/nbs_tools/script/wrapper.py", line 488, in main() File "/home/nbs/file_conversion_test/nbs_tools/script/wrapper.py", line 228, in main destinaiton_complete) + '/', compression_level=7) File "/home/nbs/file_conversion_test/S1toNetCDF/s1_reader_and_NetCDF_converter.py", line 480, in write_to_NetCDF resampled_calibration = self.getCalLayer(pixels, lines, calibration_LUT) File "/home/nbs/file_conversion_test/S1toNetCDF/s1_reader_and_NetCDF_converter.py", line 877, in getCalLayer calibration_table) File "/modules/centos7/python/3.7.9/lib/python3.7/site-packages/scipy/interpolate/fitpack2.py", line 1235, in init raise ValueError(msg) ValueError: Error on entry, no approximation returned. The following conditions must hold: xb<=x[i]<=xe, yb<=y[i]<=ye, w[i]>0, i=0..m-1 If iopt==-1, then xb<tx[kx+1]<tx[kx+2]<...<tx[nx-kx-2]<xe yb<ty[ky+1]<ty[ky+2]<...<ty[ny-ky-2]<ye