search

NasjonaltBakkeSegment / safe_to_netcdf

Python scripts for converting specific Copernicus Sentinel products in Standard Archive Format for Europe (SAFE) to NetCDF/CF
GNU General Public License v3.0
6 stars 5 forks source link

RunTime error #22

Closed ElodieFZ closed 1 year ago

ElodieFZ commented 1 year ago

Processing failed for product S2A_MSIL1C_20220808T140751_N0400_R053_T31XDK_20220808T174921

GridEngine error:

Traceback (most recent call last):
  File "/home/nbs/file_conversion/nbs_tools/script/wrapper", line 78, in <module>
    wrapper(product, product_id, cfg)
  File "/home/nbs/file_conversion/nbs_tools/tools/wrapper.py", line 206, in wrapper
    conversion_OK = conversion_object.write_to_NetCDF(tmp_dir, compression_level=cfg['nc']['compression_level'])
  File "/home/nbs/file_conversion/safe_to_netcdf/s2_reader_and_NetCDF_converter.py", line 199, in write_to_NetCDF
    subdataset = gdal.Open(k)
  File "/modules/centos7/conda/Feb2021/envs/production/lib/python3.8/site-packages/osgeo/gdal.py", line 4192, in Open
    return _gdal.Open(*args)
RuntimeError: No granule found for EPSG code 32631

No error in python logging:

2022-08-08 19:42:21,633 - safe_to_netcdf.utils - DEBUG - Main file: /lustre/storeB/project/NBS2/sentinel/production/NorwAREA/netCDFNBS_work/production/products_being_processed/S2A_MSIL1C_20220808T140751_N0400_R053_T31XDK_20220808T174921/S2A_MSIL1C_20220808T140751_N0400_R053_T31XDK_20220808T174921.SAFE/manifest.safe
2022-08-08 19:42:21,641 - safe_to_netcdf.utils - DEBUG - <osgeo.gdal.Dataset; proxy of <Swig Object of type 'GDALDatasetShadow *' at 0x2b80408f7b40> >
2022-08-08 19:42:21,642 - safe_to_netcdf.s2_reader_and_NetCDF_converter - INFO - Read view and sun angles
2022-08-08 19:42:21,705 - safe_to_netcdf.s2_reader_and_NetCDF_converter - INFO - ------------START CONVERSION FROM SAFE TO NETCDF-------------
2022-08-08 19:42:21,705 - safe_to_netcdf.s2_reader_and_NetCDF_converter - INFO - Creating NetCDF file
2022-08-08 19:42:21,705 - safe_to_netcdf.utils - DEBUG - Memory usage so far: 0.156296 Gb
2022-08-08 19:42:21,705 - safe_to_netcdf.utils - DEBUG - 0:00:00.073470
2022-08-08 20:10:57,147 - safe_to_netcdf.s2_reader_and_NetCDF_converter - INFO - Adding projection coordinates
2022-08-08 20:10:57,148 - safe_to_netcdf.utils - DEBUG - Memory usage so far: 4.399964 Gb
2022-08-08 20:10:57,148 - safe_to_netcdf.utils - DEBUG - 0:28:35.516381
2022-08-08 20:10:57,446 - safe_to_netcdf.s2_reader_and_NetCDF_converter - INFO - Adding frequency bands layers
2022-08-08 20:10:57,446 - safe_to_netcdf.utils - DEBUG - Memory usage so far: 4.399964 Gb
2022-08-08 20:10:57,446 - safe_to_netcdf.utils - DEBUG - 0:28:35.814766
ElodieFZ commented 1 year ago

After closer inspection, there is nothing wrong with the product. A complete rerun, including downloading from colhub, worked fine. The odd thing is that the md5sum of the SAFE archive was fine and no error appeared in the logs either when downloading or archiving. Maybe some I/0 error on lustre? A similar issue happened at the same time (#21)