search

pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.06k stars 293 forks source link

Multiple errors when processing OLCI data. #944

Closed simonrp84 closed 5 years ago

simonrp84 commented 5 years ago

Describe the bug When working with OLCI granules a large number of (non-fatal) error messages are produced:

Traceback (most recent call last):
  File "/network/aopp/apres/users/proud/satpyconda/lib/python3.7/site-packages/xarray/backends/file_manager.py", line 243, in __del__
  File "/network/aopp/apres/users/proud/satpyconda/lib/python3.7/site-packages/xarray/backends/file_manager.py", line 221, in close
  File "/network/aopp/apres/users/proud/satpyconda/lib/python3.7/site-packages/h5netcdf/core.py", line 701, in close
  File "/network/aopp/apres/users/proud/satpyconda/lib/python3.7/site-packages/h5py/_hl/files.py", line 431, in close
  File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper
  File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper
  File "h5py/h5f.pyx", line 267, in h5py.h5f.get_obj_ids
  File "h5py/h5i.pyx", line 43, in h5py.h5i.wrap_identifier
ImportError: sys.meta_path is None, Python is likely shutting down

These do not seem to affect processing or the ability to save images, but are quite annoying on the screen - and could also cause users to believe that processing has failed.

To Reproduce

from satpy import Scene
import glob

dirs = glob.glob('/my_OLCI_directory/*S3*.SEN3')

scenes = []
flist = []
for idir in dirs:
    flist = flist + glob.glob(idir+'/*.nc')

composite = 'true_color'
scn = Scene(reader='olci_l1b', filenames=flist)
scn.load([composite])
scn.save_datasets()

Expected behavior Datasets saved with no error messages.

Actual results Datasets saved but the error messages described above are shown on screen.

Versions

# Name                 Version                   Build  Channel
satpy                  0.17.1             pyhca3e704_0        conda-forge
h5py                   2.10.0          nompi_py37h513d04c_100 conda-forge
hdf5                   1.10.5          nompi_h3c11f04_1103    conda-forge
netcdf4                1.5.1.2          py37h73a1b54_1        conda-forge
conda                  4.7.12                   py37_0        conda-forge
python                 3.7.3                h33d41f4_1        conda-forge
mraspaud commented 5 years ago

First idea: Could it be that the files aren't close properly in the reader ?