search

pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.07k stars 296 forks source link

ConstructorError when calling available_readers() #1977

Open gerritholl opened 2 years ago

gerritholl commented 2 years ago

Describe the bug

When I call available_readers(), Satpy outputs several tracebacks where imports fail due to yaml.ConstructorError being raised.

To Reproduce

from satpy import available_readers
from satpy.utils import debug_on; debug_on()
available_readers()

Expected behavior

I expect no error messages.

Actual results Text output of actual results or error messages including full tracebacks if applicable.

No protocol specified
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/iasi_l2.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/cmsaf-claas2_l2_nc.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/modis_l2.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/geocat.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/agri_l1.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/amsub_l1c_aapp.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/abi_l1b_scmi.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-pps_nc.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/modis_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/goes-imager_hrit.yaml',)
/data/gholl/checkouts/satpy/satpy/readers/seviri_base.py:453: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  ('GsicsCalMode', np.bool),
/data/gholl/checkouts/satpy/satpy/readers/seviri_base.py:454: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here.
Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations
  ('GsicsCalValidity', np.bool),
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/mirs.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/electrol_hrit.yaml',)
[DEBUG: 2022-01-14 10:27:18 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/grib.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/omps_edr.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/slstr_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/olci_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/home/gholl/Arbeit/checkouts-perforce/dev_Accso_EBP/config/readers/world_generic_image.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-geo.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/acspo.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/avhrr_l1b_eps.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/mtsat2-imager_hrit.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/amsr2_l2_gaasp.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/hy2_scat_l2b_h5.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/jami_hrit.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/ahi_hrit.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/gpm_imerg.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/vaisala_gld360.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/ghrsst_l3c_sst.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers] Could not import reader config from: ['/data/gholl/checkouts/satpy/satpy/etc/readers/ghrsst_l3c_sst.yaml']
[DEBUG: 2022-01-14 10:27:19 : satpy.readers] Error loading YAML
Traceback (most recent call last):
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 551, in find_python_name
    __import__(module_name)
  File "/data/gholl/checkouts/satpy/satpy/readers/ghrsst_l3c_sst.py", line 26, in <module>
    from satpy.dataset import Dataset
ImportError: cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 385, in available_readers
    reader_info = read_reader_config(reader_configs)
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 305, in read_reader_config
    reader_config = load_yaml_reader_configs(*config_files, loader=loader)
  File "/data/gholl/checkouts/satpy/satpy/readers/yaml_reader.py", line 119, in load_yaml_configs
    config = recursive_dict_update(config, yaml.load(fd, Loader=loader))
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 102, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 570, in construct_python_name
    return self.find_python_name(suffix, node.start_mark)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 719, in find_python_name
    return super(UnsafeConstructor, self).find_python_name(name, mark, unsafe=True)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 553, in find_python_name
    raise ConstructorError("while constructing a Python object", mark,
yaml.constructor.ConstructorError: while constructing a Python object
cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
  in "/data/gholl/checkouts/satpy/satpy/etc/readers/ghrsst_l3c_sst.yaml", line 16, column 18
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/fci_l1c_nc.yaml',)                                                                                                                                                                                 [93/1820]
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/viirs_sdr.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/abi_l1b.yaml', '/home/gholl/Arbeit/checkouts-perforce/dev_Accso_EBP/config/readers/abi_l1b.yaml')
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/slstr_l2.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/hsaf_grib.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/tropomi_l2.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/avhrr_l1b_hrpt.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/ami_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seviri_l2_grib.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seviri_l1b_hrit.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/virr_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/ascat_l2_soilmoisture_bufr.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/olci_l2.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/fci_l2_nc.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-msg2013-hdf5.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/msi_safe.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/nucaps.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/smos_l2_wind.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/abi_l2_nc.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/mersi2_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/safe_sar_l2_ocn.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seviri_l1b_nc.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/clavrx.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/satpy_cf_nc.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/vii_l2_nc.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/avhrr_l1b_gaclac.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/viirs_edr_flood.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/viirs_compact.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/mviri_l1b_fiduceo_nc.yaml',)
[DEBUG: 2022-01-14 10:27:19 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/vii_l1b_nc.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seadas_l2.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/maia.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/abi_l1b.yaml', '/home/gholl/Arbeit/checkouts-perforce/dev_Accso_EBP/config/readers/abi_l1b.yaml')
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/avhrr_l1c_eum_gac_fdr_nc.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/avhrr_l1b_aapp.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/sar-c_safe.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/iasi_l2_so2_bufr.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seviri_l2_bufr.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/mhs_l1c_aapp.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/li_l2.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers] Could not import reader config from: ['/data/gholl/checkouts/satpy/satpy/etc/readers/li_l2.yaml']
[DEBUG: 2022-01-14 10:27:20 : satpy.readers] Error loading YAML
Traceback (most recent call last):
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 551, in find_python_name
    __import__(module_name)
  File "/data/gholl/checkouts/satpy/satpy/readers/li_l2.py", line 35, in <module>
    from satpy.dataset import Dataset
ImportError: cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 385, in available_readers
    reader_info = read_reader_config(reader_configs)
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 305, in read_reader_config
    reader_config = load_yaml_reader_configs(*config_files, loader=loader)
  File "/data/gholl/checkouts/satpy/satpy/readers/yaml_reader.py", line 119, in load_yaml_configs
    config = recursive_dict_update(config, yaml.load(fd, Loader=loader))
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 102, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 570, in construct_python_name
    return self.find_python_name(suffix, node.start_mark)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 719, in find_python_name
    return super(UnsafeConstructor, self).find_python_name(name, mark, unsafe=True)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 553, in find_python_name
    raise ConstructorError("while constructing a Python object", mark,
yaml.constructor.ConstructorError: while constructing a Python object
cannot find module 'satpy.readers.li_l2' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
  in "/data/gholl/checkouts/satpy/satpy/etc/readers/li_l2.yaml", line 47, column 18
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/ahi_l1b_gridded_bin.yaml',)                                                                                                                                                                        [12/1820]
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/viirs_edr_active_fires.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seviri_l1b_native.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/generic_image.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/viirs_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/seviri_l1b_icare.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/amsr2_l1b.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/scatsat1_l2b.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers] Could not import reader config from: ['/data/gholl/checkouts/satpy/satpy/etc/readers/scatsat1_l2b.yaml']
[DEBUG: 2022-01-14 10:27:20 : satpy.readers] Error loading YAML
Traceback (most recent call last):
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 551, in find_python_name
    __import__(module_name)
  File "/data/gholl/checkouts/satpy/satpy/readers/scatsat1_l2b.py", line 24, in <module>
    from satpy.dataset import Dataset
ImportError: cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 385, in available_readers
    reader_info = read_reader_config(reader_configs)
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 305, in read_reader_config
    reader_config = load_yaml_reader_configs(*config_files, loader=loader)
  File "/data/gholl/checkouts/satpy/satpy/readers/yaml_reader.py", line 119, in load_yaml_configs
    config = recursive_dict_update(config, yaml.load(fd, Loader=loader))
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 102, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 570, in construct_python_name
    return self.find_python_name(suffix, node.start_mark)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 719, in find_python_name
    return super(UnsafeConstructor, self).find_python_name(name, mark, unsafe=True)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 553, in find_python_name
    raise ConstructorError("while constructing a Python object", mark,
yaml.constructor.ConstructorError: while constructing a Python object
cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
  in "/data/gholl/checkouts/satpy/satpy/etc/readers/scatsat1_l2b.yaml", line 41, column 18
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/amsr2_l2.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/mimicTPW2_comp.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/caliop_l2_cloud.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers] Could not import reader config from: ['/data/gholl/checkouts/satpy/satpy/etc/readers/caliop_l2_cloud.yaml']
[DEBUG: 2022-01-14 10:27:20 : satpy.readers] Error loading YAML
Traceback (most recent call last):
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 551, in find_python_name
    __import__(module_name)
  File "/data/gholl/checkouts/satpy/satpy/readers/caliop_l2_cloud.py", line 28, in <module>
    from satpy.dataset import Dataset
ImportError: cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 385, in available_readers
    reader_info = read_reader_config(reader_configs)
  File "/data/gholl/checkouts/satpy/satpy/readers/__init__.py", line 305, in read_reader_config
    reader_config = load_yaml_reader_configs(*config_files, loader=loader)
  File "/data/gholl/checkouts/satpy/satpy/readers/yaml_reader.py", line 119, in load_yaml_configs
    config = recursive_dict_update(config, yaml.load(fd, Loader=loader))
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/__init__.py", line 81, in load
    return loader.get_single_data()
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 51, in get_single_data
    return self.construct_document(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 60, in construct_document
    for dummy in generator:
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 413, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 218, in construct_mapping
    return super().construct_mapping(node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 143, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 102, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 570, in construct_python_name
    return self.find_python_name(suffix, node.start_mark)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 719, in find_python_name
    return super(UnsafeConstructor, self).find_python_name(name, mark, unsafe=True)
  File "/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/yaml/constructor.py", line 553, in find_python_name
    raise ConstructorError("while constructing a Python object", mark,
yaml.constructor.ConstructorError: while constructing a Python object
cannot find module 'satpy.readers.caliop_l2_cloud' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
  in "/data/gholl/checkouts/satpy/satpy/etc/readers/caliop_l2_cloud.yaml", line 40, column 18
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/goes-imager_nc.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/glm_l2.yaml',)
[DEBUG: 2022-01-14 10:27:20 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/ahi_hsd.yaml',)

Environment Info:

>>> check_satpy()                                                                                                                                                                                                                                                                                                   [19/1854]
Readers
=======
abi_l1b:  ok
abi_l1b_scmi:  ok
abi_l2_nc:  ok
acspo:  ok
agri_l1:  ok
ahi_hrit:  ok
ahi_hsd:  ok
ahi_l1b_gridded_bin:  ok
ami_l1b:  ok
amsr2_l1b:  ok
amsr2_l2:  ok
amsr2_l2_gaasp:  ok
amsub_l1c_aapp:  ok
ascat_l2_soilmoisture_bufr:  ok
avhrr_l1b_aapp:  ok
avhrr_l1b_eps:  ok
avhrr_l1b_gaclac:  ok
avhrr_l1b_hrpt:  ok
avhrr_l1c_eum_gac_fdr_nc:  ok
caliop_l2_cloud:  cannot find module 'satpy.readers.caliop_l2_cloud' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
clavrx:  ok
cmsaf-claas2_l2_nc:  ok
electrol_hrit:  ok
fci_l1c_nc:  ok
fci_l2_nc:  ok
generic_image:  ok
geocat:  ok
ghrsst_l3c_sst:  cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
glm_l2:  ok
goes-imager_hrit:  ok
goes-imager_nc:  ok
gpm_imerg:  ok
grib:  ok
hsaf_grib:  ok
hy2_scat_l2b_h5:  ok
iasi_l2:  ok
iasi_l2_so2_bufr:  ok
jami_hrit:  ok
li_l2:  cannot find module 'satpy.readers.li_l2' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
maia:  ok
mersi2_l1b:  ok
mhs_l1c_aapp:  ok
mimicTPW2_comp:  ok
mirs:  ok
modis_l1b:  ok
modis_l2:  ok
msi_safe:  ok
mtsat2-imager_hrit:  ok
mviri_l1b_fiduceo_nc:  ok
nucaps:  ok
nwcsaf-geo:  ok
nwcsaf-msg2013-hdf5:  ok
nwcsaf-pps_nc:  ok
olci_l1b:  ok
olci_l2:  ok
omps_edr:  ok
safe_sar_l2_ocn:  ok
sar-c_safe:  ok
satpy_cf_nc:  ok
scatsat1_l2b:  cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/data/gholl/checkouts/satpy/satpy/dataset/__init__.py))
seadas_l2:  ok
seviri_l1b_hrit:  ok
seviri_l1b_icare:  ok
seviri_l1b_native:  ok
seviri_l1b_nc:  ok
seviri_l2_bufr:  ok
seviri_l2_grib:  ok
slstr_l1b:  ok
slstr_l2:  ok
smos_l2_wind:  ok
tropomi_l2:  ok
vaisala_gld360:  ok
vii_l1b_nc:  ok
vii_l2_nc:  ok
viirs_compact:  ok
viirs_edr_active_fires:  ok
viirs_edr_flood:  ok
viirs_l1b:  ok
viirs_sdr:  ok
virr_l1b:  ok
world_generic_image:  ok

Writers
=======
/data/gholl/mambaforge/envs/py310/lib/python3.10/site-packages/pyninjotiff/tifffile.py:154: UserWarning: failed to import the optional _tifffile C extension module.
Loading of some compressed images will be slow.
Tifffile.c can be obtained at http://www.lfd.uci.edu/~gohlke/
  warnings.warn(
awips_tiled:  ok
cf:  ok
geotiff:  ok
geotiff_nostretch:  ok
mitiff:  ok
ninjogeotiff:  ok
ninjotiff:  ok
ninjotiff_nostretch:  ok
simple_image:  ok

Extras
======
No protocol specified
cartopy:  ok
geoviews:  ok

Additional context

The failing readers appear to be not covered by unit tests, or this would come up during unit testing. If those readers are currently unsupported, which I think they are, then probably they should be explicitly excluded as such. When users following satpy documentation call available_readers() and are presented with multiple tracebacks, they may think something is wrong.

djhoese commented 2 years ago

You are right that they need to be updated. I'm not sure explicitly excluding them from available_readers imports is the best idea as we may forget and if/when the reader is updated they won't be shown in available_readers.

Looking closer though, it looks like these tracebacks are actually DEBUG log messages, right?

joleenf commented 2 years ago

I just was asked to try the caliop reader and came across this error. So the conclusion is the caliop reader is officially not supported?

djhoese commented 2 years ago

@joleenf Correct. The reader was never updated to work with xarray/dask.

ameraner commented 1 year ago

See also https://github.com/pytroll/satpy/issues/1377 and https://github.com/pytroll/satpy/issues/399