pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.05k stars 289 forks source link

abi_l2_nc reader unable to read MCMIP mesoscale files #2072

Open zebengberg opened 2 years ago

zebengberg commented 2 years ago

Describe the bug

I'm unable to open MCMIP files for GOES mesoscale datasets with Scene.

To Reproduce

Here is a minimal example following the template from one of the example notebooks.

import fsspec
from satpy.readers import FSFile
from satpy import Scene

gs_path = "gs://gcp-public-data-goes-16/ABI-L2-MCMIPM/2021/073/12/OR_ABI-L2-MCMIPM1-M6_G16_s20210731200278*.nc"
the_files = fsspec.open_files(gs_path)
fs_files = [FSFile(open_file) for open_file in the_files]
scene = Scene(fs_files, reader="abi_l2_nc")

Actual results

I get the following ValueError.

No filenames found for reader: abi_l2_nc
Don't know how to open the following files: {<FSFile "gcp-public-data-goes-16/ABI-L2-MCMIPM/2021/073/12/OR_ABI-L2-MCMIPM1-M6_G16_s20210731200278_e20210731200341_c20210731200410.nc">}

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Input In [2], in <cell line: 8>()
      6 the_files = fsspec.open_files(gs_path)
      7 fs_files = [FSFile(open_file) for open_file in the_files]
----> 8 scene = Scene(fs_files, reader="abi_l2_nc")

File ~/.conda/envs/env/lib/python3.9/site-packages/satpy/scene.py:106, in Scene.__init__(self, filenames, reader, filter_parameters, reader_kwargs)
    103 if filenames and isinstance(filenames, str):
    104     raise ValueError("'filenames' must be a list of files: Scene(filenames=[filename])")
--> 106 self._readers = self._create_reader_instances(filenames=filenames,
    107                                               reader=reader,
    108                                               reader_kwargs=reader_kwargs)
    109 self._datasets = DatasetDict()
    110 self._wishlist = set()

File ~/.conda/envs/env/lib/python3.9/site-packages/satpy/scene.py:127, in Scene._create_reader_instances(self, filenames, reader, reader_kwargs)
    122 def _create_reader_instances(self,
    123                              filenames=None,
    124                              reader=None,
    125                              reader_kwargs=None):
    126     """Find readers and return their instances."""
--> 127     return load_readers(filenames=filenames,
    128                         reader=reader,
    129                         reader_kwargs=reader_kwargs)

File ~/.conda/envs/env/lib/python3.9/site-packages/satpy/readers/__init__.py:578, in load_readers(filenames, reader, reader_kwargs)
    575         break
    577 _check_remaining_files(remaining_filenames)
--> 578 _check_reader_instances(reader_instances)
    579 return reader_instances

File ~/.conda/envs/env/lib/python3.9/site-packages/satpy/readers/__init__.py:617, in _check_reader_instances(reader_instances)
    615 def _check_reader_instances(reader_instances):
    616     if not reader_instances:
--> 617         raise ValueError("No supported files found")
    618     if not any(list(r.available_dataset_ids) for r in reader_instances.values()):
    619         raise ValueError("No dataset could be loaded. Either missing "
    620                          "requirements (such as Epilog, Prolog) or none of the "
    621                          "provided files match the filter parameters.")

ValueError: No supported files found

Additional context

Related to #1162.

It's possible that generalizing the pattern in this yaml file from {scene_abbr:1s} to {scene_abbr:s} would fix this.

djhoese commented 2 years ago

Thanks for filing the bug and coming up with a solution. Your suggested solution of replacing it with :s looks like the best option to me. It should also be applied to the C16 file pattern just above the MCMIP file pattern. What do you think about making a pull request?