search

pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.06k stars 293 forks source link

NUCAPS time format of data from CLASS #1387

Closed joleenf closed 3 years ago

joleenf commented 4 years ago

Describe the bug The Scene object is not able to load data from recently downloaded NUCAPS data from NOAA Class. There seems to have been a time format string change within the files. There is a format change of the time_coverage_start and time_coverage_end strings. I believe the format of the time variable is unchanged.

To Reproduce Get NUCAPS EDR data from NOAA Class: (https://www.avl.class.noaa.gov/saa/products/search?datatype_family=JPSS_SND)

from datetime import datetime
from satpy import find_files_and_readers, Scene
from satpy.utils import debug_on; debug_on()

files = find_files_and_readers(start_time=datetime(2020,10,1,8,29), end_time=datetime(2020,10,1,8,32))
scn = Scene(files)

Expected behavior The Scene object should load the files without error.

Actual results [DEBUG: 2020-10-02 12:24:14 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/Users/joleenf/code/satpy/satpy/etc' [DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Reading ('/Users/joleenf/code/satpy/satpy/etc/readers/nucaps.yaml',) [DEBUG: 2020-10-02 12:24:14 : satpy.readers.yaml_reader] Assigning to nucaps: ['NUCAPS-EDR_v2r0_npp_s202010010831030_e202010010831330_c202010011004460.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829270_e202010010829570_c202010011018340.nc', 'NUCAPS-EDR_v2r0_npp_s202010010831350_e202010010832050_c202010011018180.nc', 'NUCAPS-EDR_v2r0_npp_s202010010828550_e202010010829250_c202010011017490.nc', 'NUCAPS-EDR_v2r0_npp_s202010010830310_e202010010831010_c202010011003140.nc', 'NUCAPS-EDR_v2r0_npp_s202010010829590_e202010010830290_c202010011018170.nc'] Traceback (most recent call last): File "", line 1, in File "/Users/joleenf/code/satpy/satpy/scene.py", line 151, in init self._readers = self._create_reader_instances(filenames=filenames, File "/Users/joleenf/code/satpy/satpy/scene.py", line 200, in _create_reader_instances return load_readers(filenames=filenames, File "/Users/joleenf/code/satpy/satpy/readers/init.py", line 495, in load_readers reader_instance.create_filehandlers(loadables, fh_kwargs=reader_kwargs_without_filter) File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 604, in create_filehandlers filehandlers = self._new_filehandlers_for_filetype(filetype_info, File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 592, in _new_filehandlers_for_filetype return list(filtered_iter) File "/Users/joleenf/code/satpy/satpy/readers/yaml_reader.py", line 561, in filter_fh_by_metadata filehandler.metadata['start_time'] = filehandler.start_time File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 90, in start_time return self._parse_datetime(self['/attr/time_coverage_start']) File "/Users/joleenf/code/satpy/satpy/readers/nucaps.py", line 84, in _parse_datetime return datetime.strptime(datestr, "%Y-%m-%dT%H:%M:%S.%fZ") File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 568, in _strptime_datetime tt, fraction, gmtoff_fraction = _strptime(data_string, format) File "/Users/joleenf/miniconda3/envs/satpy-dev/lib/python3.8/_strptime.py", line 349, in _strptime raise ValueError("time data %r does not match format %r" % ValueError: time data '2020-10-01T08:31:03Z' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

Environment Info:

djhoese commented 4 years ago

Looks like maybe we should do a double check. First with the format with the .%f part and then if that fails, try again without the .%f part (note the . is in the format which makes it a required character).