yukaribbba commented 3 years ago

Describe the bug After I updated the conda env by conda update -c conda-forge --all, this occured. Looks like it happened in atm correction.

To Reproduce

#!/usr/bin/env python
from satpy import Scene, find_files_and_readers
files = find_files_and_readers(base_dir="/mnt/hgfs/Downloads/Sentinel-2/",
                               reader='msi_safe')
scn = Scene(filenames=files)
composite = 'true_color_marine_clean'
scn.load([composite])
scn.save_dataset(composite, filename='{name}_{start_time:%Y%m%d_%H%M%S}.tif', writer='geotiff', base_dir="/mnt/hgfs/Downloads/Sentinel-2/", num_threads=8, compress=None)

Expected behavior No error.

Actual results

/home/ll/anaconda3/lib/python3.8/site-packages/satpy/resample.py:150: UserWarning: Gradient search resampler not available, upgrade Pyresample. warnings.warn('Gradient search resampler not available, upgrade Pyresample.') /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/msi_safe.py:199: RuntimeWarning: Mean of empty slice return np.nanmean(np.dstack(arrays), -1) /home/ll/anaconda3/lib/python3.8/site-packages/pyproj/crs/crs.py:543: UserWarning: You will likely lose important projection information when converting to a PROJ string from another format. See: https://proj.org/faq.html#what-is-the-best-format-for-describing-coordinate-reference-systems proj_string = self.to_proj4() /home/ll/anaconda3/lib/python3.8/site-packages/satpy/readers/msi_safe.py:199: RuntimeWarning: Mean of empty slice return np.nanmean(np.dstack(arrays), -1) Traceback (most recent call last): File "/home/ll/Downloads/s2t.py", line 7, in scn.load([composite]) File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 984, in load keepables = self._generate_composites() File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 897, in _generate_composites return self._read_composites(nodes) File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 873, in _read_composites self._generate_composite(item, keepables) File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 813, in _generate_composite prereq_datasets = self._get_prereq_datasets( File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 763, in _get_prereq_datasets self._generate_composite(prereq_node, keepables) File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/scene.py", line 847, in _generate_composite composite = compositor(prereq_datasets, File "/home/ll/anaconda3/lib/python3.8/site-packages/satpy/composites/init.py", line 604, in call refl_cor_band = corrector.get_reflectance(sunz, satz, ssadiff, File "/home/ll/anaconda3/lib/python3.8/site-packages/pyspectral/rayleigh.py", line 248, in get_reflectance wvl = self.get_effective_wavelength(bandname) File "/home/ll/anaconda3/lib/python3.8/site-packages/pyspectral/rayleigh.py", line 187, in get_effective_wavelength rsr = RelativeSpectralResponse(self.platform_name, self.sensor) File "/home/ll/anaconda3/lib/python3.8/site-packages/pyspectral/rsr_reader.py", line 125, in init self.load() File "/home/ll/anaconda3/lib/python3.8/site-packages/pyspectral/rsr_reader.py", line 174, in load self.description = self.description.decode('utf-8') AttributeError: 'str' object has no attribute 'decode'

Environment Info:

OS: Ubuntu 20.04 64bit
Satpy Version: 0.23
PyResample Version:

Readers and writers dependencies (when relevant): [run from satpy.config import check_satpy; check_satpy()]


Readers
=======
abi_l1b:  ok
abi_l1b_scmi:  ok
abi_l2_nc:  ok
acspo:  ok
agri_l1:  ok
ahi_hrit:  ok
ahi_hsd:  ok
ahi_l1b_gridded_bin:  ok
ami_l1b:  ok
amsr2_l1b:  ok
amsr2_l2:  ok
avhrr_l1b_aapp:  ok
avhrr_l1b_eps:  ok
avhrr_l1b_gaclac:  cannot find module 'satpy.readers.avhrr_l1b_gaclac' (No module named 'pygac')
avhrr_l1b_hrpt:  cannot find module 'satpy.readers.hrpt' (No module named 'pygac')
caliop_l2_cloud:  cannot find module 'satpy.readers.caliop_l2_cloud' (No module named 'pyhdf')
clavrx:  cannot find module 'satpy.readers.clavrx' (No module named 'pyhdf')
cmsaf-claas2_l2_nc:  ok
electrol_hrit:  ok
fci_l1c_fdhsi:  ok
fci_l2_nc:  ok
generic_image:  ok
geocat:  ok
ghrsst_l3c_sst:  cannot find module 'satpy.readers.ghrsst_l3c_sst' (cannot import name 'Dataset' from 'satpy.dataset' (/home/ll/anaconda3/lib/python3.8/site-packages/satpy/dataset/__init__.py))
glm_l2:  ok
goes-imager_hrit:  ok
goes-imager_nc:  ok
gpm_imerg:  ok
grib:  cannot find module 'satpy.readers.grib' (No module named 'pygrib')
hsaf_grib:  cannot find module 'satpy.readers.hsaf_grib' (No module named 'pygrib')
hy2_scat_l2b_h5:  ok
iasi_l2:  ok
iasi_l2_so2_bufr:  cannot find module 'satpy.readers.iasi_l2_so2_bufr' (('Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes.\n           Error: ', ModuleNotFoundError("No module named 'eccodes'")))
jami_hrit:  ok
li_l2:  cannot find module 'satpy.readers.li_l2' (No module named 'h5netcdf')
maia:  ok
mersi2_l1b:  ok
mimicTPW2_comp:  ok
modis_l1b:  cannot find module 'satpy.readers.modis_l1b' (No module named 'pyhdf')
modis_l2:  cannot find module 'satpy.readers.modis_l2' (No module named 'pyhdf')
msi_safe:  ok
mtsat2-imager_hrit:  ok
nucaps:  ok
nwcsaf-geo:  ok
nwcsaf-msg2013-hdf5:  ok
nwcsaf-pps_nc:  ok
olci_l1b:  ok
olci_l2:  ok
omps_edr:  ok
safe_sar_l2_ocn:  ok
sar-c_safe:  ok
scatsat1_l2b:  cannot find module 'satpy.readers.scatsat1_l2b' (cannot import name 'Dataset' from 'satpy.dataset' (/home/ll/anaconda3/lib/python3.8/site-packages/satpy/dataset/__init__.py))
seviri_l1b_hrit:  ok
seviri_l1b_icare:  cannot find module 'satpy.readers.seviri_l1b_icare' (No module named 'pyhdf')
seviri_l1b_native:  ok
seviri_l1b_nc:  ok
seviri_l2_bufr:  cannot find module 'satpy.readers.seviri_l2_bufr' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
seviri_l2_grib:  cannot find module 'satpy.readers.seviri_l2_grib' (Missing eccodes-python and/or eccodes C-library installation. Use conda to install eccodes)
slstr_l1b:  ok
slstr_l2:  ok
smos_l2_wind:  ok
tropomi_l2:  ok
vaisala_gld360:  ok
vii_l1b_nc:  ok
vii_l2_nc:  ok
viirs_compact:  ok
viirs_edr_active_fires:  ok
viirs_edr_flood:  cannot find module 'satpy.readers.viirs_edr_flood' (No module named 'pyhdf')
viirs_l1b:  ok
viirs_sdr:  ok
virr_l1b:  ok

Writers

cf: ok geotiff: ok mitiff: ok ninjotiff: cannot find module 'satpy.writers.ninjotiff' (No module named 'pyninjotiff') scmi: ok simple_image: ok

Extras

cartopy: No module named 'cartopy' geoviews: No module named 'geoviews'



**Additional context**
Add any other context about the problem here.

djhoese commented 3 years ago

This is an incompatibility between pyspectral and h5py 3.x. It is being worked on, but the short term solution would be to downgrade your version of h5py to <3. I think conda install "h5py<3" should work.

adybbroe commented 3 years ago

@yukaribbba Sorry about that. A PR is about to be merged in https://github.com/pytroll/pyspectral Until then follow the advise by @djhoese

yukaribbba commented 3 years ago

@djhoese @adybbroe Ok thank you.

mraspaud commented 3 years ago

@yukaribbba the new pyspectral is out now, if you want to try it out!

yukaribbba commented 3 years ago

@mraspaud Yep! The problem is solved by updating pyspectral to 0.10.2. Thanks!

mraspaud commented 3 years ago

perfect, thanks for checking. Closing.

pytroll / satpy

'str' object has no attribute 'decode' during Sentinel-2 MSI processing #1436

Writers

Extras