search

pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.08k stars 298 forks source link

nwcsaf-geo reader turns uint8 into int64 on numpy 2.0 #2872

Closed gerritholl closed 3 months ago

gerritholl commented 3 months ago

Describe the bug

When using numpy 2.0, the nwcsaf-geo reader turns data of type uint8 into int64. This wastes memory and leads to other problems downstream.

To Reproduce

from satpy import Scene
from satpy.utils import debug_on; debug_on()
fn = "/media/nas/x21308/scratch/NWCSAF/202302010730/S_NWC_CT_MSG4_MSG-N-VISIR_20230201T073000Z.nc"
sc = Scene(filenames=[fn], reader=["nwcsaf-geo"])
sc.available_dataset_names()
sc.load(["ct", "ct_pal"])
print(sc["ct"].dtype, sc["ct_pal"].dtype)

Expected behavior

I expect those to be uint8, like in the original data:

$ ncdump -h /media/nas/x21308/scratch/NWCSAF/202302010730/S_NWC_CT_MSG4_MSG-N-VISIR_20230201T073000Z.nc | grep 'ct\('
        ubyte ct(ny, nx) ;

Actual results

[DEBUG: 2024-08-06 14:23:14 : satpy.readers.yaml_reader] Reading ('/data/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-geo.yaml',)
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.yaml_reader] Assigning to nwcsaf-geo: ['/media/nas/x21308/scratch/NWCSAF/202302010730/S_NWC_CT_MSG4_MSG-N-VISIR_20230201T073000Z.nc']
[DEBUG: 2024-08-06 14:23:14 : satpy.composites.config_loader] Looking for composites config file seviri.yaml
[DEBUG: 2024-08-06 14:23:14 : pyorbital.tlefile] Path to the Pyorbital configuration (where e.g. platforms.txt is found): /data/gholl/mambaforge/envs/py312/lib/python3.12/site-packages/pyorbital/etc
[DEBUG: 2024-08-06 14:23:14 : satpy.composites.config_loader] Looking for composites config file visir.yaml
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.nwcsaf_nc] Reading ct_pal.
[DEBUG: 2024-08-06 14:23:14 : h5py._conv] Creating converter from 7 to 5
[DEBUG: 2024-08-06 14:23:14 : h5py._conv] Creating converter from 5 to 7
[DEBUG: 2024-08-06 14:23:14 : h5py._conv] Creating converter from 7 to 5
[DEBUG: 2024-08-06 14:23:14 : h5py._conv] Creating converter from 5 to 7
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.yaml_reader] No coordinates found for DataID(name='ct_pal', resolution=3000, modifiers=())
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.nwcsaf_nc] Reading ct.
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.nwcsaf_nc] Reading ct_quality.
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.nwcsaf_nc] Reading ct_conditions.
[DEBUG: 2024-08-06 14:23:14 : satpy.readers.nwcsaf_nc] Reading ct_status_flag.
int64 int64

Environment Info:

Additional context

Found this while working on #2870. In trollimage, the colormap is scaled unless data are of dtype uint8, so the change of dtype results in a faulty colormap for categorical data (such as cloud type).

This problem exists only with Numy 2.0. Downgrading to numpy 1.24.4 eliminates the problem.