search

pytroll / satpy

Python package for earth-observing satellite data processing
http://satpy.readthedocs.org/en/latest/
GNU General Public License v3.0
1.07k stars 295 forks source link

Make nwcsaf-geo composites available to other geostationary imagers besides SEVIRI #1134

Open gerritholl opened 4 years ago

gerritholl commented 4 years ago

Feature Request

Is your feature request related to a problem? Please describe.

I'd like to use the NWCSAF composites defined in visir.yaml and seviri.yaml when reading a nwcsaf-geo file produced with ABI data. This fails for most composites, because the prerequisites as defined in visir.yaml are not met for geostationary data, and the composites defined in seviri.yaml are not used when data are coming from ABI, AHI, or FCI.

When reading a NWCSAF-GEO file produced with ABI:

In [25]: sc = Scene(filenames=glob("/media/nas/x21308/NWCSAF/data/*180610Z.nc
    ...: "), reader="nwcsaf-geo")
DEBUG    satpy.scene 2020-04-06 18:44:31,406 scene.__init__:108: Setting 'PPP_CONFIG_DIR' to '/home/gholl/checkouts/satpy/satpy/etc/'
DEBUG    satpy.readers 2020-04-06 18:44:31,407 __init__.read_reader_config:483: Reading ['/data/gholl/miniconda3/envs/py38/lib/python3.8/site-packages/satpy/etc/readers/nwcsaf-geo.yaml', '/home/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-geo.yaml']
DEBUG    satpy.readers.yaml_reader 2020-04-06 18:44:31,698 yaml_reader.create_filehandlers:525: Assigning to nwcsaf-geo: ['/media/nas/x21308/NWCSAF/data/S_NWC_CMA_GOES16_NEW-ENGLAND-NR_20170314T180610Z.nc', '/media/nas/x21308/NWCSAF/data/S_NWC_CT_GOES16_NEW-ENGLAND-NR_20170314T180610Z.nc', '/media/nas/x21308/NWCSAF/data/S_NWC_CTTH_GOES16_NEW-ENGLAND-NR_20170314T180610Z.nc', '/media/nas/x21308/NWCSAF/data/S_NWC_CMIC_GOES16_NEW-ENGLAND-NR_20170314T180610Z.nc']
WARNING  satpy.readers 2020-04-06 18:44:31,730 __init__.load_readers:728: Don't know how to open the following files: {'/media/nas/x21308/NWCSAF/data/S_NWC_AUX-SUN_GOES16_NEW-ENGLAND-NR_20170314T180610Z.nc'}
DEBUG    satpy.composites 2020-04-06 18:44:31,730 __init__.load_sensor_composites:85: Looking for composites config file abi.yaml
DEBUG    satpy.composites 2020-04-06 18:44:31,771 __init__.load_sensor_composites:85: Looking for composites config file visir.yaml

In [26]: print(sc.available_composite_names())
['cloud_top_height', 'cloud_top_pressure', 'cloud_top_temperature', 'cloudmask', 'cloudtype']

When reading one produced with SEVIRI:

In [28]: sc2 = Scene(filenames=glob("/media/nas/x21308/scratch/NWCSAF/2019041
    ...: 5/*_EUROPE-B-VISIR_20190415T110000Z.nc"), reader="nwcsaf-geo")
DEBUG    satpy.scene 2020-04-06 18:46:51,743 scene.__init__:108: Setting 'PPP_CONFIG_DIR' to '/home/gholl/checkouts/satpy/satpy/etc/'
DEBUG    satpy.readers 2020-04-06 18:46:51,743 __init__.read_reader_config:483: Reading ['/data/gholl/miniconda3/envs/py38/lib/python3.8/site-packages/satpy/etc/readers/nwcsaf-geo.yaml', '/home/gholl/checkouts/satpy/satpy/etc/readers/nwcsaf-geo.yaml']
DEBUG    satpy.readers.yaml_reader 2020-04-06 18:46:51,938 yaml_reader.create_filehandlers:525: Assigning to nwcsaf-geo: ['/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CMA_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CT_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CTTH_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CMIC_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_PC_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CRR_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_iSHAI_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CI_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_RDT-CW_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_ASII-NG_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc']
WARNING  satpy.readers 2020-04-06 18:46:52,035 __init__.load_readers:728: Don't know how to open the following files: {'/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_PLAX_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_CRR-Ph_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc', '/media/nas/x21308/scratch/NWCSAF/20190415/S_NWC_PC-Ph_MSG4_EUROPE-B-VISIR_20190415T110000Z.nc'}
DEBUG    satpy.composites 2020-04-06 18:46:52,035 __init__.load_sensor_composites:85: Looking for composites config file seviri.yaml
DEBUG    satpy.composites 2020-04-06 18:46:52,101 __init__.load_sensor_composites:85: Looking for composites config file visir.yaml

In [29]: print(sc2.available_composite_names())
['cloud_drop_effective_radius', 'cloud_ice_water_path', 'cloud_liquid_water_path', 'cloud_optical_thickness', 'cloud_top_height', 'cloud_top_phase', 'cloud_top_pressure', 'cloud_top_temperature', 'cloudmask', 'cloudtype', 'convection_initiation_prob30', 'convection_initiation_prob60', 'convection_initiation_prob90', 'convective_precipitation_hourly_accumulation', 'convective_rain_rate', 'lifted_index', 'precipitation_probability', 'rdt_cell_type', 'showalter_index', 'total_precipitable_water']

Describe the solution you'd like

I'd like that composites such as cloud_optical_thickness can be used for any NWCSAF-GEO data, regardless of what geostationary imager was used to produce the data.

Describe any changes to existing user workflow

None.

Additional context

Those composites were apparently first added in #41. Reading NWCSAF-GEO from ABI has been correctly possible since #1111 was closed in #1113.

gerritholl commented 4 years ago

How could this be addressed in a maintainable way? Move the composites that are currently in seviri.yaml to a new nwcsaf-geo.yaml thus providing for ABI, AHI, and FCI in addition to SEVIRI? Would we need to include them into each of abi.yaml, ahi.yaml, and fci.yaml for Satpy to find them there?