pytroll / trollflow2

Next generation Trollflow. Trollflow is for batch-processing satellite data using Satpy
https://trollflow2.readthedocs.org/
GNU General Public License v3.0
10 stars 15 forks source link

Netcdf product with multiple dataset can't be created anymore #116

Closed mraspaud closed 3 years ago

mraspaud commented 3 years ago

Describe the bug Since the introduction of #109 , the following product list makes trollflow2 crash:

To Reproduce

  areas:
    null:
      areaname: satproj
      products:
        ("chl_nn", "chl_oc4me", "trsp", "tsm_nn", "iop_nn", "mask", "latitude", "longitude"):
          min_sunlight_coverage: 10
          productname: sat_coast
          publish_topic: /nc/2C/olcil2
          formats:
            - format: nc
              writer: cf
              encoding:
                latitude: {'dtype': 'int32', 'scale_factor': 1.0e-6, '_FillValue': -200000000, 'zlib': true, 'complevel': 2}
                longitude: {'dtype': 'int32', 'scale_factor': 1.0e-6, '_FillValue': -200000000, 'zlib': true, 'complevel': 2}

Expected behavior I expect a netcdf file with the datasets "chl_nn", "chl_oc4me", "trsp", "tsm_nn", "iop_nn", "mask", "latitude", and "longitude" to be created.

Actual results

Traceback (most recent call last):
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/trollflow2/launcher.py", line 303, in process
    cwrk.pop('fun')(job, **cwrk)
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/trollflow2/plugins/__init__.py", line 339, in __call__
    if fmat['product'] not in job['resampled_scenes'].get(fmat['area'], []):
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/satpy/scene.py", line 690, in __contains__
    return name in self._datasets
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/satpy/dataset/data_dict.py", line 238, in __contains__
    key = self.get_key(item)
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/satpy/dataset/data_dict.py", line 159, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/satpy/dataset/data_dict.py", line 104, in get_key
    key = create_filtered_query(key, query)
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/satpy/dataset/dataid.py", line 719, in create_filtered_query
    ds_dict = _create_id_dict_from_any_key(dataset_key)
  File "/san1/opt/olcil2_imagery_processor/releases/olcil2_imagery_processor-0.1.5/lib/python3.9/site-packages/satpy/dataset/dataid.py", line 741, in _create_id_dict_from_any_key
    raise TypeError("Don't know how to interpret a dataset_key of type {}".format(type(dataset_key)))

Environment Info: