Problems in creating hrv_clouds RGB composite with FES Seviri data

[valegreg]

Hi,

I'm new here, so I hope I'm using the right thread. I'm having troubles in creating the 'hrv_clouds' RGB composite using the Full Earth Scan (FES) Seviri data.

I report here the lines of code to reproduce the error I get:

from satpy.scene import Scene
from satpy.writers import get_enhanced_image

composite = 'hrv_clouds'

scn = Scene(reader="seviri_l1b_native", filenames=['MSG3-SEVI-MSG15-0100-NA-20240601121242.418000000Z-NA.nat'])
scn.load([composite], upper_right_corner="NE")

res = scn.resample('msg_seviri_fes_1km', resample='nearest')
scncrop = res.crop(xy_bbox=(-15E5, 30E5, 25E5, 48E5))

scncrop.save_dataset(composite, writer='geotiff', filename='test_MSG3.tiff', compress='DEFLATE')

This is the traceback of the error I get:

Traceback (most recent call last):
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/highlevelgraph.py", line 569, in get_all_external_keys
    return self._all_external_keys
           ^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'HighLevelGraph' object has no attribute '_all_external_keys'. Did you mean: 'get_all_external_keys'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jenkinsdeploy/service/satellite/clear_index/./test.py", line 14, in <module>
    scncrop.save_dataset(composite, writer='geotiff', filename='test_MSG3.tiff', compress='DEFLATE')
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/satpy/scene.py", line 1231, in save_dataset
    return writer.save_dataset(self[dataset_id],
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/satpy/writers/__init__.py", line 885, in save_dataset
    return self.save_image(img, filename=filename, compute=compute, fill_value=fill_value, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/satpy/writers/geotiff.py", line 278, in save_image
    return img.save(filename, fformat="tif", driver=driver,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/trollimage/xrimage.py", line 248, in save
    return self.rio_save(filename, fformat=fformat, driver=driver,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/trollimage/xrimage.py", line 432, in rio_save
    res = da.store(*to_store)
          ^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/array/core.py", line 1162, in store
    sources_layer = Array.__dask_optimize__(
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/array/optimization.py", line 51, in optimize
    dsk = dsk.cull(set(keys))
          ^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/highlevelgraph.py", line 726, in cull
    all_ext_keys = self.get_all_external_keys()
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/highlevelgraph.py", line 576, in get_all_external_keys
    keys.update(layer.get_output_keys())
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/blockwise.py", line 487, in get_output_keys
    *[range(self.dims[i]) for i in self.output_indices]
            ^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/blockwise.py", line 447, in dims
    self._dims = _make_dims(self.indices, self.numblocks, self.new_axes)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/blockwise.py", line 1484, in _make_dims
    dims = broadcast_dimensions(indices, numblocks)
[satpy.txt](https://github.com/user-attachments/files/15822225/satpy.txt)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jenkinsdeploy/miniconda3/envs/test_satpy/lib/python3.12/site-packages/dask/blockwise.py", line 1475, in broadcast_dimensions
    raise ValueError("Shapes do not align %s" % g)
ValueError: Shapes do not align {'.1': {2, 12}, '.0': {1}}

Instead, if I use the same code to calculate composite = 'convection', the code works fine saving at the end the required .tiff file. Therefore, I suppose the problem has something to do with the 'HRV' Seviri channel, which resolution is higher than the resolution of the 'IR_108' channel used to calculate the composite I need.

I attach here the 'satpy.txt' satpy.txt file with the version of the libraries I'm using in the conda environment.

Thank you in advance for the support!

[ameraner]

Hi @valegreg , could you please try to add the fill_disk: True reader kwarg as in: scn = Scene(reader="seviri_l1b_native", filenames=['MSG3-SEVI-MSG15-0100-NA-20240601121242.418000000Z-NA.nat'], reader_kwargs={'fill_disk': True})

I believe this is a duplicate of https://github.com/pytroll/satpy/issues/2477

[valegreg]

Yes! It works! Thank you very much, I didn't realize my question was a duplicate. Thank you again for your time and the quick reply.

[ameraner]

No problem, you're welcome! I'll close this issue then, feel free to reopen or make a new one if there are more issues.