pytroll / pyresample

Geospatial image resampling in Python
http://pyresample.readthedocs.org
GNU Lesser General Public License v3.0
344 stars 95 forks source link

`gradient_search` fails when resampling Himawari data #507

Closed simonrp84 closed 1 year ago

simonrp84 commented 1 year ago

The gradient_search method is failing when trying to resample Himawari data from one resolution to another:

Code Sample, a minimal, complete, and verifiable piece of code

from satpy import Scene
from glob import glob

scn = Scene(glob(f'{input_dir}/*.DAT', reader='ahi_hsd')
scn.load(['true_color_nocorr'])

scn2 = scn.resample(scn.coarsest_area(), resampler='gradient_search')
scn2.save_datasets()

Problem description

The above code produces the following error message:

File ~\miniconda\lib\site-packages\pyresample\resampler.py:202, in resample_blocks(func, src_area, src_arrays, dst_area, dst_arrays, chunk_size, dtype, name, fill_value, **kwargs)
    155 """Resample dask arrays blockwise.
    156 
    157 Resample_blocks applies a function blockwise to transform data from a source
   (...)
    199 
    200 """
    201 if dst_area == src_area:
--> 202     raise ValueError("Source and destination areas are identical."
    203                      " Should you be running `map_blocks` instead of `resample_blocks`?")
    205 name = _create_dask_name(name, func,
    206                          src_area, src_arrays,
    207                          dst_area, dst_arrays,
    208                          fill_value, dtype, chunk_size, kwargs)
    209 dask_graph = dict()

ValueError: Source and destination areas are identical. Should you be running `map_blocks` instead of `resample_blocks`?

If I comment out lines 202 and 203 then it runs fine, so presumably something is being incorrectly compared in that if statement. For Himawari true color, only one band has a different resolution to the others so my guess is that the comparison is being made between B01 and B02, which are the same resolution. Perhaps this should compare all input bands or something?

Expected Output

A resampled image.

Versions of Python, package at hand and relevant dependencies

Python = 3.10 Pyresample = 1.26.1 (from conda-forge). OS: Windows 10

mraspaud commented 1 year ago

@simonrp84 thanks for reporting the issue.

It looks like pyresample complains that the source and destination areas are the same, which is actually true for some of the channels of the true color image I suppose, and probably not for others. My first reaction here is why not use the native resampler instead? Are you after the bilinear interpolation gradient search provides?

simonrp84 commented 1 year ago

No, this is just a simple example to prove the bug. I'm actually trying to use the true_color_with_night_ir composite, which is unable to use native as it has non-geos data included. It produces the same error.

mraspaud commented 1 year ago

I understand. I suppose one solution will be to make this a warning rather than an error...

djhoese commented 1 year ago

@mraspaud perhaps Satpy, if it doesn't already, should skip resampling data in these cases and instead do a .copy(deep=True) on the DataArray? Why don't we see this issue with the kdtree version?

djhoese commented 1 year ago

Wait...are we completely wasting time with all resamplers by asking to resample data to its same projection?!?!? No. I don't believe it.

simonrp84 commented 1 year ago

Yeah, that was going to be my next question: If this is a warning rather than an error then what happens with the data? Sounds like it might still get 'resampled' even for the bands where there's no need...

mraspaud commented 1 year ago

So, looking at this I can see that in the case of the kdtree resampler, the resampling will take place even if source and target areas are identical. Native works around it by nature (sizes being the same, nothing is done). Gradient search atm raises an error. Working on a PR for the latter.