search

pytroll / pyresample

Geospatial image resampling in Python
http://pyresample.readthedocs.org
GNU Lesser General Public License v3.0
347 stars 94 forks source link

Numerous `RuntimeWarning`s when resampling #187

Closed gerritholl closed 5 years ago

gerritholl commented 5 years ago

Code Sample, a minimal, complete, and verifiable piece of code


import warnings
from satpy import Scene
from pyresample.geometry import AreaDefinition
from glob import glob
from satpy.utils import debug_on
debug_on()
fn_sev = glob("/media/nas/x21308/scratch/SEVIRI/*201904151000*")
sc_sev = Scene(reader="seviri_l1b_hrit", filenames=fn_sev)
sc_sev.load(["IR_108", "IR_087"])
with warnings.catch_warnings():
    warnings.filterwarnings("error", "invalid value encountered", RuntimeWarning)
    ad = AreaDefinition("spam", "eggs", "fubar", {'a': 6378137.0, 'b':
        6356752.3, 'h': 35785863.0, 'lon_0': 0.0, 'proj': 'geos',
        'sweep': 'y', 'units': 'm'}, 1950, 910, (-2701500.0, 2644133.0,
            3149286.5, 5374500.0))
    ls = sc_sev.resample(ad)

Problem description

Resampling to this projection (which happens to be identical to the projection of data files I read with the nwcsaf-geo reader, region id EUROPE-B) leads to very numerous RuntimeWarnings. In the MWE above I have turned this into an exception so that I can inspect it closer.

Expected Output

I expect no such RuntimeWarnings. I don't know if they are problematic. They do not occur with most other projections. I don't know if the result is still correct.

Actual Result, Traceback if applicable

/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyspectral/config.py:75: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.                                                                                                                                                                                                                                                                                          [16/1831]
  config = recursive_dict_update(config, yaml.load(fp_))                                                                          
[DEBUG: 2019-05-02 17:24:50 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/home/gholl/checkouts/geotools/etc/'
[DEBUG: 2019-05-02 17:24:50 : satpy.readers] Reading ['/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/satpy/etc/readers/seviri_l1b_hrit.yaml']
[DEBUG: 2019-05-02 17:24:50 : satpy.readers.yaml_reader] Assigning to seviri_l1b_hrit: ['/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000016___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000018___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV_$
____-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000009___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000020___-201904151000-__', '/media/nas/x21308/scr
atch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000022___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000013___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000015___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000017_
__-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000014___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000011___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000024___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000019___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000010___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/
H-000-MSG4__-MSG4________-HRV______-000021___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000012___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-HRV______-000023___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000002___-201904151
000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_016___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__
-MSG4________-IR_039___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000001___-201904151000-__', '/m
edia/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_039___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4_______
_-IR_087___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_087___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000005___-201904151000-__', '/media/nas/x21
308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-
000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_097___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000006___-201904151000-__', '/media/nas/x21308/scratch/
SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_108___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000005___-20
1904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_120___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000
-MSG4__-MSG4________-IR_120___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000006___-201904151000-_
_', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-IR_134___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4
________-VIS006___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS006___-000007___-201904151000-__', '/media/
nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS
008___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-VIS008___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000001___-201904151000-__', '/media/nas/x21308/s
cratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000002___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_062___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-00000
2___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-000005___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-000008___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-000004___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-000006___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-000003___-201904151000-__', '/media/nas/x21308/scratch/SEVIR
I/H-000-MSG4__-MSG4________-WV_073___-000001___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-WV_073___-000007___-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-_________-PRO______-201904151000-__', '/media/nas/x21308/scratch/SEVIRI/H-000-MSG4__-MSG4________-_________-EPI______-201904151000-__']
[INFO: 2019-05-02 17:24:51 : hrit_msg] No IMPF configuration field found in prologue.
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Computing actual satellite position
[WARNING: 2019-05-02 17:24:51 : satpy.readers] Don't know how to open the following files: {'/media/nas/x21308/scratch/SEVIRI/seviri_201904151000.tar'}
[DEBUG: 2019-05-02 17:24:51 : satpy.composites] Looking for composites config file seviri.yaml
[DEBUG: 2019-05-02 17:24:51 : satpy.composites] Looking for composites config file visir.yaml
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.008221
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007624
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007556
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007222
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007518
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007709
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007256
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007256
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007642
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007384
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007386
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007351
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007267
[DEBUG: 2019-05-02 17:24:51 : hrit_msg] Calibration time 0:00:00.007246
[DEBUG: 2019-05-02 17:24:52 : hrit_msg] Calibration time 0:00:00.007527
[DEBUG: 2019-05-02 17:24:52 : hrit_msg] Calibration time 0:00:00.008146
[DEBUG: 2019-05-02 17:24:52 : satpy.scene] Setting 'PPP_CONFIG_DIR' to '/home/gholl/checkouts/geotools/etc/'
[DEBUG: 2019-05-02 17:24:52 : satpy.scene] Resampling DatasetID(name='IR_087', wavelength=(8.3, 8.7, 9.1), resolution=3000.403165817, polarization=None, calibration='brightness_temperature', level=None, modifiers=())
Traceback (most recent call last):
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/satpy/scene.py", line 1012, in _resampled_scene
    (slice_x, slice_y), source_area = reductions[key]
KeyError: Area ID: some_area_name
Description: On-the-fly area
Projection ID: geosmsg
Projection: {'a': '6378169.0', 'b': '6356583.800000001', 'h': '35785831.0', 'lon_0': '0.0', 'proj': 'geos', 'units': 'm'}
Number of columns: 3712
Number of rows: 3712
Area extent: (5567248.2834, 5570248.6867, -5570248.6867, -5567248.2834)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "mwe6.py", line 16, in <module>
    ls = sc_sev.resample(ad)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/satpy/scene.py", line 1074, in resample
    reduce_data=reduce_data, **resample_kwargs)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/satpy/scene.py", line 1014, in _resampled_scene
    slice_x, slice_y = source_area.get_area_slices(destination_area)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py", line 1620, in get_area_slices
    area_boundary.contour_poly)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py", line 495, in intersection
    return self._bool_oper(other, -1)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py", line 440, in _bool_oper
    inter, edge2 = edge1.get_next_intersection(arcs2)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py", line 305, in get_next_intersection
    inter = self.intersection(arc)
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py", line 284, in intersection
    for i in self.intersections(other_arc):
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py", line 257, in intersections
    eb_ = other_arc.start.cross2cart(other_arc.end).normalize()
  File "/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py", line 123, in normalize
    self.cart /= np.sqrt(np.einsum('...i, ...i', self.cart, self.cart))
RuntimeWarning: invalid value encountered in true_divide

Versions of Python, package at hand and relevant dependencies

gerritholl commented 5 years ago

Apparently this happens when the target area has an overlap with the source area that is larger than 0, but smaller than 1. Resampling to the world gives the same warning.