search

pytroll / fogpy

Satellite based fog and low stratus detection and nowcasting
http://fogpy.readthedocs.io/en/latest
GNU General Public License v3.0
20 stars 10 forks source link

Numerous warnings when generating composites #9

Open gerritholl opened 5 years ago

gerritholl commented 5 years ago

Generating composites issues numerous warnings:

from satpy import Scene
from glob import glob
from satpy.utils import debug_on
debug_on()
fn_nwcsaf = glob("/media/nas/x21308/scratch/NWCSAF/*100000Z.nc")
fn_sev = glob("/media/nas/x21308/scratch/SEVIRI/*201904151000*")
fn_dem=["/media/nas/x21308/DEM/dem_eu_1km.tif"]
sc = Scene(filenames={"seviri_l1b_hrit": fn_sev, "nwcsaf-geo": fn_nwcsaf})
sc.load(["cmic_reff", "IR_108", "IR_087", "cmic_cot", "IR_016", "VIS006",
    "IR_120", "VIS008", "cmic_lwp", "IR_039"])
ls = sc.resample("eurol")
ls.load(["fls_night"])
ls.show("fls_night")
ls.load(["fls_day", "fls_night"])
ls.show("fls_day")

Running this gives:

/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py:123: RuntimeWarning: invalid value encountered in true_divide
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/algorithms.py:1446: RuntimeWarning: invalid value encountered in less
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in less_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/dask/core.py:119: RuntimeWarning: invalid value encountered in greater_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:433: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:434: RuntimeWarning: invalid value encountered in less
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:437: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:498: RuntimeWarning: invalid value encountered in greater_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:499: RuntimeWarning: invalid value encountered in greater_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:535: RuntimeWarning: invalid value encountered in less
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:598: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:602: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:668: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:697: RuntimeWarning: invalid value encountered in less_equal
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/algorithms.py:692: RuntimeWarning: Mean of empty slice
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:757: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:855: RuntimeWarning: invalid value encountered in greater
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:1055: RuntimeWarning: Mean of empty slice
/media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/scipy/optimize/_minimize.py:517: RuntimeWarning: Method BFGS cannot handle constraints nor bounds.

The last warning is printed 4385 times, but that may be related to https://bugs.python.org/issue29672 .

gerritholl commented 5 years ago

When running python with the -W always flag, it appears many warnings are issued thousands of times. The most frequent ones:

     11 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:1364: DeprecationWarning: 'get_proj_vectors_dask' is deprecated, please use 'get_proj_vectors' with the 'chunks' keyword argument specified.
     11 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:916: PendingDeprecationWarning: 'x_size' is deprecated, use 'width' instead.
     11 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:921: PendingDeprecationWarning: 'y_size' is deprecated, use 'height' instead.
     11   warnings.warn("'x_size' is deprecated, use 'width' instead.", PendingDeprecationWarning)
     11   warnings.warn("'y_size' is deprecated, use 'height' instead.", PendingDeprecationWarning)
     13 [DEBUG/MainProcess] worker exiting after 3 tasks
     22   "'get_lonlats' with the 'chunks' keyword argument specified.", DeprecationWarning)
     22 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/geometry.py:1501: DeprecationWarning: 'get_lonlats_dask' is deprecated, please use 'get_lonlats' with the 'chunks' keyword argument specified.
     26 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/importlib/_bootstrap.py:219: ImportWarning: can't resolve package from __spec__ or __package__, falling back on __name__ and __path__
     26   return f(*args, **kwds)
     32   **kwargs))
     32 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/satpy/readers/seviri_base.py:286: PendingDeprecationWarning: xarray.ufuncs will be deprecated when xarray no longer supports versions of numpy older than v1.17. Instead, use numpy ufuncs directly.
     32 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/xarray/core/dataarray.py:1961: PendingDeprecationWarning: xarray.ufuncs will be deprecated when xarray no longer supports versions of numpy older than v1.17. Instead, use numpy ufuncs directly.
     32   xu.log((1.0 / data) * C1 * wavenumber ** 3 + 1.0))
     35 [DEBUG/MainProcess] worker exiting after 0 tasks
     38 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:1055: RuntimeWarning: Mean of empty slice
     38   result = {k: np.nanmean(v) for k, v in result.items()}
     56 [DEBUG/MainProcess] worker got sentinel -- exiting
    177 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/filters.py:697: RuntimeWarning: invalid value encountered in less_equal
    177   res = lat <= thres[self.line]
   2770   cth = np.nanmean(cthmargin)
   2770 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/fogpy/algorithms.py:692: RuntimeWarning: Mean of empty slice
   4765 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/scipy/optimize/_minimize.py:517: RuntimeWarning: Method BFGS cannot handle constraints nor bounds.
   4765   RuntimeWarning)
   5793 /media/nas/x21324/miniconda3/envs/py37d/lib/python3.7/site-packages/pyresample/spherical.py:123: RuntimeWarning: invalid value encountered in true_divide
   5793   self.cart /= np.sqrt(np.einsum('...i, ...i', self.cart, self.cart))

I don't know if this is a problem.

gerritholl commented 5 years ago

The code causing the numerous BFGS warnings appears to be:

https://github.com/gerritholl/fogpy/blob/2c9aa9436e4924f55879d52a9de14817d370df5a/fogpy/lowwatercloud.py#L481-L488

gerritholl commented 5 years ago

The invalid value encountered in true_divide warning is in pyresample and triggered because the eurol projection extends to areas not covered by the EUROPE-B area that the NWCSAF files cover.

gerritholl commented 5 years ago

The RuntimeWarning: Method BFGS cannot handle constraints nor bounds is one that @m4sth0 has seen before.

gerritholl commented 5 years ago

The RuntimeWarning: Mean of empty slice is caused by #10.