JiaweiZhuang / xESMF

Universal Regridder for Geospatial Data
http://xesmf.readthedocs.io/
MIT License
269 stars 49 forks source link

calling regridder cleans workspace #80

Open artlionel opened 4 years ago

artlionel commented 4 years ago

Hi there,

Many thanks for this function. I am experiencing a strange problem. When I create the regridder, e.g., "regridder = xe.Regridder(ds, ds_out, 'bilinear')" It cleans out my workspace (all variables and definitions) without indicating any error. Do you have any idea what the problem might be?

Thanks, Lionel

JiaweiZhuang commented 4 years ago

Thanks for reporting -- Could you reproduce this problem in Binder? Or is it specific to your computer environment?

artlionel commented 4 years ago

Thanks for the fast reply. I think it is specific to my case. Perhaps something went wrong during the installation of the package. The following lines will cause the issue:

import scipy.io as sio import numpy as np import pandas as pd import matplotlib.pyplot as plt import seaborn as sns import xarray as xr import xesmf as xe

! wget http://ldeo.columbia.edu/~rpa/NOAA_NCDC_ERSST_v3b_SST.nc ds = xr.open_dataset('NOAA_NCDC_ERSST_v3b_SST.nc') ds_out = xr.Dataset({'lat': (['lat'], np.arange(-89.5, -25.5, 1.0)), \ 'lon': (['lon'], np.arange(-179.5, 179.5, 1.0)),}) regridder = xe.Regridder(ds, ds_out, 'bilinear')

andrewpauling commented 3 years ago

I am having the same issue working on the NCAR Cheyenne system. I created a clean environment (after running module purge): conda create -n xesmf_env python=3.7 then ran: conda install -c conda-forge esmpy then: conda install -c conda-forge xesmf conda install -c conda-forge cartopy dask netcdf4

I then tried to regrid from the 1x1 degree global grid (from xe.util.grid_global(1, 1)) to the 4x5 degree grid from xe.util.grid_global(5, 4)

The command failed silently and cleaned my workspace, as reported above. I was not able to reproduce this behaviour in the binder.

andrewpauling commented 3 years ago

I also tried this on my personal desktop (a Mac), with the same intallation process as above, and the xe.regridder command created the weights file, but then cleaned my workspace afterward.

andrewpauling commented 3 years ago

I ran pytest on my local installation and got:

==================================================== test session starts ==================================================== platform darwin -- Python 3.7.9, pytest-6.0.1, py-1.9.0, pluggy-0.13.1 -- /Users/andrewpauling/anaconda3/envs/xesmf_env/bin/python cachedir: .pytest_cache rootdir: /Users/andrewpauling collected 42 items

anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py::test_warn_f_on_array PASSED [ 2%] anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py::test_warn_f_on_grid PASSED [ 4%] anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py::test_warn_lat_range PASSED [ 7%] anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py::test_esmf_grid_with_corner PASSED [ 9%] anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py::test_esmf_build_bilinear PASSED [ 11%] anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py::test_regrid Fatal Python error: Illegal instruction

Current thread 0x00000001084fb5c0 (most recent call first): File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/ESMF/interface/cbindings.py", line 2203 in ESMP_FieldRegridStoreFile File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/ESMF/util/decorators.py", line 52 in new_func File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/ESMF/api/regrid.py", line 153 in init File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/ESMF/util/decorators.py", line 64 in new_func File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/backend.py", line 280 in esmf_regrid_build File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/xesmf/tests/test_backend.py", line 139 in test_regrid File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/python.py", line 180 in pytest_pyfunc_call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 87 in File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/python.py", line 1567 in runtest File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 153 in pytest_runtest_call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 87 in File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 247 in File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 294 in from_call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 247 in call_runtest_hook File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 207 in call_and_report File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 117 in runtestprotocol File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/runner.py", line 100 in pytest_runtest_protocol File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 87 in File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/main.py", line 321 in pytest_runtestloop File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 87 in File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/main.py", line 296 in _main File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/main.py", line 240 in wrap_session File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/main.py", line 289 in pytest_cmdline_main File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/callers.py", line 187 in _multicall File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 87 in File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/manager.py", line 93 in _hookexec File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/pluggy/hooks.py", line 286 in call File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/config/init.py", line 158 in main File "/Users/andrewpauling/anaconda3/envs/xesmf_env/lib/python3.7/site-packages/_pytest/config/init.py", line 180 in console_main File "/Users/andrewpauling/anaconda3/envs/xesmf_env/bin/pytest", line 8 in Illegal instruction: 4