Closed zxdawn closed 5 years ago
conservative
works on intensive properties (density-like fields). If you have extensive properties (mass-like), divide by cell area first.
Alternatively, nearest_d2s
conserves extensive properties directly.
Thank you for your advice! nearest_d2s
works fine.
By the way, have you tried to calculate the cell area? I can't find package in python to calculate it directly. I used to calculate it by cal_area.m with pymatbridge.
nearest_d2s
works fine.
That's good to know! I would appreciate a reproducible example (code + sample data). nearest_d2s
is so rarely used so it is nice to show a use case.
have you tried to calculate the cell area?
See this gist for now: https://gist.github.com/JiaweiZhuang/fc2f860133174523051260817e75a99d
I've rewritten the regridding prat and you can check it.
It looks like nearest_d2s
and conservative
get the similar result, while bilinear
keeps the original distribution.
Thanks! Is the data file also available?
Maybe we can compare the result with 'True' value directly.
Since all methods are almost same for 0.1° 0.1°.
We can regird data to 0.1° 0.1° first and use binned_statistic_2d
to sum for 1° * 1°.
Finish. You can check here.
nearest_d2s
works fine.That's good to know! I would appreciate a reproducible example (code + sample data).
nearest_d2s
is so rarely used so it is nice to show a use case.have you tried to calculate the cell area?
See this gist for now: https://gist.github.com/JiaweiZhuang/fc2f860133174523051260817e75a99d
I encountered some problems when installing xesmf from area branch:
It's successful when xesmf is installed on my machine:
pip install --upgrade git+ssh://git@github.com/JiaweiZhuang/xESMF.git@area
Branch area set up to track remote branch area from origin.
Switched to a new branch 'area'
Requirement already satisfied, skipping upgrade: esmpy in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (7.1.0.dev0)
Requirement already satisfied, skipping upgrade: xarray in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (0.11.0)
Requirement already satisfied, skipping upgrade: numpy in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (1.15.4)
Requirement already satisfied, skipping upgrade: scipy in ./Software/anaconda3/lib/python3.6/site-packages (from xesmf==0.1.1) (1.1.0)
Requirement already satisfied, skipping upgrade: pandas>=0.19.2 in ./Software/anaconda3/lib/python3.6/site-packages (from xarray->xesmf==0.1.1) (0.23.4)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in ./Software/anaconda3/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2.6.1)
Requirement already satisfied, skipping upgrade: pytz>=2011k in ./Software/anaconda3/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2017.3)
Requirement already satisfied, skipping upgrade: six>=1.5 in ./Software/anaconda3/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas>=0.19.2->xarray->xesmf==0.1.1) (1.11.0)
Building wheels for collected packages: xesmf
Running setup.py bdist_wheel for xesmf ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-asry4otw/wheels/86/a4/73/9f5efeec79d3e63db519ba88566cc5b5f469dae1469a2d7e9b
Successfully built xesmf
Installing collected packages: xesmf
Successfully installed xesmf-0.1.1
However, I got error when installing it on hpc:
pip install --upgrade git+ssh://git@github.com/JiaweiZhuang/xESMF.git@area
Collecting git+ssh://git@github.com/JiaweiZhuang/xESMF.git@area
Cloning ssh://git@github.com/JiaweiZhuang/xESMF.git (to revision area) to /tmp/pip-req-build-k16i3mzq
Branch 'area' set up to track remote branch 'area' from 'origin'.
Switched to a new branch 'area'
Collecting esmpy (from xesmf==0.1.1)
Could not find a version that satisfies the requirement esmpy (from xesmf==0.1.1) (from versions: )
No matching distribution found for esmpy (from xesmf==0.1.1)
I checked the installed list by conda list
on hpc and found xesmf
:
xesmf 0.1.1 <pip>
But, I couldn't remove it by conda remove --force xesmf
or pip uninstall xesmf
:
Solving environment: done
PackagesNotFoundError: The following packages are missing from the target environment:
- xesmf
Skipping xesmf as it is not installed.
@zxdawn Install ESMPy with conda conda install -c conda-forge esmpy
and then install xESMF with pip. ESMPy is only installable from conda so pip cannot resolve the dependency.
conda install -c conda-forge esmpy
Actually, it has been installed:
Solving environment: done
# All requested packages already installed.
However, I tried to install xesmf again, it worked:
Collecting git+ssh://git@github.com/JiaweiZhuang/xESMF.git@area
Cloning ssh://git@github.com/JiaweiZhuang/xESMF.git (to revision area) to /tmp/pip-req-build-ufistfyw
Branch 'area' set up to track remote branch 'area' from 'origin'.
Switched to a new branch 'area'
Requirement already satisfied, skipping upgrade: esmpy in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (7.1.0.dev0)
Requirement already satisfied, skipping upgrade: xarray in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (0.11.0)
Requirement already satisfied, skipping upgrade: numpy in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (1.15.4)
Requirement already satisfied, skipping upgrade: scipy in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xesmf==0.1.1) (1.2.0)
Requirement already satisfied, skipping upgrade: pandas>=0.19.2 in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from xarray->xesmf==0.1.1) (0.24.0)
Requirement already satisfied, skipping upgrade: pytz>=2011k in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2018.9)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.5.0 in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from pandas>=0.19.2->xarray->xesmf==0.1.1) (2.7.5)
Requirement already satisfied, skipping upgrade: six>=1.5 in /chenq3/zhangxin/anaconda3/envs/python36/lib/python3.6/site-packages (from python-dateutil>=2.5.0->pandas>=0.19.2->xarray->xesmf==0.1.1) (1.12.0)
Building wheels for collected packages: xesmf
Running setup.py bdist_wheel for xesmf ... done
Stored in directory: /tmp/pip-ephem-wheel-cache-37s9eb5s/wheels/86/a4/73/9f5efeec79d3e63db519ba88566cc5b5f469dae1469a2d7e9b
Successfully built xesmf
Installing collected packages: xesmf
Found existing installation: xesmf 0.1.1
Uninstalling xesmf-0.1.1:
Successfully uninstalled xesmf-0.1.1
Successfully installed xesmf-0.1.1
When I import xesmf
in Python, I got this error:
Traceback (most recent call last):
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/loadESMF.py", line 118, in <module>
mode=ct.RTLD_GLOBAL)
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/libesmf_fullylinked.so: undefined symbol: __netcdf_MOD_nf90_get_var_eightbytereal
Traceback (most recent call last):
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/loadESMF.py", line 118, in <module>
mode=ct.RTLD_GLOBAL)
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/ctypes/__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/libesmf_fullylinked.so: undefined symbol: __netcdf_MOD_nf90_get_var_eightbytereal
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/xesmf/__init__.py", line 1, in <module>
from . import util
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/xesmf/util.py", line 4, in <module>
import ESMF
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/__init__.py", line 70, in <module>
from ESMF.api.esmpymanager import *
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/api/esmpymanager.py", line 11, in <module>
from ESMF.interface.cbindings import *
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/cbindings.py", line 13, in <module>
from ESMF.interface.loadESMF import _ESMF
File "/public/home/zhangxin/bigdata/anaconda3/envs/python36/lib/python3.6/site-packages/ESMF/interface/loadESMF.py", line 121, in <module>
raise ImportError('The ESMF shared library did not load properly.')
ImportError: The ESMF shared library did not load properly.
After export LD_LIBRARY_PATH=''
, it works now.
So, I think it's related to library again .... Thank you for your help :)
Yes ImportError: The ESMF shared library did not load properly.
is an ESMPy problem. Your HPC system probably has a separate ESMF library installed and dynamically linked, so ESMPy finds the incorrect installation of ESMF.
@JiaweiZhuang How about the unit of xe.util.cell_area
? I can't find it in the definition of get_area() and area().
I see ... It's 4*pi*(1^2) https://github.com/JiaweiZhuang/xESMF/blob/3e4610899c2006451f31db723e40688a0877a37b/xesmf/util.py#L104-L107
I want to regrid
total lightning flashcounts
(TL) generated byWRF-Chem
from 12 km to 1 degree.TL
is the sum of flashcounts in each 12 km grid.I've tried xesmf with
bilinear
orconservative
method, but both results fails. You can check notebook1 and notebook2.Do you have any advice of regridding variables like
TL
?