Closed patikit closed 3 months ago
Hi, can you provide your name list and a sample WRF file for your issue so I can try to replicate it? Thanks
Sure, thank you for the fast response! This is the name list: By sample WRF file you mean the outputfile of WRF, right?
[case] case_name = "wrf4palm_Berlin_summer_dynamic", max_pool = 4, geostrophic = "p", ## calculate geostrophic wind using the variable pressure (option "p") or geopotential height (option "z")
[domain] palm_proj = "", # projection of PALM centlat = 52.512, # latitude of domain centre centlon = 13.4063, # longitude of domain centre nx = 499, # number of grid points along x-axis ny = 499, # number of grid points along y-axis nz = 200, # number of grid points along z-axis dx = 15.0, # number of grid points along x-axis dy = 15.0, # number of grid points along y-axis dz = 15.0, # number of grid points along z-axis z_origin = 0.0, # elevated mean grid position (elevated terrain)
[stretch] dz_stretch_factor = 1.05, # stretch factor for a vertically stretched grid
dz_stretch_level = 1200.0, # Height level above which the grid is to be stretched vertically (in m)
dz_max = 40.0, # allowed maximum vertical grid spacing (in m)
[wrf] wrf_path = "./wrf_output/", wrf_output = "wrfout_d02_2022-07-19_00:00:00",
interp_mode = "linear",
start_year = 2022, start_month = 7, start_day = 19, start_hour = 0,
end_year = 2022, end_month = 7, end_day = 21, end_hour = 1,
dynamic_ts = 3600.0, # PALM dynamic driver update frequency
[soil]
dz_soil = 0.01, 0.02, 0.04, 0.06, 0.14, 0.26, 0.54, 1.86, msoil = 0.185, 0.184, 0.183, 0.181, 0.183 , 0.186, 0.220, 0.277,
By sample WRF file you mean the outputfile of WRF, right?
Yes, that's right. Only two to three slices of time would be enough.
I send you this via e-mail. Because the file type is not supported by github.
Thanks for sending the file. I ran a test with your namelist and WRF output, and WRF4PALM finished okay. I built a new wrf4palm python environment using the yml file provided. I think this could have something to do with your WRF4PALM python environment, as stated in issue #37.
It could be a compatibility issue between packages. I'm not sure why this happened - it's just Python being a bit annoying. If any of the packages were updated by any chance, you will want to build the python environment again. I have updated a clean yml file after this commit 416cddb.
Try to build a new wrf4palm environment from the yml file by using:
conda env create -f wrf4palm_env.yml
You can also edit your local yml file directly by removing the dependencies. Use this command in vim (:%s/=[^=]\+\zs=.*//
), and try the conda command.
Let me know if this doesn't work.
Cheers
Dear Donqi, thank you for your help! I appreciate it a lot. I used exactly the environment you updated. After trying it again this error occured which seems that something is not compatible.
(wrf4palm) [rv0544@hkn1990 gesamtBerlin]$ python3 run_config_wrf4palm.py namelist.wrf4palm
Traceback (most recent call last):
File "/hkfs/home/project/hk-project-subsuhi/rv0544/WRF4PALM-Berlin-Big/gesamtBerlin/run_config_wrf4palm.py", line 25, in
After that I imported salem, and tried it again: The result was the following:
python3 run_config_wrf4palm.py namelist.wrf4palm
/home/hk-project-subsuhi/rv0544/miniconda3/envs/wrf4palm/lib/python3.9/site-packages/pandas/core/arrays/masked.py:60: UserWarning: Pandas requires version '1.3.6' or newer of 'bottleneck' (version '1.3.2' currently installed).
from pandas.core import (
Reading WRF
/home/hk-project-subsuhi/rv0544/miniconda3/envs/wrf4palm/lib/python3.9/site-packages/xarray/core/utils.py:494: FutureWarning: The return type of Dataset.dims
will be changed to return a set of dimension names in future, in order to be more consistent with DataArray.dims
. To access a mapping from dimension names to lengths, please use Dataset.sizes
.
warnings.warn(
cfg file is saved: wrf4palm_Berlin_big_domain_summer_dynamic
Start horizontal interpolation
Traceback (most recent call last):
File "/hkfs/home/project/hk-project-subsuhi/rv0544/WRF4PALM-Berlin-Big/gesamtBerlin/run_config_wrf4palm.py", line 291, in
So something does not seem to be compatible... Maybe it is also an issue of the HPC? Thank you in advance ! Kind regards, Patricia
Yes, this looks like a compatibility problem. The wrf4palm environment should have pandas=1.3.3, but your error message shows 1.3.2. This is odd.
I'm not a Python expert, but can you check your output of conda list
to ensure that the package versions match those in the yml file? Feel free to share the list here.
This is the list
(wrf4palm) [rv0544@hkn1993 ~]$ conda list
_libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge affine 2.3.0 py_0 conda-forge alsa-lib 1.2.3.2 h166bdaf_0 conda-forge anyio 3.3.2 py39hf3d152e_0 conda-forge appdirs 1.4.4 pyh9f0ad1d_0 conda-forge argon2-cffi 20.1.0 py39h3811e60_2 conda-forge async_generator 1.10 py_0 conda-forge attrs 21.2.0 pyhd8ed1ab_0 conda-forge babel 2.9.1 pyh44b312d_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge beautifulsoup4 4.10.0 pypi_0 pypi bleach 4.1.0 pyhd8ed1ab_0 conda-forge blosc 1.21.0 h9c3ff4c_0 conda-forge bokeh 2.4.0 py39hf3d152e_0 conda-forge boost-cpp 1.74.0 h312852a_4 conda-forge bottleneck 1.3.2 py39hce5d2b2_5 conda-forge brotli 1.0.9 h166bdaf_7 conda-forge brotli-bin 1.0.9 h166bdaf_7 conda-forge brotlipy 0.7.0 py39hb9d737c_1004 conda-forge brunsli 0.1 h9c3ff4c_0 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge c-ares 1.17.2 h7f98852_0 conda-forge ca-certificates 2021.5.30 ha878542_0 conda-forge cairo 1.16.0 h6cf1ce9_1008 conda-forge cartopy 0.20.0 py39ha48bd0d_2 conda-forge certifi 2021.5.30 py39hf3d152e_0 conda-forge cffi 1.14.6 py39h4bc2ebd_2 conda-forge cfitsio 3.470 hb418390_7 conda-forge cftime 1.5.1.1 py39hce5d2b2_1 conda-forge chardet 4.0.0 py39hf3d152e_3 conda-forge charls 2.2.0 h9c3ff4c_0 conda-forge charset-normalizer 2.0.0 pyhd8ed1ab_0 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge click-plugins 1.1.1 py_0 conda-forge cligj 0.7.2 pyhd8ed1ab_1 conda-forge cloudpickle 2.0.0 pyhd8ed1ab_0 conda-forge cryptography 3.4.7 py39hbca0aa6_0 conda-forge curl 7.79.1 h2574ce0_1 conda-forge cycler 0.10.0 py_2 conda-forge cytoolz 0.11.0 py39h3811e60_3 conda-forge dask 2021.9.1 pyhd8ed1ab_0 conda-forge dask-core 2021.9.1 pyhd8ed1ab_0 conda-forge dbus 1.13.6 h48d8840_2 conda-forge debugpy 1.4.1 py39he80948d_1 conda-forge decorator 5.1.0 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge descartes 1.1.0 py_4 conda-forge dill 0.3.4 pypi_0 pypi distributed 2021.9.1 py39hf3d152e_0 conda-forge entrypoints 0.3 pyhd8ed1ab_1003 conda-forge expat 2.4.1 h9c3ff4c_0 conda-forge fiona 1.8.20 py39h427c1bf_2 conda-forge font-ttf-dejavu-sans-mono 2.37 hab24e00_0 conda-forge font-ttf-inconsolata 3.000 h77eed37_0 conda-forge font-ttf-source-code-pro 2.038 h77eed37_0 conda-forge font-ttf-ubuntu 0.83 h77eed37_2 conda-forge fontconfig 2.13.1 hba837de_1005 conda-forge fonts-conda-ecosystem 1 0 conda-forge fonts-conda-forge 1 0 conda-forge freetype 2.10.4 h0708190_1 conda-forge freexl 1.0.6 h7f98852_0 conda-forge fsspec 2021.10.0 pyhd8ed1ab_0 conda-forge gdal 3.3.2 py39h218ed2d_4 conda-forge geopandas 0.9.0 pyhd8ed1ab_0 conda-forge geopandas-base 0.9.0 pyha770c72_1 conda-forge geos 3.9.1 h9c3ff4c_2 conda-forge geotiff 1.7.0 hcfb7246_3 conda-forge gettext 0.19.8.1 h73d1719_1008 conda-forge giflib 5.2.1 h36c2ea0_2 conda-forge glib 2.68.4 h9c3ff4c_1 conda-forge glib-tools 2.68.4 h9c3ff4c_1 conda-forge gst-plugins-base 1.18.5 hf529b03_0 conda-forge gstreamer 1.18.5 h76c114f_0 conda-forge hdf4 4.2.15 h10796ff_3 conda-forge hdf5 1.12.1 nompi_h2750804_101 conda-forge heapdict 1.0.1 py_0 conda-forge icu 68.1 h58526e2_0 conda-forge idna 3.1 pyhd3deb0d_0 conda-forge imagecodecs 2021.7.30 py39h44211f0_1 conda-forge imageio 2.9.0 py_0 conda-forge importlib-metadata 4.8.1 py39hf3d152e_1 conda-forge importlib_metadata 4.8.1 hd8ed1ab_1 conda-forge importlib_resources 5.2.2 pyhd8ed1ab_0 conda-forge ipykernel 6.2.0 py39hef51801_0 conda-forge ipython 7.28.0 py39hef51801_0 conda-forge ipython_genutils 0.2.0 pyhd8ed1ab_1 conda-forge jbig 2.1 h7f98852_2003 conda-forge jedi 0.18.0 py39hf3d152e_3 conda-forge jinja2 3.0.1 pyhd8ed1ab_0 conda-forge joblib 1.0.1 pyhd8ed1ab_0 conda-forge jpeg 9d h36c2ea0_0 conda-forge json-c 0.15 h98cffda_0 conda-forge json5 0.9.5 pyh9f0ad1d_0 conda-forge jsonschema 4.0.1 pyhd8ed1ab_0 conda-forge jupyter_client 7.0.5 pyhd8ed1ab_0 conda-forge jupyter_core 4.8.1 py39hf3d152e_0 conda-forge jupyter_server 1.11.0 pyhd8ed1ab_0 conda-forge jupyterlab 3.1.14 pyhd8ed1ab_0 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_server 2.8.2 pyhd8ed1ab_0 conda-forge jxrlib 1.1 h7f98852_2 conda-forge kealib 1.4.14 h87e4c3c_3 conda-forge keyutils 1.6.1 h166bdaf_0 conda-forge kiwisolver 1.3.2 py39h1a9c180_1 conda-forge krb5 1.19.2 h3790be6_4 conda-forge lcms2 2.12 hddcbb42_0 conda-forge ld_impl_linux-64 2.36.1 hea4e1c9_2 conda-forge lerc 2.2.1 h9c3ff4c_0 conda-forge libaec 1.0.6 h9c3ff4c_0 conda-forge libblas 3.9.0 11_linux64_openblas conda-forge libbrotlicommon 1.0.9 h166bdaf_7 conda-forge libbrotlidec 1.0.9 h166bdaf_7 conda-forge libbrotlienc 1.0.9 h166bdaf_7 conda-forge libcblas 3.9.0 11_linux64_openblas conda-forge libclang 11.1.0 default_ha53f305_1 conda-forge libcurl 7.79.1 h2574ce0_1 conda-forge libdap4 3.20.6 hd7c4107_2 conda-forge libdeflate 1.7 h7f98852_5 conda-forge libedit 3.1.20191231 he28a2e2_2 conda-forge libev 4.33 h516909a_1 conda-forge libevent 2.1.10 h9b69904_4 conda-forge libffi 3.4.2 h7f98852_5 conda-forge libgcc-ng 11.2.0 h1d223b6_16 conda-forge libgdal 3.3.2 h9c9eb65_4 conda-forge libgfortran-ng 11.2.0 h69a702a_16 conda-forge libgfortran5 11.2.0 h5c6108e_16 conda-forge libglib 2.68.4 h174f98d_1 conda-forge libgomp 11.2.0 h1d223b6_16 conda-forge libiconv 1.16 h516909a_0 conda-forge libkml 1.3.0 h238a007_1014 conda-forge liblapack 3.9.0 11_linux64_openblas conda-forge libllvm11 11.1.0 hf817b99_3 conda-forge libnetcdf 4.8.1 nompi_hb3fd0d9_101 conda-forge libnghttp2 1.43.0 h812cca2_1 conda-forge libnsl 2.0.0 h7f98852_0 conda-forge libogg 1.3.4 h7f98852_1 conda-forge libopenblas 0.3.17 pthreads_h8fe5266_1 conda-forge libopus 1.3.1 h7f98852_1 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libpq 13.3 hd57d9b9_3 conda-forge librttopo 1.1.0 h1185371_6 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libspatialindex 1.9.3 h9c3ff4c_4 conda-forge libspatialite 5.0.1 h8796b1e_9 conda-forge libssh2 1.10.0 ha56f1ee_2 conda-forge libstdcxx-ng 11.2.0 he4da1e4_16 conda-forge libtiff 4.3.0 hf544144_1 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libvorbis 1.3.7 h9c3ff4c_0 conda-forge libwebp-base 1.2.1 h7f98852_0 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxkbcommon 1.0.3 he3ba5ed_0 conda-forge libxml2 2.9.12 h72842e0_0 conda-forge libzip 1.8.0 h4de3113_1 conda-forge libzlib 1.2.11 h166bdaf_1014 conda-forge libzopfli 1.0.3 h9c3ff4c_0 conda-forge locket 0.2.0 py_2 conda-forge lz4-c 1.9.3 h9c3ff4c_1 conda-forge mapclassify 2.4.3 pyhd8ed1ab_0 conda-forge markupsafe 2.0.1 py39h3811e60_1 conda-forge matplotlib 3.4.3 py39hf3d152e_2 conda-forge matplotlib-base 3.4.3 py39h2fa2bec_2 conda-forge matplotlib-inline 0.1.3 pyhd8ed1ab_0 conda-forge metpy 1.1.0 pyhd8ed1ab_0 conda-forge mistune 0.8.4 pyh1a96a4e_1006 conda-forge msgpack-python 1.0.2 py39h1a9c180_2 conda-forge multiprocess 0.70.12.2 pypi_0 pypi munch 2.5.0 py_0 conda-forge mysql-common 8.0.25 ha770c72_3 conda-forge mysql-libs 8.0.25 hfa10184_3 conda-forge nbclassic 0.3.2 pyhd8ed1ab_0 conda-forge nbclient 0.5.4 pyhd8ed1ab_0 conda-forge nbconvert 6.2.0 py39hf3d152e_0 conda-forge nbformat 5.1.3 pyhd8ed1ab_0 conda-forge ncurses 6.2 h58526e2_4 conda-forge nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge netcdf4 1.5.7 nompi_py39h64b754b_103 conda-forge networkx 2.3 py_0 conda-forge notebook 6.4.4 pyha770c72_0 conda-forge nspr 4.30 h9c3ff4c_0 conda-forge nss 3.69 hb5efdd6_1 conda-forge numpy 1.22.4 pypi_0 pypi olefile 0.46 pyh9f0ad1d_1 conda-forge openjpeg 2.4.0 hb52868f_1 conda-forge openssl 1.1.1l h7f98852_0 conda-forge packaging 24.1 pypi_0 pypi pandas 2.2.2 pypi_0 pypi pandoc 2.14.2 h7f98852_0 conda-forge pandocfilters 1.5.0 pyhd8ed1ab_0 conda-forge parso 0.8.2 pyhd8ed1ab_0 conda-forge partd 1.2.0 pyhd8ed1ab_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pexpect 4.8.0 pyh1a96a4e_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.3.2 py39ha612740_0 conda-forge pint 0.17 pyhd8ed1ab_1 conda-forge pip 21.2.4 pyhd8ed1ab_0 conda-forge pixman 0.40.0 h36c2ea0_0 conda-forge pooch 1.5.1 pyhd8ed1ab_0 conda-forge poppler 21.09.0 ha39eefc_3 conda-forge poppler-data 0.4.11 hd8ed1ab_0 conda-forge postgresql 13.3 h2510834_3 conda-forge proj 8.1.1 h277dcde_2 conda-forge prometheus_client 0.11.0 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.20 pyha770c72_0 conda-forge protobuf 3.18.1 pypi_0 pypi psutil 5.8.0 py39h3811e60_2 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pygments 2.10.0 pyhd8ed1ab_0 conda-forge pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge pyparsing 2.4.7 pyhd8ed1ab_1 conda-forge pyproj 3.2.1 py39ha81a305_2 conda-forge pyqt 5.12.3 py39hf3d152e_8 conda-forge pyqt-impl 5.12.3 py39hde8b62d_8 conda-forge pyqt5-sip 4.19.18 py39he80948d_8 conda-forge pyqtchart 5.12 py39h0fcd23e_8 conda-forge pyqtwebengine 5.12.1 py39h0fcd23e_8 conda-forge pyrsistent 0.17.3 py39h3811e60_3 conda-forge pyshp 2.1.3 pyh44b312d_0 conda-forge pysocks 1.7.1 pyha2e5f31_6 conda-forge python 3.9.7 hb7a2778_3_cpython conda-forge python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge python_abi 3.9 4_cp39 conda-forge pytz 2021.1 pyhd8ed1ab_0 conda-forge pywavelets 1.1.1 py39hce5d2b2_4 conda-forge pyyaml 5.4.1 py39h3811e60_1 conda-forge pyzmq 22.3.0 py39headdf64_2 conda-forge qt 5.12.9 hda022c4_4 conda-forge rasterio 1.2.9 py39hb37810a_2 conda-forge readline 8.1 h46c0cb4_0 conda-forge requests 2.26.0 pyhd8ed1ab_1 conda-forge requests-unixsocket 0.2.0 py_0 conda-forge rioxarray 0.7.1 pyhd8ed1ab_0 conda-forge rtree 0.9.7 py39hb102c33_3 conda-forge salem 0.3.5 pyhd8ed1ab_0 conda-forge scikit-image 0.18.3 py39hde0f152_1 conda-forge scikit-learn 1.0.2 py39h4dfa638_0 conda-forge scipy 1.7.1 py39hee8e79c_0 conda-forge send2trash 1.8.0 pyhd8ed1ab_0 conda-forge setuptools 58.0.4 py39hf3d152e_2 conda-forge shapely 1.7.1 py39ha61afbd_5 conda-forge siphon 0.9 pypi_0 pypi six 1.16.0 pyh6c4a22f_0 conda-forge snappy 1.1.8 he1b5a44_3 conda-forge sniffio 1.2.0 py39hf3d152e_3 conda-forge snuggs 1.4.7 py_0 conda-forge sortedcontainers 2.4.0 pyhd8ed1ab_0 conda-forge soupsieve 2.2.1 pypi_0 pypi sqlite 3.36.0 h9cd32fc_2 conda-forge tblib 1.7.0 pyhd8ed1ab_0 conda-forge terminado 0.12.1 py39hf3d152e_1 conda-forge testpath 0.5.0 pyhd8ed1ab_0 conda-forge threadpoolctl 3.0.0 pyh8a188c0_0 conda-forge tifffile 2021.8.30 pyhd8ed1ab_0 conda-forge tiledb 2.3.4 he87e0bf_0 conda-forge tk 8.6.11 h27826a3_1 conda-forge toolz 0.11.1 py_0 conda-forge tornado 6.1 py39hb9d737c_3 conda-forge tqdm 4.62.3 pypi_0 pypi traitlets 5.1.0 pyhd8ed1ab_0 conda-forge typing_extensions 3.10.0.2 pyha770c72_0 conda-forge tzcode 2021a h7f98852_2 conda-forge tzdata 2024.1 pypi_0 pypi urllib3 1.26.7 pyhd8ed1ab_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 pyhd8ed1ab_2 conda-forge websocket-client 0.57.0 py39hf3d152e_6 conda-forge wheel 0.37.0 pyhd8ed1ab_1 conda-forge xarray 2023.12.0 pypi_0 pypi xerces-c 3.2.3 h9d8b166_3 conda-forge xorg-kbproto 1.0.7 h7f98852_1002 conda-forge xorg-libice 1.0.10 h7f98852_0 conda-forge xorg-libsm 1.2.3 hd9c2040_1000 conda-forge xorg-libx11 1.7.2 h7f98852_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xorg-libxext 1.3.4 h7f98852_1 conda-forge xorg-libxrender 0.9.10 h7f98852_1003 conda-forge xorg-renderproto 0.11.1 h7f98852_1002 conda-forge xorg-xextproto 7.3.0 h7f98852_1002 conda-forge xorg-xproto 7.0.31 h7f98852_1007 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h7f98852_2 conda-forge zeromq 4.3.4 h9c3ff4c_1 conda-forge zfp 0.5.5 h9c3ff4c_8 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.5.1 pyhd8ed1ab_0 conda-forge zlib 1.2.11 h166bdaf_1014 conda-forge zstd 1.5.0 ha95c52a_0 conda-forge
I couldn't go through all the packages, but your xarray version is 2023.12.0
, which is more up-to-date than the one compatible with WRF4PALM (0.19.0
).
This means WRF4PALM will need an update... but given my current workload, I don't think this update will come anytime soon.
You can either try to reinstall xarray, or remove your current wrf4palm Python environment entirely and start from scratch again. Hope this helps.
Dear Dongqi, I created the wrf4palm environment once more and forced it to really take all the packages given in the yml file. Then I installed salem manually (because it still made some problems). Now, it created the dynamic driver. Actually I did not changed anything compared to what I have done before. Anyhow, I am verry happy that it worked. And thank you very much for your help. I appreciate it a lot.
Cheers, Patricia
Thanks Patricia. I'll close this issue now. The compatibility of Python environments has been an issue for many projects, so I'll try to apply a fix in the next update (I don't know when this will happen, though :).
Kind regards and thank you in advance!
Originally posted by @patikit in https://github.com/dongqi-DQ/WRF4PALM/issues/37#issuecomment-2228102193