search

ESCOMP / CTSM

Community Terrestrial Systems Model (includes the Community Land Model of CESM)
http://www.cesm.ucar.edu/models/cesm2.0/land/
Other
309 stars 313 forks source link

Subset_data point creates solar files with integer time values with older xarray -- needs conda env update #1974

Closed olyson closed 9 months ago

olyson commented 1 year ago

Brief summary of bug

The values of time in the solar files created by subset_data are integers

General bug information

CTSM version you are using: ctsm5.1.dev118

Does this bug cause significantly incorrect results in the model's science? Model doesn't run

Configurations affected: --res CLM_USRDAT --compset I2000Clm51BgcCrop --run-unsupported --user-mods-dirs /glade/scratch/oleson/SITE_TEST/user_mods/

Details of bug

./subset_data point --lat 40.0 --lon 270.0 --site SITE_TEST --create-surface --create-datm --datm-syr 2000 --datm-eyr 2000 --create-user-mods --outdir /glade/scratch/oleson/SITE_TEST

It completed without errors. However, the time values in the Solar file are integers despite the fact that the variable is float, e.g.,

ncdump -v time /glade/scratch/oleson/SITE_TEST/datmdata/clmforc.GSWP3.c2011.0.5x0.5.Solr.SITE_TEST.2000-01.nc

time = 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30 ;

The precip and TPQW files appear to be normal.

Important output or errors that show the problem

The model crashes with ERROR: (shr_stream_verifyTCoord) ERROR: elapsed seconds must be increasing

ekluzek commented 1 year ago

@olyson I've tried replicating this and haven't been able to.

What python environment are you using when you run this? I think that might be a key here. Let me know how you are running it and on what machine. We want this to be consistent for different users and different machines. One thing we might need to do is to ensure users have certain versions of packages (or don't have specific versions known to fail).

When I did the following in a ctsm5.1.dev118 clone

./py_env_create
conda activate ctsm_pylib
cd tools/site_and_regional
./subset_data point --lat 40.0 --lon 270.0 --site SITE_TEST --create-surface --create-datm --datm-syr 2000 --datm-eyr 2000 --create-user-mods --outdir SITE_TEST

time for the Solr file seemed to be fine.

ekluzek commented 1 year ago

I also didn't see anything where the Solr file was treated differently. The NetCDF format of the parent files is all classic format so I didn't see anything different between the files to explain why this might happen. The values of time is different between the different files, but it's float for all three of the parent files.

olyson commented 1 year ago

Thanks for looking at this. I tried this again and got the same result.

On cheyenne: cd /glade/work/oleson/ctsm5.1.dev118/tools/site_and_regional ../../py_env_create conda activate ctsm_pylib ./subset_data point --lat 40.0 --lon 270.0 --site SITE_TEST --create-surface --create-datm --datm-syr 2000 --datm-eyr 2000 --create-user-mods --outdir SITE_TEST

ncdump -v time SITE_TEST/datmdata/clmforc.GSWP3.c2011.0.5x0.5.Solr.SITE_TEST.2000-01.nc

time = 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 5, 5, 5, 5, 6, 6, 6, 6, 6, 6, 6, 6, 7, 7, 7, 7, 7, 7, 7, 7, 8, 8, 8, 8, 8, 8, 8, 8, 9, 9, 9, 9, 9, 9, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 11, 11, 11, 11, 11, 11, 11, 11, 12, 12, 12, 12, 12, 12, 12, 12, 13, 13, 13, 13, 13, 13, 13, 13, 14, 14, 14, 14, 14, 14, 14, 14, 15, 15, 15, 15, 15, 15, 15, 15, 16, 16, 16, 16, 16, 16, 16, 16, 17, 17, 17, 17, 17, 17, 17, 17, 18, 18, 18, 18, 18, 18, 18, 18, 19, 19, 19, 19, 19, 19, 19, 19, 20, 20, 20, 20, 20, 20, 20, 20, 21, 21, 21, 21, 21, 21, 21, 21, 22, 22, 22, 22, 22, 22, 22, 22, 23, 23, 23, 23, 23, 23, 23, 23, 24, 24, 24, 24, 24, 24, 24, 24, 25, 25, 25, 25, 25, 25, 25, 25, 26, 26, 26, 26, 26, 26, 26, 26, 27, 27, 27, 27, 27, 27, 27, 27, 28, 28, 28, 28, 28, 28, 28, 28, 29, 29, 29, 29, 29, 29, 29, 29, 30, 30, 30, 30, 30, 30, 30, 30 ;

olyson commented 1 year ago

Currently Loaded Modules: 1) ncarenv/1.3 3) ncarcompilers/0.5.0 5) netcdf/4.8.1 7) nco/4.7.9 9) R/4.1.2 11) git/2.33.1 2) intel/19.1.1 4) mpt/2.25 6) ncview/2.1.7 8) ncl/6.6.2 10) python/3.7.9

ekluzek commented 1 year ago

Thanks for the extra info @olyson. Hmmm, this is strange. Can you do a which on conda for me? Your modules don't include conda, so you must be pointing to it by hand, or using a local version or something.

Also, besides "which conda", do a

conda list

So we can see what python modules you are running with. And just to be exhaustive we should see your env variables. So do a...

env

Maybe there are env variables that dictate netCDF and/or python behavior?

And what shell are you using?

And let me know what login node you are on in cheyenne. I ran on cheyenne1, so make sure you try there if you didn't already. You might try casper and a different login node just to check...

olyson commented 1 year ago

which conda:

conda: aliased to source /glade/work/oleson/miniconda3/etc/profile.d/conda.csh

conda list:

packages in environment at /glade/work/oleson/miniconda3:

Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 1_gnu conda-forge aiohttp 3.7.4 py37h5e8e339_0 conda-forge argon2-cffi 20.1.0 py37h5e8e339_2 conda-forge asaptools 0.6.2.post46 pypi_0 pypi async-timeout 3.0.1 py_1000 conda-forge async_generator 1.10 py_0 conda-forge attrs 21.2.0 pyhd8ed1ab_0 conda-forge backcall 0.2.0 pyh9f0ad1d_0 conda-forge backports 1.0 py_2 conda-forge backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge bash 4.4.18 h67949de_1000 conda-forge bash_kernel 0.7.2 py37_1 conda-forge bleach 3.3.0 pyh44b312d_0 conda-forge bokeh 2.3.2 py37h89c1867_0 conda-forge brotlipy 0.7.0 py37h5e8e339_1001 conda-forge bzip2 1.0.8 h7f98852_4 conda-forge ca-certificates 2020.12.5 ha878542_0 conda-forge certifi 2020.12.5 py37h89c1867_1 conda-forge cf_units 2.0.1 py37h3010b51_1002 conda-forge cffi 1.13.2 py37h8022711_0 conda-forge cftime 1.4.1 py37h902c9e0_0 conda-forge chardet 4.0.0 py37h89c1867_1 conda-forge click 7.1.2 pyh9f0ad1d_0 conda-forge cloudpickle 1.6.0 py_0 conda-forge conda 4.10.1 py37h89c1867_0 conda-forge conda-package-handling 1.7.3 py37h5e8e339_0 conda-forge cryptography 3.4.7 py37h5d9358c_0 conda-forge cytoolz 0.11.0 py37h5e8e339_3 conda-forge dask 2.3.0 py_0 conda-forge dask-core 2.3.0 py_0 conda-forge dask-labextension 1.1.0 py_0 conda-forge decorator 5.0.7 pyhd8ed1ab_0 conda-forge defusedxml 0.7.1 pyhd8ed1ab_0 conda-forge distributed 2.3.2 py_1 conda-forge entrypoints 0.3 pyhd8ed1ab_1003 conda-forge expat 2.3.0 h9c3ff4c_0 conda-forge freetype 2.10.4 h0708190_1 conda-forge fsspec 2021.4.0 pyhd8ed1ab_0 conda-forge heapdict 1.0.1 py_0 conda-forge icu 64.2 he1b5a44_1 conda-forge idna 2.10 pyh9f0ad1d_0 conda-forge ilamb 2.5 pypi_0 pypi importlib-metadata 4.0.1 py37h89c1867_0 conda-forge ipykernel 5.5.4 py37h085eea5_0 conda-forge ipython 7.23.1 py37h085eea5_0 conda-forge ipython_genutils 0.2.0 py_1 conda-forge ipywidgets 7.5.1 py_0 conda-forge jedi 0.18.0 py37h89c1867_2 conda-forge jinja2 2.11.3 pyh44b312d_0 conda-forge jpeg 9d h36c2ea0_0 conda-forge json5 0.9.5 pyh9f0ad1d_0 conda-forge jsonschema 3.2.0 pyhd8ed1ab_3 conda-forge jupyter-server-proxy 1.2.0 py_0 conda-forge jupyter_client 6.1.12 pyhd8ed1ab_0 conda-forge jupyter_contrib_core 0.3.3 py_2 conda-forge jupyter_contrib_nbextensions 0.5.1 py37_0 conda-forge jupyter_core 4.7.1 py37h89c1867_0 conda-forge jupyter_dashboards 0.7.0 py37_1000 conda-forge jupyter_highlight_selected_word 0.2.0 py37h89c1867_1002 conda-forge jupyter_latex_envs 1.4.6 pyhd8ed1ab_1002 conda-forge jupyter_nbextensions_configurator 0.4.1 py37h89c1867_2 conda-forge jupyterlab 1.2.6 py_0 conda-forge jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge jupyterlab_server 1.0.6 py_0 conda-forge lcms2 2.12 hddcbb42_0 conda-forge libblas 3.9.0 9_openblas conda-forge libcblas 3.9.0 9_openblas conda-forge libedit 3.1.20191231 h46ee950_2 conda-forge libffi 3.2.1 he1b5a44_1007 conda-forge libgcc-ng 9.3.0 h2828fa1_19 conda-forge libgfortran-ng 9.3.0 hff62375_19 conda-forge libgfortran5 9.3.0 hff62375_19 conda-forge libgomp 9.3.0 h2828fa1_19 conda-forge libiconv 1.16 h516909a_0 conda-forge liblapack 3.9.0 9_openblas conda-forge libopenblas 0.3.15 pthreads_h8fe5266_0 conda-forge libpng 1.6.37 h21135ba_2 conda-forge libsodium 1.0.18 h36c2ea0_1 conda-forge libstdcxx-ng 9.3.0 h6de172a_19 conda-forge libtiff 4.2.0 hdc55705_1 conda-forge libuv 1.34.0 h516909a_0 conda-forge libwebp-base 1.2.0 h7f98852_2 conda-forge libxml2 2.9.10 hee79883_0 conda-forge libxslt 1.1.33 h31b3aaa_0 conda-forge locket 0.2.0 py_2 conda-forge lxml 4.6.3 py37h77fd288_0 conda-forge lz4-c 1.9.3 h9c3ff4c_0 conda-forge markupsafe 1.1.1 py37h5e8e339_3 conda-forge matplotlib-inline 0.1.2 pyhd8ed1ab_2 conda-forge mistune 0.8.4 py37h5e8e339_1003 conda-forge msgpack-python 1.0.2 py37h2527ec5_1 conda-forge multidict 5.1.0 py37h5e8e339_1 conda-forge nb_conda_kernels 2.2.2 py37_0 conda-forge nbclient 0.5.3 pyhd8ed1ab_0 conda-forge nbconvert 6.0.7 py37h89c1867_3 conda-forge nbformat 5.1.3 pyhd8ed1ab_0 conda-forge ncurses 6.1 hf484d3e_1002 conda-forge nest-asyncio 1.5.1 pyhd8ed1ab_0 conda-forge nodejs 13.8.0 h10a4023_0 conda-forge notebook 6.3.0 pyha770c72_1 conda-forge numpy 1.20.2 py37h038b26d_0 conda-forge olefile 0.46 pyh9f0ad1d_1 conda-forge openjpeg 2.4.0 hf7af979_0 conda-forge openssl 1.1.1k h7f98852_0 conda-forge packaging 20.9 pyh44b312d_0 conda-forge pandas 1.2.4 py37h219a48f_0 conda-forge pandoc 2.12 h7f98852_0 conda-forge pandocfilters 1.4.2 py_1 conda-forge parso 0.8.2 pyhd8ed1ab_0 conda-forge partd 1.2.0 pyhd8ed1ab_0 conda-forge pexpect 4.8.0 pyh9f0ad1d_2 conda-forge pickleshare 0.7.5 py_1003 conda-forge pillow 8.1.2 py37h4600e1f_1 conda-forge pip 20.0.2 py_2 conda-forge prometheus_client 0.10.1 pyhd8ed1ab_0 conda-forge prompt-toolkit 3.0.18 pyha770c72_0 conda-forge psutil 5.8.0 py37h5e8e339_1 conda-forge ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge pyaverager 0.9.17 pypi_0 pypi pycosat 0.6.3 py37h5e8e339_1006 conda-forge pycparser 2.20 pyh9f0ad1d_2 conda-forge pygments 2.9.0 pyhd8ed1ab_0 conda-forge pyopenssl 20.0.1 pyhd8ed1ab_0 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyrsistent 0.17.3 py37h5e8e339_2 conda-forge pysocks 1.7.1 py37h89c1867_3 conda-forge python 3.7.3 h5b0a415_0 conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 3.7 1_cp37m conda-forge pytz 2021.1 pyhd8ed1ab_0 conda-forge pyyaml 5.4.1 py37h5e8e339_0 conda-forge pyzmq 22.0.3 py37h336d617_1 conda-forge readline 7.0 hf8c457e_1001 conda-forge requests 2.25.1 pyhd3deb0d_0 conda-forge ruamel_yaml 0.15.80 py37h5e8e339_1004 conda-forge send2trash 1.5.0 py_0 conda-forge setuptools 49.6.0 py37h89c1867_3 conda-forge simpervisor 0.4 pyhd8ed1ab_0 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge sortedcontainers 2.3.0 pyhd8ed1ab_0 conda-forge sqlite 3.31.1 h7b6447c_0 defaults tblib 1.7.0 pyhd8ed1ab_0 conda-forge terminado 0.9.4 py37h89c1867_0 conda-forge testpath 0.4.4 py_0 conda-forge tk 8.6.10 h21135ba_1 conda-forge toolz 0.11.1 py_0 conda-forge tornado 6.1 py37h5e8e339_1 conda-forge tqdm 4.60.0 pyhd8ed1ab_0 conda-forge traitlets 4.3.3 py37hc8dfbb8_1 conda-forge typing-extensions 3.7.4.3 0 conda-forge typing_extensions 3.7.4.3 py_0 conda-forge udunits2 2.2.27.27 h975c496_1 conda-forge urllib3 1.26.4 pyhd8ed1ab_0 conda-forge wcwidth 0.2.5 pyh9f0ad1d_2 conda-forge webencodings 0.5.1 py_1 conda-forge wheel 0.36.2 pyhd3deb0d_0 conda-forge widgetsnbextension 3.5.1 py37_0 conda-forge xz 5.2.5 h516909a_1 conda-forge yaml 0.2.5 h516909a_0 conda-forge yarl 1.6.3 py37h5e8e339_1 conda-forge zeromq 4.3.4 h9c3ff4c_0 conda-forge zict 2.0.0 py_0 conda-forge zipp 3.4.1 pyhd8ed1ab_0 conda-forge zlib 1.2.11 h516909a_1010 conda-forge zstd 1.4.9 ha95c52a_0 conda-forge

env:

USER=oleson LOGNAME=oleson HOME=/glade/u/home/oleson PATH=/glade/u/home/oleson/.local/bin:/glade/apps/contrib/virtualenv/12.0.7:/glade/u/apps/opt/usr/bin/:/glade/u/home/mickelso/swift/swift-0.94.1-RC1/bin/:/glade/u/apps/ch/opt/git/2.33.1/gnu/9.1.0/bin:/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/bin:/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/bin:/glade/u/apps/ch/opt/ncl/6.6.2/intel/19.1.1/bin:/glade/u/apps/ch/opt/nco/4.7.9/gnu/8.3.0/bin:/glade/u/apps/ch/opt/ncview/2.1.7/gnu/8.3.0/bin:/glade/u/apps/ch/opt/lmod/8.7.13/bin:/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/bin:/glade/u/apps/ch/opt/ncarcompilers/0.5.0/intel/19.1.1/mpi:/glade/u/apps/ch/opt/mpt/2.25/bin:/glade/u/apps/ch/opt/ncarcompilers/0.5.0/intel/19.1.1:/glade/u/apps/opt/intel/2020u1/vtune_amplifier/bin64:/glade/u/apps/opt/intel/2020u1/inspector/bin64:/glade/u/apps/opt/intel/2020u1/advisor/bin64:/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux/bin/intel64:/glade/u/apps/opt/vncmgr:/glade/u/apps/opt/globus-utils:/glade/u/apps/ch/opt/usr/bin:/glade/u/home/oleson/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin/X11:/usr/games:/opt/ibutils/bin:/usr/lib/mit/bin:/glade/u/apps/opt/hpss/hpss-client.5.0.2.p5/bin:/glade/u/apps/ch/os/usr/bin:/glade/u/apps/ch/opt/hpe-cpe/22.02/bin:/opt/pbs/bin MAIL=/var/mail/oleson SHELL=/bin/tcsh SSH_CLIENT=128.117.70.165 59626 22 SSH_CONNECTION=128.117.70.165 59626 128.117.211.244 22 SSH_TTY=/dev/pts/443 TERM=xterm DISPLAY=cheyenne1:97.0 XAUTHLOCALHOSTNAME=cheyenne1 HOSTTYPE=x86_64 VENDOR=unknown OSTYPE=linux MACHTYPE=x86_64-suse-linux SHLVL=1 PWD=/glade/scratch/oleson/archive/cam6ctsm51_cesm23a12c_ne30pg3g17_CPLHIST_1850/cpl/hist GROUP=ncar HOST=cheyenne1 CSHEDIT=emacs CPU=x86_64 HOSTNAME=cheyenne1.cheyenne.ucar.edu INPUTRC=/etc/inputrc LESS=-M -I -R LESSOPEN=lessopen.sh %s LESSCLOSE=lessclose.sh %s %s LESS_ADVANCED_PREPROCESSOR=no LESSKEY=/etc/lesskey.bin PAGER=less MORE=-sl MINICOM=-c on MANPATH=/glade/u/apps/ch/opt/git/2.33.1/gnu/9.1.0/share/man:/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/share/man:/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/share/man:/glade/u/apps/ch/opt/ncl/6.6.2/intel/19.1.1/man:/glade/u/apps/ch/opt/nco/4.7.9/gnu/8.3.0/share/man:/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/share/man:/glade/u/apps/ch/opt/mpt/2.25/man:/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux/man/common:/usr/share/man:/usr/local/man:/opt/pbs/share/man XKEYSYMDB=/usr/X11R6/lib/X11/XKeysymDB XNLSPATH=/usr/share/X11/nls COLORTERM=1 LIBGL_DEBUG=quiet JAVA_BINDIR=/usr/lib64/jvm/jre-openjdk/bin JAVA_ROOT=/usr/lib64/jvm/jre-openjdk JAVA_HOME=/usr/lib64/jvm/jre-openjdk JRE_HOME=/usr/lib64/jvm/jre-openjdk JDK_HOME=/usr/lib64/jvm/java-openjdk SDK_HOME=/usr/lib64/jvm/java-openjdk CVS_RSH=ssh QT_SYSTEM_DIR=/usr/share/desktop-data LANG=en_US.UTF-8 INSTALLPATH_ROOT=/glade/u/apps/ch/opt PROFILEREAD=true LUAROCKS_PREFIX=/glade/u/apps/ch/opt/lmod/8.7.13 LUA_PATH=/glade/u/apps/ch/opt/lmod/8.7.13/share/lua/5.1/?.lua;/glade/u/apps/ch/opt/lmod/8.7.13/share/lua/5.1/?/init.lua;; LUA_CPATH=/glade/u/apps/ch/opt/lmod/8.7.13/lib/lua/5.1/?.so;; BASH_ENV=/glade/u/apps/ch/opt/lmod/8.7.13/lmod/lmod/init/bash MODULEPATH_ROOT=/glade/u/apps/ch/modulefiles/default MODULEPATH=/glade/u/apps/ch/modulefiles/default/compilers:/glade/u/apps/ch/modulefiles/default/idep:/glade/u/apps/ch/modulefiles/default/intel/19.1.1:/glade/u/apps/ch/modulefiles/default/mpt/2.25/intel/19.1.1 LMOD_SETTARG_CMD=: LMOD_FULL_SETTARG_SUPPORT=no LMOD_ROOT=/glade/u/apps/ch/opt/lmod/8.7.13/lmod LMOD_PKG=/glade/u/apps/ch/opt/lmod/8.7.13/lmod/lmod LMOD_CMD=/glade/u/apps/ch/opt/lmod/8.7.13/lmod/lmod/libexec/lmod LMOD_DIR=/glade/u/apps/ch/opt/lmod/8.7.13/lmod/lmod/libexec MODULESHOME=/glade/u/apps/ch/opt/lmod/8.7.13/lmod/lmod TARG_TITLE_BAR_PAREN= LMOD_SETTARG_FULL_SUPPORT=no LMOD_VERSION=8.7.13 LMOD_SYSTEM_NAME=ch LMOD_SYSTEM_DEFAULT_MODULES=ncarenv/1.3:intel/19.1.1:ncarcompilers/0.5.0:mpt/2.25:netcdf/4.8.1 ADVISOR_2020_DIR=/glade/u/apps/opt/intel/2020u1/advisor CC=icc CESMDATAROOT=/glade/p/cesmdata/cseg CESMROOT=/glade/p/cesm COMPILER=intel COMPILER_VERSION=19.1.1 LMOD_REF_COUNT_CPATH=/glade/u/apps/ch/opt/mpt_fmods/2.25/intel/19.1.1:1;/glade/u/apps/ch/opt/mpt/2.25/include:1 CPATH=/glade/u/apps/ch/opt/mpt_fmods/2.25/intel/19.1.1:/glade/u/apps/ch/opt/mpt/2.25/include CXX=icpc DASK_ROOT_CONFIG=/glade/u/apps/config/dask ENV=/etc/profile.d/modules.sh F77=ifort F90=ifort FC=ifort INSPECTOR_2020_DIR=/glade/u/apps/opt/intel/2020u1/inspector INTEL_BASE_PATH=/glade/u/apps/opt/intel/2020u1 INTEL_COMPILER_BASE_PATH=/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux INTEL_LICENSE_FILE=28518@128.117.177.41 INTEL_MAJOR_VERSION=19.1 INTEL_MINOR_VERSION=1.1 LMOD_REF_COUNT_LD_LIBRARY_PATH=/glade/u/apps/ch/opt/mpt_fmods/2.25/intel/19.1.1:1;/glade/u/apps/ch/opt/mpt/2.25/lib:1;/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux/lib/intel64:1;/glade/u/apps/ch/os/usr/lib64:1;/glade/u/apps/ch/os/usr/lib:1;/glade/u/apps/ch/os/lib64:1;/glade/u/apps/ch/os/lib:1 LD_LIBRARY_PATH=/glade/u/home/swensosc/gvtemp/libXaw3d-1.6/local/lib:/glade/u/apps/ch/opt/mpt_fmods/2.25/intel/19.1.1:/glade/u/apps/ch/opt/mpt/2.25/lib:/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux/lib/intel64:/glade/u/apps/ch/os/usr/lib64:/glade/u/apps/ch/os/usr/lib:/glade/u/apps/ch/os/lib64:/glade/u/apps/ch/os/lib LMOD_REF_COUNT_LIBRARY_PATH=/glade/u/apps/ch/opt/mpt/2.25/lib:1 LIBRARY_PATH=/glade/u/apps/ch/opt/mpt/2.25/lib LMOD_COMPILER=intel LMOD_COMPILER_VERSION=19.1.1 LMOD_FAMILY_COMPILER=intel LMOD_FAMILY_COMPILER_VERSION=19.1.1 LMOD_FAMILY_MPI=mpt LMOD_FAMILY_MPI_VERSION=2.25 LMOD_MPI=mpt_fmods LMOD_MPI_VERSION=2.25 LM_LICENSE_FILE=28518@128.117.177.41 LOADEDMODULES=ncarenv/1.3:intel/19.1.1:ncarcompilers/0.5.0:mpt/2.25:netcdf/4.8.1:ncview/2.1.7:nco/4.7.9:ncl/6.6.2:R/4.1.2:python/3.7.9:git/2.33.1 LMOD_REF_COUNT_MANPATH=/glade/u/apps/ch/opt/git/2.33.1/gnu/9.1.0/share/man:1;/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/share/man:1;/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/share/man:1;/glade/u/apps/ch/opt/ncl/6.6.2/intel/19.1.1/man:1;/glade/u/apps/ch/opt/nco/4.7.9/gnu/8.3.0/share/man:1;/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/share/man:1;/glade/u/apps/ch/opt/mpt/2.25/man:1;/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux/man/common:1;/usr/share/man:1;/usr/local/man:1;/opt/pbs/share/man:1 LMOD_REF_COUNT_MODULEPATH=/glade/u/apps/ch/modulefiles/default/compilers:1;/glade/u/apps/ch/modulefiles/default/idep:1;/glade/u/apps/ch/modulefiles/default/intel/19.1.1:1;/glade/u/apps/ch/modulefiles/default/mpt/2.25/intel/19.1.1:1 MPICC_CC=icc MPICXX_CXX=icpc MPIF08_F08=ifort MPIF90_F90=ifort MPI_IB_CONGESTED=1 MPI_MEMMAP_OFF=true MPI_ROOT=/glade/u/apps/ch/opt/mpt/2.25 MPI_SYSLOG_COPY=2 MPI_XPMEM_ENABLED=false MPT_VERSION=2.25 MP_COMPILER=intel NCAR_EXCLUDE_ASNEEDED=1 NCAR_GLOBUS_AWS=3a1f3e98-1a93-11e9-9f9f-0a06afd4a22e NCAR_GLOBUS_CAMPAIGN=6b5ab960-7bbf-11e8-9450-0a6d4e044368 NCAR_GLOBUS_DSS=dd1ee92a-6d04-11e5-ba46-22000b92c6ec NCAR_GLOBUS_GDRIVE=397f7166-9af5-402f-abfc-c3b184d609ba NCAR_GLOBUS_GLADE=d33b3614-6d04-11e5-ba46-22000b92c6ec NCAR_GLOBUS_QUASAR=58bc6c98-8bba-11e9-b808-0a37f382de32 NCAR_GLOBUS_STRATUS=b9cf5e6c-9245-11eb-b7a4-f57b2d55370d NCAR_HOST=cheyenne NCAR_INC_NETCDF=/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/include NCAR_LDFLAGS_NETCDF=/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/lib NCAR_LIBS_MPT=-lrt -ldl NCAR_LIBS_NETCDF=-Wl,-Bstatic -lnetcdff -lnetcdf -lhdf5hl_fortran -lhdf5_hl -lhdf5_fortran -lhdf5 -lsz -lz -Wl,-Bdynamic -lcurl -lm -ldl NCAR_PBS_CA=casper-pbs NCAR_PBS_CH=chadmin1.ib0.cheyenne.ucar.edu NCAR_RANK_MPT=1000 NCAR_ROOT_INTEL=/glade/u/apps/opt/intel/2020u1 NCAR_ROOT_MPT=/glade/u/apps/ch/opt/mpt/2.25 NCAR_ROOT_NETCDF=/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/ NCAR_WRAPPER_MPI=/glade/u/apps/ch/opt/ncarcompilers/0.5.0/intel/19.1.1/mpi NETCDF=/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/ LMOD_REF_COUNT_NLSPATH=/glade/u/apps/opt/intel/2020u1/compilers_andlibraries/linux/lib/intel64/locale/%l%t/%N:1 NLSPATH=/glade/u/apps/opt/intel/2020u1/compilers_andlibraries/linux/lib/intel64/locale/%l%t/%N OMP_NUM_THREADS=1 OMP_STACKSIZE=64000K OSHCC_CC=icc OSHCXX_CXX=icpc OSHF90_F90=ifort LMOD_REF_COUNT_PATH=/glade/u/apps/ch/opt/git/2.33.1/gnu/9.1.0/bin:1;/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/bin:1;/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/bin:1;/glade/u/apps/ch/opt/ncl/6.6.2/intel/19.1.1/bin:1;/glade/u/apps/ch/opt/nco/4.7.9/gnu/8.3.0/bin:1;/glade/u/apps/ch/opt/ncview/2.1.7/gnu/8.3.0/bin:1;/glade/u/apps/ch/opt/lmod/8.7.13/bin:2;/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/bin:1;/glade/u/apps/ch/opt/ncarcompilers/0.5.0/intel/19.1.1/mpi:2;/glade/u/apps/ch/opt/mpt/2.25/bin:1;/glade/u/apps/ch/opt/ncarcompilers/0.5.0/intel/19.1.1:1;/glade/u/apps/opt/intel/2020u1/vtune_amplifier/bin64:1;/glade/u/apps/opt/intel/2020u1/inspector/bin64:1;/glade/u/apps/opt/intel/2020u1/advisor/bin64:1;/glade/u/apps/opt/intel/2020u1/compilers_and_libraries/linux/bin/intel64:1;/glade/u/apps/opt/vncmgr:1;/glade/u/apps/opt/globus-utils:1;/glade/u/apps/ch/opt/usr/bin:1;/glade/u/home/oleson/bin:1;/usr/bin:1;/bin:2;/usr/sbin:1;/sbin:2;/usr/local/bin:1;/usr/bin/X11:1;/usr/games:1;/opt/ibutils/bin:1;/usr/lib/mit/bin:1;/glade/u/apps/opt/hpss/hpss-client.5.0.2.p5/bin:1;/glade/u/apps/ch/os/usr/bin:1;/glade/u/apps/ch/opt/hpe-cpe/22.02/bin:1;/opt/pbs/bin:1 PERL5LIB=/glade/u/apps/ch/os/usr/lib/perl5/5.18.2/ LMOD_REF_COUNT_PKG_CONFIG_PATH=/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/lib/pkgconfig:1;/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/lib64/pkgconfig:1;/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/lib/pkgconfig:1 PKG_CONFIG_PATH=/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/lib/pkgconfig:/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/lib64/pkgconfig:/glade/u/apps/ch/opt/netcdf/4.8.1/intel/19.1.1/lib/pkgconfig QSCACHE_SERVER=cheyenne TMPDIR=/glade/scratch/oleson/tmp VTUNE_AMPLIFIER_2020_DIR=/glade/u/apps/opt/intel/2020u1/vtune_amplifier WRFIO_NCD_LARGE_FILE_SUPPORT=1 LMFILES=/glade/u/apps/ch/modulefiles/default/idep/ncarenv/1.3.lua:/glade/u/apps/ch/modulefiles/default/compilers/intel/19.1.1.lua:/glade/u/apps/ch/modulefiles/default/intel/19.1.1/ncarcompilers/0.5.0.lua:/glade/u/apps/ch/modulefiles/default/intel/19.1.1/mpt/2.25.lua:/glade/u/apps/ch/modulefiles/default/intel/19.1.1/netcdf/4.8.1.lua:/glade/u/apps/ch/modulefiles/default/idep/ncview/2.1.7.lua:/glade/u/apps/ch/modulefiles/default/idep/nco/4.7.9.lua:/glade/u/apps/ch/modulefiles/default/intel/19.1.1/ncl/6.6.2.lua:/glade/u/apps/ch/modulefiles/default/idep/R/4.1.2.lua:/glade/u/apps/ch/modulefiles/default/idep/python/3.7.9.lua:/glade/u/apps/ch/modulefiles/default/idep/git/2.33.1.lua ModuleTable001=X01vZHVsZVRhYmxlXyA9IHsKTVR2ZXJzaW9uID0gMywKY19yZWJ1aWxkVGltZSA9IGZhbHNlLApjX3Nob3J0VGltZSA9IGZhbHNlLApkZXB0aFQgPSB7fSwKZmFtaWx5ID0gewpjb21waWxlciA9ICJpbnRlbCIsCm1waSA9ICJtcHQiLAp9LAptVCA9IHsKUiA9IHsKZm4gPSAiL2dsYWRlL3UvYXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L2lkZXAvUi80LjEuMi5sdWEiLApmdWxsTmFtZSA9ICJSLzQuMS4yIiwKbG9hZE9yZGVyID0gOSwKcHJvcFQgPSB7fSwKc3RhY2tEZXB0aCA9IDAsCnN0YXR1cyA9ICJhY3RpdmUiLAp1c2VyTmFtZSA9ICJSIiwKd1YgPSAiXjAwMDAwMDA0LjAwMDAwMDAwMS4wMDAwMDAwMDIuKnpmaW5hbCIsCn0sCmdpdCA9IHsKZm4gPSAiL2dsYWRlL3Uv ModuleTable002=YXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L2lkZXAvZ2l0LzIuMzMuMS5sdWEiLApmdWxsTmFtZSA9ICJnaXQvMi4zMy4xIiwKbG9hZE9yZGVyID0gMTEsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAiZ2l0IiwKd1YgPSAiMDAwMDAwMDAyLjAwMDAwMDAzMy4wMDAwMDAwMDEuKnpmaW5hbCIsCn0sCmludGVsID0gewpmbiA9ICIvZ2xhZGUvdS9hcHBzL2NoL21vZHVsZWZpbGVzL2RlZmF1bHQvY29tcGlsZXJzL2ludGVsLzE5LjEuMS5sdWEiLApmdWxsTmFtZSA9ICJpbnRlbC8xOS4xLjEiLApsb2FkT3JkZXIgPSAyLApwcm9wVCA9IHt9LApzdGFja0RlcHRoID0gMCwKc3RhdHVzID0gImFjdGl2ZSIsCnVzZXJOYW1l ModuleTable003=ID0gImludGVsLzE5LjEuMSIsCndWID0gIl4wMDAwMDAxOS4wMDAwMDAwMDEuMDAwMDAwMDAxLip6ZmluYWwiLAp9LAptcHQgPSB7CmZuID0gIi9nbGFkZS91L2FwcHMvY2gvbW9kdWxlZmlsZXMvZGVmYXVsdC9pbnRlbC8xOS4xLjEvbXB0LzIuMjUubHVhIiwKZnVsbE5hbWUgPSAibXB0LzIuMjUiLApsb2FkT3JkZXIgPSA0LApwcm9wVCA9IHt9LApzdGFja0RlcHRoID0gMCwKc3RhdHVzID0gImFjdGl2ZSIsCnVzZXJOYW1lID0gIm1wdC8yLjI1IiwKd1YgPSAiXjAwMDAwMDAyLjAwMDAwMDAyNS4qemZpbmFsIiwKfSwKbmNhcmNvbXBpbGVycyA9IHsKZm4gPSAiL2dsYWRlL3UvYXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L2ludGVsLzE5LjEuMS9uY2FyY29tcGlsZXJzLzAu ModuleTable004=NS4wLmx1YSIsCmZ1bGxOYW1lID0gIm5jYXJjb21waWxlcnMvMC41LjAiLApsb2FkT3JkZXIgPSAzLApwcm9wVCA9IHt9LApzdGFja0RlcHRoID0gMCwKc3RhdHVzID0gImFjdGl2ZSIsCnVzZXJOYW1lID0gIm5jYXJjb21waWxlcnMvMC41LjAiLAp3ViA9ICIwMDAwMDAwMDAuMDAwMDAwMDA1Lip6ZmluYWwiLAp9LApuY2FyZW52ID0gewpmbiA9ICIvZ2xhZGUvdS9hcHBzL2NoL21vZHVsZWZpbGVzL2RlZmF1bHQvaWRlcC9uY2FyZW52LzEuMy5sdWEiLApmdWxsTmFtZSA9ICJuY2FyZW52LzEuMyIsCmxvYWRPcmRlciA9IDEsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAibmNhcmVudi8xLjMiLAp3ViA9ICJeMDAwMDAw ModuleTable005=MDEuMDAwMDAwMDAzLip6ZmluYWwiLAp9LApuY2wgPSB7CmZuID0gIi9nbGFkZS91L2FwcHMvY2gvbW9kdWxlZmlsZXMvZGVmYXVsdC9pbnRlbC8xOS4xLjEvbmNsLzYuNi4yLmx1YSIsCmZ1bGxOYW1lID0gIm5jbC82LjYuMiIsCmxvYWRPcmRlciA9IDgsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAibmNsLzYuNi4yIiwKd1YgPSAiMDAwMDAwMDA2LjAwMDAwMDAwNi4wMDAwMDAwMDIuKnpmaW5hbCIsCn0sCm5jbyA9IHsKZm4gPSAiL2dsYWRlL3UvYXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L2lkZXAvbmNvLzQuNy45Lmx1YSIsCmZ1bGxOYW1lID0gIm5jby80LjcuOSIsCmxvYWRPcmRlciA9IDcsCnByb3BUID0g _ModuleTableSz=9 __Init_Default_Modules=1 LMOD_MODULERCFILE=/glade/u/apps/ch/modulefiles/default/localinit/hidden OSCAR_HOME=/opt/oscar LANGUAGE=en_US.UTF-8 FROM_HEADER= NNTPSERVER=news WINDOWMANAGER=env GNOME_SHELL_SESSION_MODE=classic SLE_CLASSIC_MODE=1 gnome-session --session gnome-classic SDL_AUDIODRIVER=pulse AUDIODRIVER=pulseaudio QEMU_AUDIO_DRV=pa ALSA_CONFIG_PATH=/etc/alsa-pulse.conf PYTHONSTARTUP=/etc/pythonstart XDG_DATA_DIRS=/usr/share XDG_CONFIG_DIRS=/etc/xdg G_BROKEN_FILENAMES=1 G_FILENAME_ENCODING=@locale,UTF-8,ISO-8859-15,CP1252 CSHRCREAD=true LS_COLORS=no=00:fi=00:di=01;34:ln=00;36:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=41;33;01:ex=00;32:.cmd=00;32:.exe=01;32:.com=01;32:.bat=01;32:.btm=01;32:.dll=01;32:.tar=00;31:.tbz=00;31:.tgz=00;31:.rpm=00;31:.deb=00;31:.arj=00;31:.taz=00;31:.lzh=00;31:.lzma=00;31:.zip=00;31:.zoo=00;31:.z=00;31:.Z=00;31:.gz=00;31:.bz2=00;31:.tb2=00;31:.tz2=00;31:.tbz2=00;31:.xz=00;31:.avi=01;35:.bmp=01;35:.fli=01;35:.gif=01;35:.jpg=01;35:.jpeg=01;35:.mng=01;35:.mov=01;35:.mpg=01;35:.pcx=01;35:.pbm=01;35:.pgm=01;35:.png=01;35:.ppm=01;35:.tga=01;35:.tif=01;35:.xbm=01;35:.xpm=01;35:.dl=01;35:.gl=01;35:.wmv=01;35:.aiff=00;32:.au=00;32:.mid=00;32:.mp3=00;32:.ogg=00;32:.voc=00;32:*.wav=00;32: LS_OPTIONS=-N --color=tty -T 0 GPG_TTY=/dev/pts/443 NCAR_ROOT_NCVIEW=/glade/u/apps/ch/opt/ncview/2.1.7/gnu/8.3.0/ ModuleTable006=e30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAibmNvLzQuNy45IiwKd1YgPSAiMDAwMDAwMDA0LjAwMDAwMDAwNy4wMDAwMDAwMDkuKnpmaW5hbCIsCn0sCm5jdmlldyA9IHsKZm4gPSAiL2dsYWRlL3UvYXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L2lkZXAvbmN2aWV3LzIuMS43Lmx1YSIsCmZ1bGxOYW1lID0gIm5jdmlldy8yLjEuNyIsCmxvYWRPcmRlciA9IDYsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwKdXNlck5hbWUgPSAibmN2aWV3IiwKd1YgPSAiMDAwMDAwMDAyLjAwMDAwMDAwMS4wMDAwMDAwMDcuKnpmaW5hbCIsCn0sCm5ldGNkZiA9IHsKZm4gPSAiL2dsYWRlL3UvYXBwcy9jaC9t NCAR_ROOT_NCO=/glade/u/apps/ch/opt/nco/4.7.9/gnu/8.3.0/ NCARG_ROOT=/glade/u/apps/ch/opt/ncl/6.6.2/intel/19.1.1/ NCAR_ROOT_NCL=/glade/u/apps/ch/opt/ncl/6.6.2/intel/19.1.1/ ModuleTable007=b2R1bGVmaWxlcy9kZWZhdWx0L2ludGVsLzE5LjEuMS9uZXRjZGYvNC44LjEubHVhIiwKZnVsbE5hbWUgPSAibmV0Y2RmLzQuOC4xIiwKbG9hZE9yZGVyID0gNSwKcHJvcFQgPSB7fSwKc3RhY2tEZXB0aCA9IDAsCnN0YXR1cyA9ICJhY3RpdmUiLAp1c2VyTmFtZSA9ICJuZXRjZGYvNC44LjEiLAp3ViA9ICJeMDAwMDAwMDQuMDAwMDAwMDA4LjAwMDAwMDAwMS4qemZpbmFsIiwKfSwKcHl0aG9uID0gewpmbiA9ICIvZ2xhZGUvdS9hcHBzL2NoL21vZHVsZWZpbGVzL2RlZmF1bHQvaWRlcC9weXRob24vMy43LjkubHVhIiwKZnVsbE5hbWUgPSAicHl0aG9uLzMuNy45IiwKbG9hZE9yZGVyID0gMTAsCnByb3BUID0ge30sCnN0YWNrRGVwdGggPSAwLApzdGF0dXMgPSAiYWN0aXZlIiwK NCAR_ROOT_R=/glade/u/apps/ch/opt/R/4.1.2/gnu/10.1.0/ RSTUDIO_COMPILER=gnu/10.1.0 NCAR_INC_PYTHON=/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/include NCAR_LDFLAGS_PYTHON=/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0/lib NCAR_ROOT_PYTHON=/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0 NPLROOT=/glade/u/apps/ch/opt/python/3.7.9/gnu/9.1.0 PYMAJOR=3 ModuleTable008=dXNlck5hbWUgPSAicHl0aG9uIiwKd1YgPSAiXjAwMDAwMDAzLjAwMDAwMDAwNy4wMDAwMDAwMDkuKnpmaW5hbCIsCn0sCn0sCm1wYXRoQSA9IHsKIi9nbGFkZS91L2FwcHMvY2gvbW9kdWxlZmlsZXMvZGVmYXVsdC9jb21waWxlcnMiLCAiL2dsYWRlL3UvYXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L2lkZXAiCiwgIi9nbGFkZS91L2FwcHMvY2gvbW9kdWxlZmlsZXMvZGVmYXVsdC9pbnRlbC8xOS4xLjEiLCAiL2dsYWRlL3UvYXBwcy9jaC9tb2R1bGVmaWxlcy9kZWZhdWx0L21wdC8yLjI1L2ludGVsLzE5LjEuMSIsCn0sCnN5c3RlbUJhc2VNUEFUSCA9ICIvZ2xhZGUvdS9hcHBzL2NoL21vZHVsZWZpbGVzL2RlZmF1bHQvY29tcGlsZXJzOi9nbGFkZS91L2FwcHMvY2gvbW9k NCAR_ROOT_GIT=/glade/u/apps/ch/opt/git/2.33.1/gnu/9.1.0/ ModuleTable009=dWxlZmlsZXMvZGVmYXVsdC9pZGVwIiwKfQo= SVNREPO=https://svn-ccsm-models.cgd.ucar.edu EDITOR=vi ILAMB_ROOT=/glade/p/cesm/lmwg_dev/oleson/ILAMB/ILAMB_all POSTPROCESS_PATH=/glade/work/oleson/postprocessing POSTPROCESS_PATH_DAV=/glade/work/oleson/postprocessing_dav PBS_ACCOUNT=P93300041 CONDA_EXE=/glade/work/oleson/miniconda3/bin/conda _CONDA_ROOT=/glade/work/oleson/miniconda3 _CONDA_EXE=/glade/work/oleson/miniconda3/bin/conda CONDA_PYTHON_EXE=/glade/work/oleson/miniconda3/bin/python CONDA_SHLVL=0

echo $SHELL:

/bin/tcsh

I am on cheyenne1

ekluzek commented 1 year ago

OK, this is all interesting and disconcerting. But, this is what I've found.

The conda version you are using is really old 4.10.1 while the one that is supported on cheyenne is 22,.11.1. I'm guessing is that you installed miniconda on cheyenne years ago before it was available, and have been using that version since. Normally that might not be a problem, but maybe it is in this case?

The python environment the older version of conda creates is a little screwy. Some of the versions deviate from the ones set in the file, and it doesn't include xarray and netcdf4 which it has to have to actually work. And it also has some things like ilamb, pyaverager and asaptools. So I'm not sure that is the pure ctsm_pylib environment. We might need to have you delete your current ctsm_pylib and rerun it.

I tried replicating that environment by making small changes to the ctsm_pylib environment, but I didn't see a change in behavior. I've been trying the whole environment, but running into trouble.

So I think we should probably have you move over to using the supported version of conda rather than your own version. And see if that helps here.

It might be best if we do a Google Meet or Zoom call to go through this together.

olyson commented 1 year ago

Yes, I originally installed that version so that I could run ilamb. I think that's the only thing I'm using it for. A meeting would be good, I don't want to cause a problem with my ilamb environment. This is low priority so at your convenience.

wwieder commented 1 year ago

Thanks for looking at this in the first place Keith and digging into the issue, Erik. I agree, this particular issue is potentially now priority to address. I wonder if there is more we need to do prevent conda environment issues from tripping up other users (or force checks in scripts for using ctsm_pylib)? I assed this as a focused discussion topic for this weeks SE meeting.

negin513 commented 1 year ago

Hello @olyson and @ekluzek , I see this and I thought I might be able to help.

  1. First, I see your conda output does not show xarray, so I think maybe you are listing your base environment. The conda list should be done after activate the ctsm_pylib environment, so:
# list all your envs
conda env list
conda activate ctsm_pylib

This should show the xarray version you are using. If you are using an outdated xarray this is the most likely culprit.

  1. To use the updated version of conda, can you please load conda/latest:

    module load conda/latest
which conda

    If it points to a conda other than your original, please do the following and you should have the environment set. 

    conda create --force -n $CTSM_PATH/python/conda_env_ctsm_py.txt
conda activate ctsm_pylib
olyson commented 1 year ago

Thanks for looking at this @negin513 . From your statement, I guess you mean this:

conda activate ctsm_pylib conda env list

not:

conda env list conda activate ctsm_pylib

Either way here's what I get:

conda environments:

# npl /glade/u/apps/opt/conda/envs/npl npl-2022b /glade/u/apps/opt/conda/envs/npl-2022b npl-2023a /glade/u/apps/opt/conda/envs/npl-2023a npl-2207 /glade/u/apps/opt/conda/envs/npl-2207 pygpu-dask /glade/u/apps/opt/conda/envs/pygpu-dask r-4.2 /glade/u/apps/opt/conda/envs/r-4.2 ctsm_py /glade/work/oleson/conda-envs/ctsm_py ctsm_pylib * /glade/work/oleson/conda-envs/ctsm_pylib base /glade/work/oleson/miniconda3 aws_urban /glade/work/oleson/miniconda3/envs/aws_urban ctsm_pylib /glade/work/oleson/miniconda3/envs/ctsm_pylib ilamb /glade/work/oleson/miniconda3/envs/ilamb ilamb_newscore /glade/work/oleson/miniconda3/envs/ilamb_newscore ppe-py /glade/work/oleson/miniconda3/envs/ppe-py python-tutorial /glade/work/oleson/miniconda3/envs/python-tutorial

Then,

module load conda/latest which conda

gives me:

conda: aliased to source /glade/work/oleson/miniconda3/etc/profile.d/conda.csh

so I must be overwriting the module load somehow.

olyson commented 1 year ago

Ok, I removed some conda related things from my .tcshrc and then did module load conda/latest which conda

and got:

conda: aliased to source /glade/u/apps/opt/conda/etc/profile.d/conda.csh

Then I did:

conda env remove -n ctsm_pylib ../../py_env_create

During the installation it said it was installing: xarray conda-forge/noarch::xarray-0.16.2-pyhd8ed1ab_0

conda activate ctsm_pylib ./subset_data point --lat 40.0 --lon 270.0 --site SITE_TEST --create-surface --create-datm --datm-syr 2000 --datm-eyr 2000 --create-user-mods --outdir SITE_TEST

But my solar files still have integer time values.

I will try the --force method next.

olyson commented 1 year ago

conda create --force -n /glade/work/oleson/ctsm5.1.dev118/python/conda_env_ctsm_py.txt

gives me:

CondaValueError: Invalid environment name: '/glade/work/oleson/ctsm5.1.dev118/python/conda_env_ctsm_py.txt' Characters not allowed: {'#', ':', '/', ' '}

olyson commented 1 year ago

@ekluzek I tried this:

conda create --force -n ctsm_pylib --file /glade/work/oleson/ctsm5.1.dev118/python/conda_env_ctsm_py.txt

and had the same problem. The environment was the same as before:

diff conda_list_force_keith conda_list_keith -> no differences

wwieder commented 1 year ago

what about using this suggestion.

conda env create -n ctsm_pylib -f /glade/work/oleson/ctsm5.1.dev118/python/conda_env_ctsm_py.txt || conda env update -n ctsm_pylib -f /glade/work/oleson/ctsm5.1.dev118/python/conda_env_ctsm_py.txt

taken from the end of this thread https://github.com/conda/conda/issues/7819

olyson commented 1 year ago

@wwieder , tried that thanks, but didn't work.

ekluzek commented 1 year ago

I've tried to replicate Keith's environment as closely as I can, and I still haven't seen what's causing the difference. The following changes replicate the python environment when I remove the previous environment beforehand.

diff --git a/python/conda_env_ctsm_py.txt b/python/conda_env_ctsm_py.txt
index 4eb724ab5..e8814008e 100644
--- a/python/conda_env_ctsm_py.txt
+++ b/python/conda_env_ctsm_py.txt
@@ -11,12 +11,22 @@
 # conda activate ctsm_pylib                                  # Do this anytime you want to run a CTSM python script
 #
 python=3.7.9
-pandas
-tqdm
-scipy
-netcdf4
-requests
-packaging
+pandas=1.3.5
+tqdm=4.65.0
+scipy=1.7.3
+netcdf4=1.5.8
+idna=3.4
+pathspec=0.11.1
+platformdirs=3.2.0
+pyopenssl=23.1.1
+pytz=2023.3
+typing-extensions=4.5.0
+typing_extensions=4.5.0
+urllib3=1.26.15
+wheel=0.40.0
+readline=8.2
+requests=2.28.2
+packaging=23.0
 numpy=1.18.5
 xarray=0.16.2
 pylint=2.8.3

I'm using the same modules as well.

The one thing I'm wondering about that I'm trying is when I replicate the shell environment

source ~oleson/.tcshrc
source ~oleson/.aliases

I note then that python3 is using a python 3.4.10 version, which might have to do with the problem. But, when I try to use that it dies with a syntax error rather than running and giving bad results.

olyson commented 1 year ago

I was searching on the ESDS Zulip site and found this suggestion:

conda update --all --prune

I tried it and it fixed the solar time variable problem. Model ran for five days. Probably not what we really want to do since it seems to update all the packages within the environment to the latest, but it got me going.

ekluzek commented 1 year ago

Hmmm, OK. At least I'm glad you got something that works for you. That does suggest that it is something in the python environment that was causing the problem. But, still not clear what it was.

olyson commented 1 year ago

I tried just updating xarray in conda_env_ctsm_py.txt from xarray=0.16.2 to xarray=0.17.0 and then did:

conda install -n ctsm_pylib --file conda_env_ctsm_py.txt

and that fixed the problem. Is that a useful clue to what is going on?

olyson commented 1 year ago

As suggested by @ekluzek , I filed an issue with CISL (RC-17597).

negin513 commented 1 year ago

Thanks for looking at this @negin513 . From your statement, I guess you mean this:

conda activate ctsm_pylib conda env list

not:

conda env list conda activate ctsm_pylib

Either way here's what I get:

conda environments:

npl /glade/u/apps/opt/conda/envs/npl npl-2022b /glade/u/apps/opt/conda/envs/npl-2022b npl-2023a /glade/u/apps/opt/conda/envs/npl-2023a npl-2207 /glade/u/apps/opt/conda/envs/npl-2207 pygpu-dask /glade/u/apps/opt/conda/envs/pygpu-dask r-4.2 /glade/u/apps/opt/conda/envs/r-4.2 ctsm_py /glade/work/oleson/conda-envs/ctsm_py ctsm_pylib * /glade/work/oleson/conda-envs/ctsm_pylib base /glade/work/oleson/miniconda3 aws_urban /glade/work/oleson/miniconda3/envs/aws_urban ctsm_pylib /glade/work/oleson/miniconda3/envs/ctsm_pylib ilamb /glade/work/oleson/miniconda3/envs/ilamb ilamb_newscore /glade/work/oleson/miniconda3/envs/ilamb_newscore ppe-py /glade/work/oleson/miniconda3/envs/ppe-py python-tutorial /glade/work/oleson/miniconda3/envs/python-tutorial

Then,

module load conda/latest which conda

gives me:

conda: aliased to source /glade/work/oleson/miniconda3/etc/profile.d/conda.csh

so I must be overwriting the module load somehow.

Thanks for looking at this @negin513 . From your statement, I guess you mean this:

conda activate ctsm_pylib conda env list

not:

conda env list conda activate ctsm_pylib

First, thanks for your response, @olyson. No I exactly meant as stated in my comment:

conda env list 
conda activate ctsm_pylib

These two commands allow you to first list all available environments on your system and then activate the desired ctsm_pylib environment.

I believe you may have mistaken conda list for conda env list in your previous comment, but the steps I provided are accurate.

olyson commented 1 year ago

We are taking this offline with a CISL help request, but for completeness, here is the output of "conda env list":

conda environments:

# base /glade/u/apps/opt/conda npl /glade/u/apps/opt/conda/envs/npl npl-2022b /glade/u/apps/opt/conda/envs/npl-2022b npl-2023a /glade/u/apps/opt/conda/envs/npl-2023a npl-2207 /glade/u/apps/opt/conda/envs/npl-2207 pygpu-dask /glade/u/apps/opt/conda/envs/pygpu-dask r-4.2 /glade/u/apps/opt/conda/envs/r-4.2 ctsm_pylib /glade/work/oleson/conda-envs/ctsm_pylib /glade/work/oleson/miniconda3 /glade/work/oleson/miniconda3/envs/aws_urban /glade/work/oleson/miniconda3/envs/ctsm_pylib /glade/work/oleson/miniconda3/envs/ilamb /glade/work/oleson/miniconda3/envs/ilamb_newscore /glade/work/oleson/miniconda3/envs/ppe-py /glade/work/oleson/miniconda3/envs/python-tutorial

negin513 commented 1 year ago

Hello Keith and Erik, Great news! After conducting a thorough investigation, I was able to reproduce the exact values you encountered and managed to identify the source of the issue.

The problem seems to originate from a combination of the solar input file and the versions of xarray (0.16.2) and cftime.

To reproduce your exact values, I pinned my xarray to version 0.16.2. I noticed that only your solar files exhibit integer-like values, whereas the precipitation and TPQWL files do not.

This discrepancy intrigued me, so I developed a simple script to show the origin of the issue. This script simply reads your input file with xarray and writes it out again, illustrating the discrepancy between the read and written NetCDF files for solar files and not precipitation files. This script demonstrates that the issue arises directly from the combination of xarray and solar input files.

Running this code reveals the differing outputs for solar files (but not for precipitation files). This is potentially due to some encoding in the solar files that caused xarray version 16.2 not being successful in interpreting the times when writing it out. 

import xarray as xr                                                                                                                                                                                                                   
import cftime

# Print xarray version
print(xr.__version__)
print(cftime.__version__)

# Open and process the solar dataset
solar_file_in = "/glade/p/cesmdata/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Solar/clmforc.GSWP3.c2011.0.5x0.5.Solr.2000-01.nc"
solar_file_out = "solarc.nc"

# Read solar dataset
solar_ds = xr.open_dataset(solar_file_in)
print('Read in times for solar dataset:')
print(solar_ds.time)

# Save solar dataset
solar_ds.to_netcdf(path=solar_file_out, mode="w", format="NETCDF3_64BIT")

# Read saved solar dataset
solar_ds2 = xr.open_dataset(solar_file_out)
print('Written out times for solar dataset:')
print(solar_ds2.time)

print('---')
#------------------------------------------#
# Open and process the precipitation dataset
precip_file_in = "/glade/p/cesmdata/inputdata/atm/datm7/atm_forcing.datm7.GSWP3.0.5d.v1.c170516/Precip/clmforc.GSWP3.c2011.0.5x0.5.Prec.2000-01.nc"
precip_file_out = "prec.nc"

# Read precipitation dataset
precip_ds = xr.open_dataset(precip_file_in)
print('Read in times for precipitation dataset:')
print(precip_ds.time)

# Save precipitation dataset
precip_ds.to_netcdf(path=precip_file_out, mode="w", format="NETCDF3_64BIT")

# Read saved precipitation dataset
precip_ds2 = xr.open_dataset(precip_file_out)
print('Written out times for precipitation dataset:')
print(precip_ds2.time)

This script demonstrates that the issue arises directly from the combination of xarray and solar input files. (nothing regarding subset_data or any other ctsm related code just xarray read in and write out).

I reviewed the xarray release history and discovered that version 17 includes updates that directly address this issue (here is the relevant pr: https://github.com/pydata/xarray/pull/4684).

I examined the encoding between precipitation and solar files but could not pinpoint the exact trigger for the different behavior in xarray/cftime. I did notice that their attributes vary slightly.

As a solution, I would recommend updating your xarray version to a more recent release. Since version 16.2 (2020), 20 newer tags of xarray have been made that include similar bug fixes (https://docs.xarray.dev/en/stable/whats-new.html[)|https://docs.xarray.dev/en/stable/whats-new.html] , and upgrading xarray should resolve the issue.

Please feel free to reach out if you encounter any further problems or have any other questions. I think updating the xarray version will easily take care of this issue but I'm happy to help if you have any other questions!

Warm regards,

Negin

negin513 commented 1 year ago

I suggest @ekluzek also try this to make sure he is getting similar answers.

Here are my answers from running the above example code (xarray read/write) with two different versions of xarray (16.2 vs 17.0): Please note how the Written out times for solar dataset: is different between the different versions of xarray. 

 conda activate ctsm_py
python solar_time_issue.py (code snippet from the above)

output:

0.16.2
1.6.0
Read in times for solar dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 3, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 6, 0, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 15, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 18, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 21, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 00:00:00 ... 2000-01-31 21:00:00
Attributes:
    long_name:  observation time
Written out times for solar dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 0, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 00:00:00 ... 2000-01-31 00:00:00
Attributes:
    long_name:  observation time
---
Read in times for precipitation dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 1, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 4, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 7, 30, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 16, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 19, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 22, 30, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 01:30:00 ... 2000-01-31 22:30:00
Written out times for precipitation dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 1, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 4, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 7, 30, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 16, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 19, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 22, 30, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 01:30:00 ... 2000-01-31 22:30:00 
conda install xarray=0.17.0 
python solar_time_issue.py
 2022.6.0
1.6.1
Read in times for solar dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 3, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 6, 0, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 15, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 18, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 21, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 00:00:00 ... 2000-01-31 21:00:00
Attributes:
    long_name:  observation time
Written out times for solar dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 0, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 3, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 6, 0, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 15, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 18, 0, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 21, 0, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 00:00:00 ... 2000-01-31 21:00:00
Attributes:
    long_name:  observation time
---
Read in times for precipitation dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 1, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 4, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 7, 30, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 16, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 19, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 22, 30, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 01:30:00 ... 2000-01-31 22:30:00
Written out times for precipitation dataset:
<xarray.DataArray 'time' (time: 248)>
array([cftime.DatetimeNoLeap(2000, 1, 1, 1, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 4, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 1, 7, 30, 0, 0, has_year_zero=True), ...,
       cftime.DatetimeNoLeap(2000, 1, 31, 16, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 19, 30, 0, 0, has_year_zero=True),
       cftime.DatetimeNoLeap(2000, 1, 31, 22, 30, 0, 0, has_year_zero=True)],
      dtype=object)
Coordinates:
  * time     (time) object 2000-01-01 01:30:00 ... 2000-01-31 22:30:00
ekluzek commented 1 year ago

@negin513 thanks so much for this nice analysis! This undoubtably identifies the problem. Thanks for finding the specific issue/PR in xarray that shows the issue. I could see that xarray had changes in the next version, but didn't see an easy way to figure out which issue/PR was the culprit. Is there a trick to how you found it?

ekluzek commented 1 year ago

The thing that I've found now is that the issue isn't so much "why doesn't this work for Keith" -- it's "why does it work for me"? It turns out I can change my conda environment to a different version of xarray/cftime, but when I run python it's still using later versions of them inside of python. Your test script pointed that out for me, and I also see it when I just pull up python and type some lines in the command line. I've been trying to figure out why it's overriding the xarray version from my conda environment -- but haven't figured it out. @negin513 let me know if you have any ideas for me to try...

ekluzek commented 1 year ago

OK @negin513 and @olyson I figured out what was interfering in my conda environment so that I was getting the wrong version of xarray. I had apparently installed xarray with pip at some point. So the pip version of xarray was overriding the conda environment. There are special cases when you need both conda and pip to setup environments, but as we see here they don't play well together in terms of being able to have different environments. So I'm going to avoid using pip in order to control my environments using conda.

Now, after removing xarray from pip I'm able to reproduce the results with @negin513 test script. So something we should put in place is a check that the version of xarray being used in subset_data is at least 0.17.

negin513 commented 1 year ago

@ekluzek, I'm glad you were able to resolve the Xarray version mismatch. In general, CISL recommends minimizing the use of pip for package installation, although some packages (such as TensorFlow) require or recommend using pip.

If possible, I suggest updating your dependencies to either the latest versions or matching them with the latest npl (e.g., npl-2023a), such as:

pandas      1.5.3
dask        2022.10.0 
tqdm        4.64.1 
xarray      2023.1.0
scipy       1.10.0 
black       19.10b0  
numpy                     1.23.5

Checking npl can serve as a guideline for updating your dependencies. There are hundreds of different packages available in npl, and you can update your dependencies (which is a subset of npl) on the same cadence as npl to ensure that the combination of versions works. This approach would allow users to use either ctsm_pylib or the latest npl. For example, if you remember there was an issue with Dask versions having conflict during installation, so this ensures that similar issues won't happen. This is just a suggestion, so feel free to ignore it if it doesn't work for you. :-)

negin513 commented 1 year ago

@negin513 thanks so much for this nice analysis! This undoubtably identifies the problem. Thanks for finding the specific issue/PR in xarray that shows the issue. I could see that xarray had changes in the next version, but didn't see an easy way to figure out which issue/PR was the culprit. Is there a trick to how you found it?

Hey @ekluzek, great question. Honestly, there were no magic involved in figuring it out. Once I knew version 17 worked I checked the Xarray What's New Page which pointed to the updates and PRs. Then I navigated those PRs until I've found it.

olyson commented 9 months ago

Closing this as it has been resolved.