Open jr3cermak opened 2 years ago
Reloading an environment that was previously stated as working may yield a result like this:
(gridtest) jrcermakiii@chinook03:~/src/gridtools/examples$ python mkGridsExample01.py
Traceback (most recent call last):
File "mkGridsExample01.py", line 12, in <module>
from gridtools.gridutils import GridUtils
File "/import/AKWATERS/jrcermakiii/src/gridtools/gridtools/gridutils.py", line 5, in <module>
import xarray as xr
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/xarray/__init__.py", line 1, in <module>
from . import testing, tutorial, ufuncs
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/xarray/testing.py", line 8, in <module>
from xarray.core import duck_array_ops, formatting, utils
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/xarray/core/duck_array_ops.py", line 13, in <module>
import pandas as pd
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/__init__.py", line 22, in <module>
from pandas.compat import (
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/compat/__init__.py", line 15, in <module>
from pandas.compat.numpy import (
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/compat/numpy/__init__.py", line 7, in <module>
from pandas.util.version import Version
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/util/__init__.py", line 1, in <module>
from pandas.util._decorators import ( # noqa
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/util/_decorators.py", line 14, in <module>
from pandas._libs.properties import cache_readonly # noqa
File "/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/__init__.py", line 13, in <module>
from pandas._libs.interval import Interval
ImportError: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by /import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so)
This is an indication that the underlying system (or cluster) you are using is falling out of date. The supported python versions of modules managed by conda are under constant integration (CI) are are rebuilt using newer systems and hence are linked to more recent versions of GLIBC. On my particular system, the underlying GLIBC is version 2.12. One or more of the loaded python modules now requires GLIBC 2.14. Unless you have a backup of the conda environment, there is no way to recover the original operating environment. You will likely have to resort to conventional manual installation of modules to restore the environment.
To see what your conda installation is using for GLIBC, use conda info
. The version of glibc should be noted in the virtual packages.
$ conda info
active environment : gridtest
active env location : /import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest
shell level : 1
user config file : /home/jrcermakiii/.condarc
populated config files : /home/jrcermakiii/.condarc
conda version : 4.9.2
conda-build version : not installed
python version : 3.9.1.final.0
virtual packages : __glibc=2.12=0
__unix=0=0
__archspec=1=x86_64
base environment : /import/AKWATERS/jrcermakiii/local/miniconda3 (writable)
channel URLs : https://conda.anaconda.org/pyviz/linux-64
https://conda.anaconda.org/pyviz/noarch
https://conda.anaconda.org/conda-forge/linux-64
https://conda.anaconda.org/conda-forge/noarch
https://repo.anaconda.com/pkgs/main/linux-64
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/r/linux-64
https://repo.anaconda.com/pkgs/r/noarch
package cache : /import/AKWATERS/jrcermakiii/local/miniconda3/pkgs
/home/jrcermakiii/.conda/pkgs
envs directories : /import/AKWATERS/jrcermakiii/local/miniconda3/envs
/home/jrcermakiii/.conda/envs
platform : linux-64
user-agent : conda/4.9.2 requests/2.25.0 CPython/3.9.1 Linux/2.6.32-754.35.1.el6.61015g0000.x86_64 centos/6.10 glibc/2.12
UID:GID : 3739:2161
netrc file : None
offline mode : False
The second failure mode is updating the base
environment for conda. This message is typically seen when a new conda version is available.
==> WARNING: A newer version of conda exists. <==
current version: 4.9.2
latest version: 4.12.0
Please update conda by running
$ conda update -n base -c defaults conda
However, due to the age of your operating system, doing the above can render your conda unusable even though the above command will succeed! You may see the following error messages after updating for one or more of the conda commands:
$ conda create -n gridtest python==3.7.10
Collecting package metadata (current_repodata.json): failed
CondaHTTPError: HTTP 000 CONNECTION FAILED for url <https://conda.anaconda.org/pyviz/linux-64/current_repodata.json>
Elapsed: -
An HTTP error occurred when trying to retrieve this URL.
HTTP errors are often intermittent, and a simple retry will get you on your way.
'https://conda.anaconda.org/pyviz/linux-64'
Again, there is no easy way to recover from this incident unless you have a backup of your conda installation directory. The only way to recover is to reinstall conda with a last known working version and reload your individual conda environments.
If conda is broken beyond repair, there may be a way to salvage your environments. If you
were fortunate to keep the miniconda shell installer archive (Miniconda3-py39_4.9.2-Linux-x86_64.sh
),
then you may be in luck.
Process:
envs
directoryenvs
directoryWe have also discovered that this GLIBC issue is currently confined to one particular package: pandas
. The fix is to load the conda environment as usual for gridtools. Manually uninstall and reinstall pandas
via pip.
$ pip uninstall pandas
Found existing installation: pandas 1.3.5
Uninstalling pandas-1.3.5:
Would remove:
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas-1.3.5-py3.7.egg-info
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/*
Would not remove (might be manually added):
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/algos.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/arrays.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/groupby.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/hashing.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/hashtable.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/index.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/indexing.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/internals.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/interval.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/join.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/json.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/lib.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/missing.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/ops.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/ops_dispatch.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/parsers.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/properties.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/reduction.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/reshape.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/sparse.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/testing.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslib.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/base.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/ccalendar.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/conversion.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/dtypes.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/fields.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/nattype.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/np_datetime.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/offsets.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/parsing.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/period.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/strptime.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/timedeltas.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/timestamps.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/timezones.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/tzconversion.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/tslibs/vectorized.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/window/aggregations.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/window/indexers.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/_libs/writers.cpython-37m-x86_64-linux-gnu.so
/import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages/pandas/io/sas/_sas.cpython-37m-x86_64-linux-gnu.so
Proceed (Y/n)? y
Successfully uninstalled pandas-1.3.5
(gridtest) jrcermakiii@chinook03:~/src/gridtools/docs/conda$ pip install pandas==1.3.5
Collecting pandas==1.3.5
Downloading pandas-1.3.5.tar.gz (4.7 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.7/4.7 MB 7.6 MB/s eta 0:00:00
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: python-dateutil>=2.7.3 in /import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages (from pandas==1.3.5) (2.8.2)
Requirement already satisfied: numpy>=1.17.3 in /import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages (from pandas==1.3.5) (1.21.6)
Requirement already satisfied: pytz>=2017.3 in /import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages (from pandas==1.3.5) (2022.1)
Requirement already satisfied: six>=1.5 in /import/AKWATERS/jrcermakiii/local/miniconda3/envs/gridtest/lib/python3.7/site-packages (from python-dateutil>=2.7.3->pandas==1.3.5) (1.16.0)
Building wheels for collected packages: pandas
Building wheel for pandas (pyproject.toml) ... done
Created wheel for pandas: filename=pandas-1.3.5-cp37-cp37m-linux_x86_64.whl size=29515586 sha256=f5ace4f1391d0a3f8d58490453859e86cdd194c47247b879cade9e05bebbb4e2
Stored in directory: /import/home/jrcermakiii/.cache/pip/wheels/8b/a0/9b/00c6a08ec7b87bf8d7bf4d9ddde7dd7da7c6e822041c580287
Successfully built pandas
Installing collected packages: pandas
Successfully installed pandas-1.3.5
The pinned versions of various pythonic items for gridtools still works. Conda support begins to break down when authors recompile prior versions with newer underlying operating systems. Pinned versions of compiled python modules will begin to fail on compute clusters with very old GLIBC libraries. Documentation to follow.