can't install gsw - build failing with pip and conda

[sophieclayton]

I've tried using both conda and pip to install gsw, but the build is failing. Any ideas what the issue might be? I am running Python 3.9.7, pip 21.2.4 and conda 4.10.3

Here is the error log when trying to install using pip:

(soph) ekman:analysis sclayton$ pip install gsw
Collecting gsw
  Using cached gsw-3.4.0.tar.gz (2.6 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: numpy in /Users/sclayton/opt/anaconda3/envs/soph/lib/python3.9/site-packages (from gsw) (1.21.2)
Building wheels for collected packages: gsw
  Building wheel for gsw (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/sclayton/opt/anaconda3/envs/soph/bin/python /Users/sclayton/opt/anaconda3/envs/soph/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /var/folders/1z/b51_h5pj2n37kyl3zp0xnlx80000gn/T/tmp87l22lvn
       cwd: /private/var/folders/1z/b51_h5pj2n37kyl3zp0xnlx80000gn/T/pip-install-17k9ksbx/gsw_168e5845630f4dbbb9692d2949dc7609
  Complete output (59 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-10.9-x86_64-3.9
  creating build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/stability.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/_version.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/conversions.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/utility.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/__init__.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/energy.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/_utilities.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/_fixed_wrapped_ufuncs.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/density.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/ice.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/_wrapped_ufuncs.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/geostrophy.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  copying gsw/freezing.py -> build/lib.macosx-10.9-x86_64-3.9/gsw
  running egg_info
  writing gsw.egg-info/PKG-INFO
  writing dependency_links to gsw.egg-info/dependency_links.txt
  writing requirements to gsw.egg-info/requires.txt
  writing top-level names to gsw.egg-info/top_level.txt
  listing git files failed - pretending there aren't any
  reading manifest file 'gsw.egg-info/SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no files found matching 'src/c_gsw/LICENSE'
  no previously-included directories found matching 'docs'
  no previously-included directories found matching 'tools'
  no previously-included directories found matching 'notebooks'
  warning: no previously-included files matching '*.so' found anywhere in distribution
  warning: no previously-included files found matching '*.yml'
  warning: no previously-included files found matching '*.enc'
  warning: no previously-included files found matching '.gitignore'
  warning: no previously-included files found matching '.isort.cfg'
  adding license file 'LICENSE.txt'
  writing manifest file 'gsw.egg-info/SOURCES.txt'
  creating build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/_WIP_test_ufuncs.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/check_functions.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/geo_strf_dyn_height.npy -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/geo_strf_velocity.npy -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/gsw_check_functions_save.m -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/gsw_cv_v3_0.npz -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/list_check_functions.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/test_check_functions.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/test_geostrophy.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/test_utility.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/test_xarray.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  copying gsw/tests/write_geo_npyfiles.py -> build/lib.macosx-10.9-x86_64-3.9/gsw/tests
  running build_ext
  building 'gsw._gsw_ufuncs' extension
  creating build/temp.macosx-10.9-x86_64-3.9
  creating build/temp.macosx-10.9-x86_64-3.9/src
  creating build/temp.macosx-10.9-x86_64-3.9/src/c_gsw
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/sclayton/opt/anaconda3/envs/soph/include -arch x86_64 -I/Users/sclayton/opt/anaconda3/envs/soph/include -fPIC -O2 -isystem /Users/sclayton/opt/anaconda3/envs/soph/include -arch x86_64 -I/private/var/folders/1z/b51_h5pj2n37kyl3zp0xnlx80000gn/T/pip-build-env-pez8phnu/overlay/lib/python3.9/site-packages/numpy/core/include -I/private/var/folders/1z/b51_h5pj2n37kyl3zp0xnlx80000gn/T/pip-install-17k9ksbx/gsw_168e5845630f4dbbb9692d2949dc7609/src/c_gsw -I/Users/sclayton/opt/anaconda3/envs/soph/include/python3.9 -c src/_ufuncs.c -o build/temp.macosx-10.9-x86_64-3.9/src/_ufuncs.o
  xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for gsw
Failed to build gsw
ERROR: Could not build wheels for gsw which use PEP 517 and cannot be installed directly

[efiring]

It looks like your pip build is failing because you don't have the command-line compiler tools installed.
To install them, execute:

xcode-select --install

However, for installing any released version of gsw, I recommend using conda with the conda-forge channel.

In your activated conda environment, try:

conda install -c conda-forge gsw

[sophieclayton]

thanks @efiring, I worked that out right after I posted this issue...

that said, I have been unable to install gsw via conda using the above command because of various package conflicts that conda can't resolve.

I think that this is the relevant output:

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  - argon2-cffi -> python[version='2.7.*|3.5.*|3.6.*|3.6.12|3.6.12|3.7.10|3.7.10|3.7.12|3.7.9|3.6.9|3.6.9|3.6.9|3.6.9|>=3.5,<3.6.0a0|3.4.*|3.7.*|3.9.*|3.8.*|3.10.*',build='0_73_pypy|1_73_pypy|2_73_pypy|5_73_pypy|5_73_pypy|0_73_pypy|0_73_pypy|1_73_pypy|4_73_pypy|3_73_pypy']
  - backcall -> python[version='>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.6,<3.7.0a0']
  - cffi -> python[version='2.7.*|3.5.*|3.6.*|3.6.12|3.6.12|3.7.10|3.7.10|3.7.12|>=3.10,<3.11.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.9,<3.10.0a0|>=3.6,<3.7.0a0|3.7.9|3.6.9|3.6.9|3.6.9|>=2.7,<2.8.0a0|3.6.9|>=3.5,<3.6.0a0|3.4.*',build='0_73_pypy|1_73_pypy|2_73_pypy|5_73_pypy|5_73_pypy|0_73_pypy|0_73_pypy|1_73_pypy|4_73_pypy|3_73_pypy']
  - importlib_metadata -> python[version='>=2.7,<2.8.0a0|>=3.8,<3.9.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']
  - ipython_genutils -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']
  - olefile -> python[version='2.7.*|3.5.*|3.6.*|3.4.*|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=2.7,<2.8.0a0']
  - pexpect -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0|3.4.*']
  - pexpect -> python[version='3.7.*|3.8.*']
  - ptyprocess -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.5,<3.6.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|3.4.*|>=3.8,<3.9.0a0']
  - wcwidth -> python[version='2.7.*|3.4.*|3.5.*|3.6.*|>=3.6,<3.7.0a0|>=2.7,<2.8.0a0|>=3.8,<3.9.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']

Your python: python=3.9

Is the issue that I am running python ver 3.9?

I was able to install using pip, but then importing gsw fails in the script.

[sophieclayton]

re-opened, issue outlined in edited comment above...

[efiring]

Sophie, for conda, try this first:

conda config --append channels conda-forge

My preference is to use miniconda, make conda-forge the default source, and put things in my working environment manually rather than starting with the anaconda distribution. See https://currents.soest.hawaii.edu/ocn_data_analysis/installation.html.

[sophieclayton]

Thanks Eric. I already have conda-forge set as the default source, and I have a separate working environment set up that I am trying to install gsw into. I'm not sure what's going on as I haven't had any issues installing or using gsw in the past... but I did just do a clean reinstall of conda and upgraded to python 3.9, so I'm not surprised that everything is broken now.

[DocOtak]

Would it be worth investigating the usage of pypa/cibuildwheel to make wheels all the various combinations of os/arch combinations?

[sophieclayton]

Update: I was able to get the installation to work only be creating a brand new environment, installing gsw and then adding other needed packages. This is fine for now, but wondering if others may run into the same issue I had above?

[ocefpaf]

@DocOtak our current setup creates all wheels. I'm just lagging behind automating the upload of the rest of them due to a hiccup with my PyPI account. With that said, cibuildwheels improved a lot and automates many of the steps we wrote here, so it is worth taking some time to implement it.

@sophieclayton conflicts are bound to happen with big environments. Please check conda-forge usage docs for the best practices on installing software.

https://conda-forge.org/docs/user/introduction.html#how-can-i-install-packages-from-conda-forge

[sophieclayton]

thanks @ocefpaf as I said above, my original issue occurred when trying to install gsw into a separate environment with a relatively small number of packages already installed (numpy, pandas, xarray, matplotlib, scipy + dependencies), not conda base.

I'll close this now as I have gsw working in a new environment.

[alexhoppsalot]

I'm using visual studios and keep gettng the following error every time I try to install gsw.

Collecting gsw Using cached gsw-3.4.0.tar.gz (2.6 MB) Installing build dependencies ... error error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> [20 lines of output] Collecting pip>9.0.1 Using cached pip-22.1-py3-none-any.whl (2.1 MB) Collecting setuptools>=42 Using cached setuptools-62.3.2-py3-none-any.whl (1.2 MB) Collecting wheel Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB) Collecting setuptools_scm[toml]>=3.4 Using cached setuptools_scm-6.4.2-py3-none-any.whl (37 kB) Collecting cython Using cached Cython-0.29.30-py2.py3-none-any.whl (985 kB) Collecting numpy Using cached numpy-1.22.3-cp310-cp310-win_amd64.whl (14.7 MB) Collecting tomli>=1.0.0 Using cached tomli-2.0.1-py3-none-any.whl (12 kB) Collecting packaging>=20.0 Using cached packaging-21.3-py3-none-any.whl (40 kB) Collecting pyparsing!=3.0.5,>=2.0.2 Using cached pyparsing-3.0.9-py3-none-any.whl (98 kB) ERROR: To modify pip, please run the following command: C:\Users\khopp\AppData\Local\Programs\Python\Python310\python.exe -m pip install --ignore-installed --no-user --prefix C:\Users\khopp\AppData\Local\Temp\pip-build-env-4zfk1_z4\overlay --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- pip >9.0.1 setuptools>=42 wheel setuptools_scm[toml]>=3.4 cython numpy [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully. │ exit code: 1 ╰─> See above for output.

[ocefpaf]

@alexhoppsalot you have an old pip and your setup is having trouble updating it. I don't know the first thing about visual studios to help you further, sorry. You'll have to ask at visual studios how tou can update your pip.