2.6.0 install via pip fails (Ubuntu 22.04.1 LTS, Python 3.11.1 in a virtual env - not conda)

[stuart-nolan]

Problem description

Cantera version 2.6.0 install failed. Note I worked around this by building and installing the development version from source so this issue can be closed at your convenience.

Steps to reproduce

(p3111) [61] $ pip install cantera
Collecting cantera
  Using cached Cantera-2.6.0.tar.gz (4.5 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: numpy>=1.12.0 in /home/ul/.pyenv/versions/3.11.1/envs/p3111/lib/python3.11/site-packages (from cantera) (1.24.2)
Requirement already satisfied: ruamel.yaml>=0.15.34 in /home/ul/.pyenv/versions/3.11.1/envs/p3111/lib/python3.11/site-packages (from cantera) (0.17.21)
Building wheels for collected packages: cantera
  Building wheel for cantera (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for cantera (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [57 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-311
      creating build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/__init__.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/utils.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/composite.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/onedim.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/liquidvapor.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/ck2yaml.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/ctml_writer.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/ctml2yaml.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/ck2cti.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/cti2yaml.py -> build/lib.linux-x86_64-cpython-311/cantera
      copying cantera/interrupts.py -> build/lib.linux-x86_64-cpython-311/cantera
      creating build/lib.linux-x86_64-cpython-311/cantera/data
      copying cantera/data/__init__.py -> build/lib.linux-x86_64-cpython-311/cantera/data
      creating build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_reaction.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/__init__.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_equilibrium.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_jacobian.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_utils.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_kinetics.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_composite.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_func1.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_onedim.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/utilities.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_transport.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_reactor.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_mixture.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_convert.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_purefluid.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      copying cantera/test/test_thermo.py -> build/lib.linux-x86_64-cpython-311/cantera/test
      creating build/lib.linux-x86_64-cpython-311/cantera/test/data
      copying cantera/test/data/__init__.py -> build/lib.linux-x86_64-cpython-311/cantera/test/data
      creating build/lib.linux-x86_64-cpython-311/cantera/examples
      copying cantera/examples/__init__.py -> build/lib.linux-x86_64-cpython-311/cantera/examples
      running egg_info
      writing Cantera.egg-info/PKG-INFO
      writing dependency_links to Cantera.egg-info/dependency_links.txt
      writing entry points to Cantera.egg-info/entry_points.txt
      writing requirements to Cantera.egg-info/requires.txt
      writing top-level names to Cantera.egg-info/top_level.txt
      reading manifest file 'Cantera.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE.txt'
      writing manifest file 'Cantera.egg-info/SOURCES.txt'
      error: Error: setup script specifies an absolute path:

          /tmp/pip-install-5pbxnfgu/cantera_612913a30a6e446b8111c9edcc655594/ext/fmt/format.cc

      setup() arguments must *always* be /-separated paths relative to the
      setup.py directory, *never* absolute paths.

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for cantera
Failed to build cantera
ERROR: Could not build wheels for cantera, which is required to install pyproject.toml-based projects

I found a similar error reported here; however, none of the "fixes" referenced worked for me. Perhaps this is related to setuptools >= 64? I have:

(p3111) [68] $ pip list | grep setuptools
setuptools                        67.2.0
setuptools-scm                    7.1.0

System information

• Cantera version: 2.6.0
• OS: Ubuntu 22.04.1 LTS
• Python 3.11.1 in a virtual environment

Additional context

While the documentation/build instructions are very good, building the development version (from commit 808023c) is annoying.

Consider not recommending to install "scons" to the Ubuntu system python (via apt). Alternatively, indicate that scons is a python package required for building Canetera. I've never used scons, installed it with "apt" as recommended, and didn't notice that it is a python package that installed to the Ubuntu "system" python. This resulted in "issues" until I realized I needed scons in my python virtual env.

Consider adding "doxygen" to the Ubuntu apt package list. scons doxygen will fail (on Ubuntu) without it.

Consider adding hdf5 include and library directory configuration options to the Common Options here. Like openblas, Ubuntu buries these under /usr/lib/x86_64-linux-gnu/ where they are not auto detected.

The build, test, install commands that worked for me are:

scons build blas_lapack_dir=/usr/lib/x86_64-linux-gnu/openblas-pthread/ sundials_include=~/local/include/sundials-6.5.0.ob sundials_libdir=~/local/lib/sundials-6.5.0.ob hdf_libdir=/usr/lib/x86_64-linux-gnu/hdf5/serial hdf_include=/usr/lib/x86_64-linux-gnu/hdf5/serial/include > build-$(date +"%Y%m%d-%H%M").log 2>&1

scons test
...
Tests passed: 4831
Up-to-date tests skipped: 0
Tests failed: 0

scons install prefix=~/.pyenv/versions/p3111
...
*************** Cantera 3.0.0a3 has been successfully installed ****************

File locations:

  library files               /home/ul/.pyenv/versions/3.11.1/envs/p3111/lib
  C++ headers                 /home/ul/.pyenv/versions/3.11.1/envs/p3111/include
  samples                     /home/ul/.pyenv/versions/3.11.1/envs/p3111/share/cantera/samples
  data files                  /home/ul/.pyenv/versions/3.11.1/envs/p3111/share/cantera/data
  input file converters       /home/ul/.pyenv/versions/3.11.1/envs/p3111/bin
  Python package              /home/ul/.pyenv/versions/3.11.1/envs/p3111/lib/python3.11/site-packages
  Python examples             /home/ul/.pyenv/versions/3.11.1/envs/p3111/share/cantera/samples/python

********************************************************************************

scons: done building targets.

Lastly, it's not uncommon to have separate libraries like sundials compiled with pthreads or openmp depending on if one is using openblas or MKL. I assume I need to be consistent with the blas and threading library (openblas and all pthreads in this instance); but I'm not an expert. If you have some insight about this (like "don't mix blas and threading libraries"), please include it in the build instructions.

HTH

[bryanwweber]

Hi @stuart-nolan! Thank you so much for this detailed report. If you have suggestions to edit the website, please suggest a pull request at https://github.com/cantera/cantera-website. All the pages are in the same structure as on the website, in the pages folder. They're all either Markdown or RestructuredText, so there's no database or anything that needs to get updated, it's all plain text. Thanks!

I'm going to move this issue over to the website repo so that we can keep track of it in the place where these suggestions will do the most good. Thanks again!

[ischoegl]

@bryanwweber ... thanks for transferring; I do, however, wonder whether part of this doesn't apply to Cantera/pypi-packages?

@stuart-nolan: regarding the HDF instructions, this is a very recent (and optional) addition; as it is still considered experimental in the current alpha version, it probably shouldn't be added on the website instructions yet. Your point is very well taken though.

[bryanwweber]

@ischoegl Ah, good point. I should have mentioned that I'm not surprised that 2.6.0 on PyPI doesn't support Python 3.11. There typically need to be updates to support new Python versions, for example, for Cython or (in this case) for setuptools. I think this is probably something that we'd only support for 3.0, not backport to 2.6.

[ischoegl]

I should have mentioned that I'm not surprised that 2.6.0 on PyPI doesn't support Python 3.11.

I overlooked that! I guess this answers the main underlying problem on this issue.

@stuart-nolan ... could you verify that pip install works in a virtual environment for a less recent Python version?

[stuart-nolan]

@stuart-nolan ... could you verify that pip install works in a virtual environment for a less recent Python version?

Sorry, I just removed all my older python versions recently. I suspect the install will still break for different reasons - sundials_config.h seems to be missing from the Cantera-2.6.0.tar.gz file downloaded by pip.

I installed cantera out of personal interest and curiosity so don't put too much weight on any of my suggestions. Even for reasonably savy users, it is probably best just to recommend installing via conda and provide some generic (non os or python implementation) dependent recommendations for dependencies.

I'm also unfamiliar with eigen. It seems eigen uses suitesparse for it's sparse solvers. As you may be aware, suitesparse (specifically CHOLMOD) is broken if using openblas. Hence, I do not recommend anyone else install as I did above - I don't know how likely it is for cantera to use the sparse solvers in eigen. MKL along with openmp for sundials and any other associated libraries should be safe.

One other 3.0.0a3 install quirk is that sphinxcontrib-matlabdomain is required for scons sphinx

Extension error:
Could not import extension sphinxcontrib.matlab (exception: No module named 'sphinxcontrib.matlab')
scons: *** [build/docs/sphinx/html/index.html] Error 2
scons: building terminated because of errors.

The conda requirements seemed to indicate this package was optional.

[bryanwweber]

I suspect the install will still break for different reasons

We do have wheels available for older versions of Python, which would hopefully work. Thanks for pointing out that file is missing, it should be generated and added but perhaps we missed it. Python's story for bundling dependencies or using system dependencies is sadly lacking here.

The conda requirements seemed to indicate this package was optional.

We generally consider building local documentation to be optional.

[speth]

Given we're on to Cantera 3.0.0 now, and we do have PyPI packages for Python 3.11, I think this can be closed.