ischoegl
commented
9 months ago Closing as there is insufficient information and I currently do not have the time to troubleshoot.
Closed ischoegl closed 9 months ago
ischoegl
commented
9 months ago Closing as there is insufficient information and I currently do not have the time to troubleshoot.
santoshshanbhogue
commented
9 months ago Hi @ischoegl - if it helps:
I looked at my config.log and compiled the file causing the error individually. here is what I get:
g++ -o .sconf_temp/conftest_a4c09ef5b1a9d7f569d7b807c37828ab_0_ed8c1642fe6cca76c1ce173236e40055 .sconf_temp/conftest_a4c09ef5b1a9d7f569d7b807c37828ab_0.o -L/Users/santosh1/opt/anaconda3/envs/py39/lib
ld: warning: ignoring file '/Users/santosh1/packages/canteraSource/.sconf_temp/conftest_a4c09ef5b1a9d7f569d7b807c37828ab_0.o': found architecture 'x86_64', required architecture 'arm64'
ld: warning: ignoring file '/Users/santosh1/opt/anaconda3/envs/py39/lib/libc++.1.0.dylib': found architecture 'x86_64', required architecture 'arm64'
ld: Undefined symbols:
_main, referenced from:
<initial-undefines>
clang: error: linker command failed with exit code 1 (use -v to see invocation)Is this because scons now (after XCode 15) points to the wrong architecture - x86_64 instead of arm64?
Thanks!
ischoegl
commented
9 months ago @santoshshanbhogue ... thanks, this is interesting. However, in my case, I am getting a different error (Xcode 15 broke my Ventura toolchain as well - see top). I should have the arm64 packages installed. I likewise compiled the offending code bit
#include <cmath>
#define Q(x) #x
#define QUOTE(x) Q(x)
#include <iostream>
int main(int argc, char** argv) {
std::cout << cos(0. * argc) << std::endl;
return 0;
}separately with the same arguments, and get the exact same error as the one shown on top for scons build (the linking step actually works, but the executable crashes).
If I omit the library -L/opt/homebrew/Caskroom/miniforge/base/envs/cantera-dev/lib in the linking step, things work. I reinstalled conda / miniforge etc. numerous times (on both my machines), and haven't found a constellation that works. The symptoms pretty much point at some dependency that is installed in the environment and doesn't play nice.
The main question I have is whether you were able to resolve things on your M1?
name: cantera-dev
channels:
- conda-forge
dependencies:
- python # Cantera supports Python 3.8 and up
- scons # build system
- boost-cpp # C++ dependency
- hdf5 # optional C++ dependency
- highfive # C++ dependency; uncomment to override Cantera default
- sundials # uncomment to override Cantera default
- fmt # uncomment to override Cantera default
- eigen # uncomment to override Cantera default
- yaml-cpp # uncomment to override Cantera default
# - libgomp # optional (OpenMP implementation when using GCC)
- cython # needed to build Python package
- numpy # needed to build Python package
- pip # needed to build Python package
- wheel # needed to build Python package
- setuptools # needed to build Python package
- pytest # needed for the Python test suite
# - pytest-cov # optional (needed if running with test coverage enabled)
- ruamel.yaml # needed for converter scripts
# - pandas # optional (needed for pandas interface)
# - scipy # optional (needed for some examples)
- matplotlib # optional (needed for plots)
- python-graphviz # optional (needed for reaction path diagrams)
- ipython # optional (needed for nicer interactive command line)
- jupyter # optional (needed for Jupyter Notebook)
- sphinx # optional (needed for documentation)
- pydata-sphinx-theme # optional (needed for documentation)
- sphinx-argparse # optional (needed for documentation)
- doxygen # optional (needed for documentation)
- graphviz # optional (needed for documentation)
- texlive-core # optional (needed for documentation)
- perl # optional (needed for documentation)
- pip: # optional (list of PyPI managed packages)
- sphinxcontrib-matlabdomain # optional (needed for documentation)
- sphinxcontrib-doxylink # optional (needed for documentation)
PS: I may post the output of conda info as well (this one is for Sonoma)
% conda info
active environment : cantera-dev
active env location : /opt/homebrew/Caskroom/miniforge/base/envs/cantera-dev
shell level : 2
user config file : /Users/ischoegl/.condarc
populated config files : /opt/homebrew/Caskroom/miniforge/base/.condarc
conda version : 23.3.1
conda-build version : not installed
python version : 3.10.12.final.0
virtual packages : __archspec=1=arm64
__osx=14.0=0
__unix=0=0
base environment : /opt/homebrew/Caskroom/miniforge/base (writable)
conda av data dir : /opt/homebrew/Caskroom/miniforge/base/etc/conda
conda av metadata url : None
channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64
https://conda.anaconda.org/conda-forge/noarch
package cache : /opt/homebrew/Caskroom/miniforge/base/pkgs
/Users/ischoegl/.conda/pkgs
envs directories : /opt/homebrew/Caskroom/miniforge/base/envs
/Users/ischoegl/.conda/envs
platform : osx-arm64
user-agent : conda/23.3.1 requests/2.31.0 CPython/3.10.12 Darwin/23.0.0 OSX/14.0
UID:GID : 501:20
netrc file : None
offline mode : FalseHere is where I am:
If I start with a clean conda environment (not even numpy) I get the same error that you get with python =3.9
If I start a clean conda environment with python=3.11, I don't get the linker error anymore that you are getting (and I was getting too). See this very basic cpp program that was tripping with python 3.9:
(py311) santosh1@gayatri tmp % clang++ -c test.cpp -o tm.o
(py311) santosh1@gayatri tmp % clang++ -o tm.out tm.o -L/Users/santosh1/opt/anaconda3/envs/py311/lib
(py311) santosh1@gayatri tmp % ./tm.out
hello world 1
1
(py311) santosh1@gayatri tmp % cat test.cpp
#include <iostream>
#include <cmath>
#define Q(x) #x
#define QUOTE(x) Q(x)
int main(int argc, char** argv) {
std::cout << "hello world " << argc << " " << std::endl;
std:: cout << cos(0 * argc) << std::endl;
}
(py311) santosh1@gayatri tmp %
Now, if I do scons build to compile cantera in my new python 3.11 environment, I get the output below:
(py311) santosh1@gayatri canteraSource % scons build
/Users/santosh1/opt/anaconda3/envs/py311/bin/scons:104: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
scons: Reading SConscript files ...
SCons 3.1.2 is using the following Python interpreter:
/Users/santosh1/opt/anaconda3/envs/py311/bin/python (Python 3.11)
INFO: Compiling on 'Apple M1 Pro'
INFO: Building Cantera from git commit '9696e27ea'
INFO: Configuration variables read from 'cantera.conf' and command line:
prefix = '/Users/santosh1/packages/ct_dev_version'
python_package = 'full'
matlab_toolbox = 'y'
matlab_path = '/Applications/MATLAB_R2021a.app/'
f90_interface = 'n'
system_eigen = 'n'
system_fmt = 'n'
system_sundials = 'n'
boost_inc_dir = '/usr/local/include/boost/'
INFO: Adding conda include and library paths: /Users/santosh1/opt/anaconda3/envs/py311
Checking for C++ header file cmath... yes
INFO: Using private installation of fmt library.
INFO: Using fmt version 9.1.0
Checking for YAML::Node().Mark()... no
INFO: Using private installation of yaml-cpp library.
Checking for C++ header file gtest/gtest.h... no
Checking for C++ header file gmock/gmock.h... no
INFO: Using Googletest from Git submodule
INFO: Using private installation of Eigen.
INFO: Found Eigen version 3.4.0
Checking whether __GLIBCXX__ is declared... no
Checking whether _LIBCPP_VERSION is declared... yes
Checking whether __clang__ is declared... yes
INFO: Not checking for OpenMP support due to using XCode compiler.
INFO: Found Boost version 1.76
Checking for CVodeCreate(CV_BDF, CV_NEWTON) in C++ library sundials_cvodes... no
Checking for CVodeCreate(CV_BDF) in C++ library sundials_cvodes... no
Checking for SUNContext ctx; SUNContext_Create(0, &ctx) in C++ library sundials_cvodes... no
Checking for double x; log(x) in C library None... yes
INFO: Using private installation of Sundials version 5.3.
Checking for C library hdf5... no
WARNING: Unexpected output while checking Python dependency versions:
| No module named 'numpy'
| No module named 'Cython'
INFO: NumPy not found.
INFO: Cython not found.
ERROR: Unable to build the full Python package because compatible versions of Numpy and/or Cython could not be found.The main finding here is that scons has progressed beyond the cmath stage, where it would fail:
Checking for C++ header file cmath... yes
But now, if I install numpy in my conda environment, it trips, even at the cmath stage:
(py311) santosh1@gayatri canteraSource % scons build
/Users/santosh1/opt/anaconda3/envs/py311/bin/scons:104: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import pkg_resources
scons: Reading SConscript files ...
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
Intel MKL WARNING: Support of Intel(R) Streaming SIMD Extensions 4.2 (Intel(R) SSE4.2) enabled only processors has been deprecated. Intel oneAPI Math Kernel Library 2025.0 will require Intel(R) Advanced Vector Extensions (Intel(R) AVX) instructions.
SCons 3.1.2 is using the following Python interpreter:
/Users/santosh1/opt/anaconda3/envs/py311/bin/python (Python 3.11)
INFO: Compiling on 'Apple M1 Pro'
INFO: Building Cantera from git commit '9696e27ea'
INFO: Configuration variables read from 'cantera.conf' and command line:
prefix = '/Users/santosh1/packages/ct_dev_version'
python_package = 'full'
matlab_toolbox = 'y'
matlab_path = '/Applications/MATLAB_R2021a.app/'
f90_interface = 'n'
system_eigen = 'n'
system_fmt = 'n'
system_sundials = 'n'
boost_inc_dir = '/usr/local/include/boost/'
INFO: Adding conda include and library paths: /Users/santosh1/opt/anaconda3/envs/py311
Checking for C++ header file cmath... yes
dyld[72010]: Symbol not found: __ZNKSt3__16locale9use_facetERNS0_2idE
Referenced from: <2F8C62B8-30CF-3928-8151-9E51D0414662> /Users/santosh1/packages/canteraSource/.sconf_temp/conftest_1
Expected in: <no uuid> unknown
sh: line 1: 72010 Abort trap: 6 .sconf_temp/conftest_1 > ".sconf_temp/conftest_1.out"
ERROR: The C++ compiler is not correctly configured (failed at linking stage).
See 'config.log' for details.
Hope this helps. I'm still debugging....
santoshshanbhogue
commented
9 months ago Latest update:
With my anaconda 3.11 environment, if I install numpy via
pip install numpy
instead of
conda install numpy or
conda install nomkl numpy
cantera is successfully compiling files. I have absolutely no idea why...
ischoegl
commented
9 months ago @santoshshanbhogue ... this is great news!
For myself, I sadly upgraded my other machine to Sonoma as well, and I don't get beyond the cmath step even in a bare conda environment (only python=3.11, cython, scons and boost-cpp; compiling with python_package=none).
PS: same result with cython and scons installed via pip in an environment that just contains python and boost-cpp. I cannot reduce further, as otherwise I'd lose Python itself ...
santoshshanbhogue
commented
9 months ago @ischoegl If I use boost-cpp from conda I get the error too. I used boost that is downloaded from boost.org
santoshshanbhogue
commented
9 months ago @ischoegl can you remove boost-cpp from your conda environment and see if you can get past the cmath error stage?
ischoegl
commented
9 months ago Interesting. Using a bare environment just with python=3.11, then pip install scons and cython gets me past the cmath step as well. Didn't bother installing boost (so there's a different failure), but that appears to be the culprit. Thanks!
santoshshanbhogue
commented
9 months ago To me it seems its the libcxx dependency that is causing it. Both numpy and boost-cpp install this sub package
ischoegl
commented
9 months ago I think your diagnosis is correct. It's not even boost: I can make the example crash with the same error in this simple environment:
% conda create -c conda-forge -n test fmtFwiw, the contents of the conda library are now just
% ls /opt/homebrew/Caskroom/miniforge/base/envs/test/lib
cmake libc++.a libfmt.10.1.0.dylib pkgconfig
libc++.1.0.dylib libc++.dylib libfmt.10.dylib
libc++.1.dylib libc++experimental.a libfmt.dylib@santoshshanbhogue How did you install your anaconda?
ischoegl
commented
9 months ago I guess I found a workaround as well: simply run
rm -f libc++*in the conda library after creating the full environment. I installed everything from conda as usual, and didn't need to resort to pip. So my takeaway is that the bug is in conda itself.
santoshshanbhogue
commented
9 months ago @ischoegl I installed anaconda many years ago by downloading the binaries from anaconda.org
Here's the latest on my end - I can successfully execute scons build and scons test and install via:
conda install -n py311 python=3.11
conda install cython ruamel.yaml
pip install numpy
pip install pytestand manually downloading boost v 1.83 from boost.org and passing the info to scons via boost_inc_dir
But here is the strange thing. Something seems off about the python package installation. See this output ( I am pressing the TAB key after ct. to see all the functions via command line completion)
(py311) santosh1@gayatri cantera % python
Python 3.11.5 (main, Sep 11 2023, 08:19:27) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import cantera as ct
>>> ct.__
ct.__annotations__ ct.__file__ ct.__init__( ct.__new__( ct.__sizeof__()
ct.__class__( ct.__format__( ct.__init_subclass__( ct.__package__ ct.__spec__
ct.__delattr__( ct.__ge__( ct.__le__( ct.__path__ ct.__str__()
ct.__dict__ ct.__getattribute__( ct.__loader__ ct.__reduce__() ct.__subclasshook__(
ct.__dir__( ct.__getstate__() ct.__lt__( ct.__reduce_ex__(
ct.__doc__ ct.__gt__( ct.__name__ ct.__repr__()
ct.__eq__( ct.__hash__() ct.__ne__( ct.__setattr__(
>>> ct.__
ct.__annotations__ ct.__file__ ct.__init__( ct.__new__( ct.__sizeof__()
ct.__class__( ct.__format__( ct.__init_subclass__( ct.__package__ ct.__spec__
ct.__delattr__( ct.__ge__( ct.__le__( ct.__path__ ct.__str__()
ct.__dict__ ct.__getattribute__( ct.__loader__ ct.__reduce__() ct.__subclasshook__(
ct.__dir__( ct.__getstate__() ct.__lt__( ct.__reduce_ex__(
ct.__doc__ ct.__gt__( ct.__name__ ct.__repr__()
ct.__eq__( ct.__hash__() ct.__ne__( ct.__setattr__(
>>> So there is no ct.Solution etc. I checked my PYTHON_PATH etc and there seems to be no problem there. I checked my cantera.conf to ensure I am doing the full python package too
bryanwweber
commented
9 months ago @santoshshanbhogue It looks like you've added the two __ before pressing tab: >>> ct.__ Is that why? What if you actually just type ct.Solution()? It'd also be useful to see the output of ct.__file__.
santoshshanbhogue
commented
9 months ago @bryanwweber python automatically added the two underscores because there are no other functions. Here is the output from the commands you suggested:
>>> ct.Solution()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: module 'cantera' has no attribute 'Solution'
>>> ct.__file__
>>> @santoshshanbhogue did you install Cantera to the environment or set your Python path to point to the built version?
ischoegl
commented
9 months ago @santoshshanbhogue ... I cannot reproduce this behavior on my end after running scons install. I get
>>> ct.Solution()
>>> <cantera.composite.Solution at 0x107743ed0>Also, the entire test suite passes.
PS: I get
>>> ct.__file__
>>> '/opt/homebrew/Caskroom/miniforge/base/envs/cantera-dev/lib/python3.10/site-packages/cantera/__init__.py'@bryanwweber and @ischoegl I think something was wrong with my PYTHONPATH. After I fix it, things work fine
So all good now
Thanks!!
bryanwweber
commented
9 months ago @ischoegl Thanks for the follow up!
@santoshshanbhogue So scons install did not install into a directory inside your conda environment. You must have set prefix or some other variable in your cantera.conf file which produced that install structure. If you want to export something on your PYTHONPATH, it needs to be PYTHONPATH=/Users/santosh1/packages/ct_dev_version/lib/python3.11/site-packages, the whole path to site-packages.
santoshshanbhogue
commented
9 months ago @bryanwweber so I manually set my PYTHONPATH to the right package. But here is my cantera.conf. Should I be doing something different?
prefix = '/Users/santosh1/packages/ct_dev_version'
python_package = 'full'
matlab_toolbox = 'y'
matlab_path = '/Applications/MATLAB_R2021a.app/'
f90_interface = 'n'
system_eigen = 'n'
system_fmt = 'n'
system_sundials = 'n'
boost_inc_dir = '/Users/santosh1/packages/boost_1_83_0'No, you don't need to do anything differently as long as you don't mind setting PYTHONPATH. But you can also delete the prefix variable and Cantera will install to the right place
anthony-walker
commented
8 months ago @ischoegl's work around worked for me, I am on MacOS 14.1 (Sonoma).
rm -f $CONDA_PREFIX/lib/libc++*As a note, read on a MacOS that part of this issue was due to XCode and it is fixed with the beta XCode 15.1, I however was not able to successfully replicate this fix.
speth
commented
8 months ago I'd be curious whether things work if you install Clang++ (conda package clangxx via conda-forge) and specify it as the compiler when building Cantera (set CC=clang and CXX=clang++). This of course would only matter for a Conda environment where you haven't broken the libcxx package by manually deleting files.
santoshshanbhogue
commented
8 months ago @speth I tried it, but get an error. What I did was create a new conda environment, install all packages - clangxx, cython, scons, humpy and hdf5 via conda (clangxx via conda-forge as you specified). I had to specify the new compiler location as well has include paths via CPATH and LD_LIBRARY_PATH. See the config.log below
cat config.log
file /Users/santosh1/packages/canteraSource/SConstruct,line 1128:
Configure(confdir = .sconf_temp)
scons: Configure: Checking for C++ header file cmath...
scons: Configure: ".sconf_temp/conftest_0.cpp" is up to date.
scons: Configure: The original builder output was:
|.sconf_temp/conftest_0.cpp <-
| |
| |#include <cmath>
| |
| |
|
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/clang++ -o .sconf_temp/conftest_0.o -c -isystem /Users/santosh1/opt/anaconda3/envs/py311-cxx/include -isystem /Users/santosh1/packages/boost_1_83_0 -std=c++17 -fcolor-diagnostics -O3 -g -DNDEBUG .sconf_temp/conftest_0.cpp
In file included from .sconf_temp/conftest_0.cpp:2:
In file included from /Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/cmath:317:
In file included from /Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:316:
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/stdlib.h:150:34: error: unknown type name 'ldiv_t'
inline _LIBCPP_INLINE_VISIBILITY ldiv_t div(long __x, long __y) _NOEXCEPT {
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/stdlib.h:151:12: error: no member named 'ldiv' in the global namespace
return ::ldiv(__x, __y);
~~^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/stdlib.h:154:34: error: unknown type name 'lldiv_t'
inline _LIBCPP_INLINE_VISIBILITY lldiv_t div(long long __x,
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/stdlib.h:156:12: error: no member named 'lldiv' in the global namespace
return ::lldiv(__x, __y);
~~^
In file included from .sconf_temp/conftest_0.cpp:2:
In file included from /Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/cmath:317:
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:388:31: error: use of undeclared identifier 'FP_NAN'
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL, FP_SUBNORMAL, FP_ZERO, __x);
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:388:39: error: use of undeclared identifier 'FP_INFINITE'
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL, FP_SUBNORMAL, FP_ZERO, __x);
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:388:52: error: use of undeclared identifier 'FP_NORMAL'
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL, FP_SUBNORMAL, FP_ZERO, __x);
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:388:63: error: use of undeclared identifier 'FP_SUBNORMAL'
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL, FP_SUBNORMAL, FP_ZERO, __x);
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:388:77: error: use of undeclared identifier 'FP_ZERO'
return __builtin_fpclassify(FP_NAN, FP_INFINITE, FP_NORMAL, FP_SUBNORMAL, FP_ZERO, __x);
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:393:21: error: use of undeclared identifier 'FP_ZERO'
return __x == 0 ? FP_ZERO : FP_NORMAL;
^
/Users/santosh1/opt/anaconda3/envs/py311-cxx/bin/../include/c++/v1/math.h:393:31: error: use of undeclared identifier 'FP_NORMAL'
return __x == 0 ? FP_ZERO : FP_NORMAL;
^
11 errors generated.
scons: Configure: noIssue persists on macOS 14.3 and Xcode 15.2 (for conda 23.3.1 as well as 23.11.0); solution is still to manually remove
rm -f $CONDA_PREFIX/lib/libc++*A related issue - I recently explored using XCode to build cantera code with cpp. I had to set my architecture as Intel (Rosetta) for it to work. The specific error was that the linker couldn't find arm64 in libcantera_shared.3.1.0.dylib, only x86_64 (related to the -lcantera_shared setting in the linker flags)
Problem description
Compilation on Xcode 15.0 fails.
Steps to reproduce
brew install miniforge(orbrew install miniconda)scons buildBehavior
On Ventura:
On Sonoma:
System information
mainbrew install miniforge) + Apple clang version 15.0.0 (clang-1500.0.40.1), xcode-select version 2397 (CLI tools only)Other thoughts
This is presumably a conda issue that may have to be resolved upstream. At the same time, compilations still work for other projects.