MacOS Monterey: import libcellml gives errors

[aditya-ml]

I installed libcellml-0.3.2.dev1 using pip. However, importing it generates the following error and I can't find any online resource on this issue.

Traceback (most recent call last):
  File "/Users/adityasinghal/PycharmProjects/combine-notebooks/src/combine_notebooks/example_cellml.py", line 13, in <module>
    import libcellml
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/__init__.py", line 12, in <module>
    from libcellml.analyser import Analyser
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/analyser.py", line 13, in <module>
    from . import _analyser
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/_analyser.so, 0x0002): Library not loaded: @rpath/Python
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/_analyser.so
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/./Python' (no such file), '/usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/./Python' (no such file), '/usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/Python' (no such file), '/usr/lib/Python' (no such file)

Process finished with exit code 1

Lines 11-20 in analyser.py

# Import the low-level C/C++ module
if __package__ or "." in __name__:
    from . import _analyser
else:
    import _analyser

try:
    import builtins as __builtin__
except ImportError:
    import __builtin__

Please guide.

[hsorby]

Does pip install libcellml==0.2.0 result in the same outcome?

This could be a problem with the rpath settings in the libCellML package. If you create a virtual environment:

python -m venv venv
source venv/bin/activate
pip install libcellml
python
>>> import libcellml

Do bad things happen?

[aditya-ml]

Does pip install libcellml==0.2.0 result in the same outcome? It results in error:

ERROR: Could not find a version that satisfies the requirement libcellml==0.2.0 (from versions: 0.2.0.dev17, 0.2.0.dev18, 0.2.0.dev19, 0.2.0.dev20, 0.2.0.dev21, 0.3.2.dev1, 0.3.2.dev2)
ERROR: No matching distribution found for libcellml==0.2.0

I'm already working in a virtual environment, tried your suggestion in a new venv, got same errors.

This could be a problem with the rpath settings in the libCellML package. It seems like that, but any hints on how can I resolve it?

[hsorby]

It is trying to find the Python library. You can try and set DYLD_LIBRARY_PATH to help it. I'm not sure where this library is but with bash you can try:

export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.9/lib/

[hsorby]

or maybe:

export DYLD_LIBRARY_PATH=/Library/Frameworks/Python.framework/

[matthiaskoenig]

Just to add to the information. This is an M1 laptop and the build has probably not been tested on the M1. This could be related to this.

[matthiaskoenig]

Lucian Smith solved the builds of roadrunner for the M1, so he could help providing builds for the M1. Basically all new Macs will not work with (python) libcellml at the moment.

[hsorby]

We are trying to get an ARM based Mac so we can test our applications, currently though we don't have access to one. We have one on order ...

[hsorby]

We now have a macOS M1 in our possession, I will work on adding it to our CI over the coming days. When we next meet for cellml (Tuesday NZDT) we will review and determine how we are placed for making a new release with macOS M1 support.

[matthiaskoenig]

Thanks. This sounds great.

[hsorby]

libcellml-0.2.0.dev21-cp39-cp39-macosx_11_0_arm64.whl.zip

I have (sort of) manually created this wheel with GitHub actions could you try it out and let me know if it works. You will need to remove the .zip extension that I added to the filename so GitHub would upload the file.

[matthiaskoenig]

We tried this right now by copying the folders in the virtualenv python3.9/site-packages folder of the virtual environment. pip is recognizing the package, i.e. pip list | grep cellml gives libcellml 0.2.0.dev21 but we still get the following error

"/Applications/Python 3.9/IDLE.app/Contents/MacOS/Python" /Users/adityasinghal/PycharmProjects/combine-notebooks/src/combine_notebooks/example_cellml.py
Traceback (most recent call last):
  File "/Users/adityasinghal/PycharmProjects/combine-notebooks/src/combine_notebooks/example_cellml.py", line 13, in <module>
    import libcellml
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/__init__.py", line 12, in <module>
    from libcellml.analyser import Analyser
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/analyser.py", line 13, in <module>
    from . import _analyser
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/_analyser.so, 0x0002): Library not loaded: @rpath/Python
  Referenced from: /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/_analyser.so
  Reason: tried: '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/./Python' (no such file), '/usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/Python' (no such file), '/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/libcellml/./Python' (no such file), '/usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/Python' (no such file), '/usr/lib/Python' (no such file)

Not sure how else to test this, but seems to be the same issue as before

[hsorby]

Okay would you mind trying out this wheel (again I have added the .zip extension, to allow uploading the file, which will need to be removed). libcellml-0.3.99rc2-cp39-cp39-macosx_12_0_arm64.whl.zip

[matthiaskoenig]

@aditya-ml Can you try the new version and give feedback. I have no access to a Mac M1. Just make sure to remove the old folder from your virtualenv

[aditya-ml]

I removed the old folders and copied the new ones from the wheel. Still getting the same error.

[hsorby]

This is what I get when I create a new virtual environment for Python 3.9 and install libCellML with pip.

libcellml@bn438065 tmp % /opt/homebrew/Frameworks/Python.framework/Versions/3.9/bin/python3 -m venv venv_test_install
libcellml@bn438065 tmp % . venv_test_install/bin/activate
(venv_test_install) libcellml@bn438065 tmp % 
(venv_test_install) libcellml@bn438065 tmp % 
(venv_test_install) libcellml@bn438065 tmp % pip install libcellml
Collecting libcellml
  Downloading libcellml-0.3.100-cp39-cp39-macosx_11_0_arm64.whl (1.6 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.6/1.6 MB 35.0 MB/s eta 0:00:00
Installing collected packages: libcellml
Successfully installed libcellml-0.3.100
WARNING: There was an error checking the latest version of pip.
(venv_test_install) libcellml@bn438065 tmp % python
Python 3.9.13 (main, May 24 2022, 21:13:51) 
[Clang 13.1.6 (clang-1316.0.21.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import libcellml
>>> libcellml.versionString()
'0.3.100'
>>> libcellml.__file__
'/Users/libcellml/Documents/tmp/venv_test_install/lib/python3.9/site-packages/libcellml/__init__.py'
>>> 
(venv_test_install) libcellml@bn438065 tmp % otool -L /Users/libcellml/Documents/tmp/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so 
/Users/libcellml/Documents/tmp/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so:
    @rpath/libcellml.0.3.100.dylib (compatibility version 0.0.0, current version 0.3.100)
    @rpath/Python.framework/Versions/3.9/Python (compatibility version 3.9.0, current version 3.9.0)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
    /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 1300.23.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1311.100.3)
(venv_test_install) libcellml@bn438065 tmp % otool -l /Users/libcellml/Documents/tmp/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so | grep -A3 RPATH
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/. (offset 12)
Load command 17
--
          cmd LC_RPATH
      cmdsize 40
         path /opt/homebrew/Frameworks (offset 12)
(venv_test_install) libcellml@bn438065 tmp % 

[hsorby]

Could you run through these commands and share the output here, I am hoping this will help me figure out what is going wrong.

Commands:

python3 -m venv venv_test_install
source venv_test_install/bin/activate
pip install libcellml
python
>>> import libcellml
>>> libcellml.versionString()
>>> libcellml.__file__
<location of __init__.py file>
>>> exit()
otool -L <lcoation of __init__.py file (less the __init__.py part>/_analyser.so
otool -l <lcoation of __init__.py file (less the __init__.py part>/_analyser.so | grep -A3 RPATH

Thanks.

[aditya-ml]

Here's what I get: (pythonProject) adityasinghal@Adityas-MacBook-Air pythonProject % python3 -m venv venv_test_install

(pythonProject) adityasinghal@Adityas-MacBook-Air pythonProject % source venv_test_install/bin/activate

(venv_test_install) adityasinghal@Adityas-MacBook-Air pythonProject % pip install libcellml Collecting libcellml Downloading libcellml-0.3.2.dev2-cp39-cp39-macosx_10_15_x86_64.whl (1.7 MB) |████████████████████████████████| 1.7 MB 11.8 MB/s Installing collected packages: libcellml Successfully installed libcellml-0.3.2.dev2

(venv_test_install) adityasinghal@Adityas-MacBook-Air pythonProject % python Python 3.9.0 (v3.9.0:9cf6752276, Oct 5 2020, 11:29:23) [Clang 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information.

>>> import libcellml Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/__init__.py", line 12, in <module> from libcellml.analyser import Analyser File "/Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/analyser.py", line 13, in <module> from . import _analyser ImportError: dlopen(/Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so, 0x0002): Library not loaded: @rpath/Python Referenced from: /Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so Reason: tried: '/Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/./Python' (no such file), '/usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/Python' (no such file), '/Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/./Python' (no such file), '/usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/Python' (no such file), '/usr/lib/Python' (no such file)

>>> libcellml.versionString() Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'libcellml' is not defined

>>> libcellml.__file__ Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'libcellml' is not defined

>>> exit() (venv_test_install) adityasinghal@Adityas-MacBook-Air pythonProject %

The package is getting installed but importing it throws errors.

[hsorby]

And what does the output of:

otool -L <lcoation of __init__.py file (less the __init__.py part>/_analyser.so
otool -l <lcoation of __init__.py file (less the __init__.py part>/_analyser.so | grep -A3 RPATH

give you?

[hsorby]

or I think in your case:

otool -L /Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so

and

otool -l /Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so | grep -A3 RPATH

[hsorby]

One thing I have just noticed is that you are getting libcellml-0.3.2.dev2 where I am getting libcellml-0.3.100. That is strange.

[hsorby]

If you were to do pip install -U libcellml do you get a different version?

[aditya-ml]

or I think in your case:

otool -L /Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so

and

otool -l /Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so | grep -A3 RPATH

/Users/adityasinghal/PycharmProjects/pythonProject/venv_test_install/lib/python3.9/site-packages/libcellml/_analyser.so:
        @rpath/libcellml.0.3.2-dev.2.dylib (compatibility version 0.0.0, current version 0.3.2)
        @rpath/Python (compatibility version 3.9.0, current version 3.9.0)
        /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
        /usr/lib/libxml2.2.dylib (compatibility version 10.0.0, current version 10.9.0)
        /usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 904.4.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1292.60.1)

and

          cmd LC_RPATH
      cmdsize 32
         path @loader_path/. (offset 12)
Load command 16
--
          cmd LC_RPATH
      cmdsize 80
         path /usr/local/opt/python/Frameworks/Python.framework/Versions/3.9/ (offset 12)

If you were to do pip install -U libcellml do you get a different version?

No, still get the same Requirement already up-to-date: libcellml in ./venv_test_install/lib/python3.9/site-packages (0.3.2.dev2)

[hsorby]

I think we have now fixed this issue, please re-open if I am wrong about this.

[matthiaskoenig]

Thanks a lot. @aditya-ml Please check if this solves the issue.