ImportError for `highspy`

[goroderickgo]

Hello, I'm on an M1 Mac (MacOS Ventura, Python 3.10.9) and downloaded HiGHS via PyPI (highspy) and am getting this import error.

I didn't seen any similar issues asked previously, so I figured I'd ask here. Appreciate any help!

Python 3.10.9 (main, Jan 11 2023, 09:18:20) [Clang 14.0.6 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import highspy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/e3-work/opt/anaconda3/envs/nmt-pins/lib/python3.10/site-packages/highspy/__init__.py", line 1, in <module>
    from .highs import (
  File "/Users/e3-work/opt/anaconda3/envs/nmt-pins/lib/python3.10/site-packages/highspy/highs.py", line 1, in <module>
    from .highs_bindings import (
ImportError: dlopen(/Users/e3-work/opt/anaconda3/envs/nmt-pins/lib/python3.10/site-packages/highspy/highs_bindings.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN5Highs10clearModelEv'

[mckib2]

MacOS will use Clang to compile C++ sources, so we can use llvm-cxxfilt to find out exactly what it's complaining about:

$ llvm-cxxfilt --strip-underscore '__ZN5Highs10clearModelEv'
Highs::clearModel()

Looks like the library on PyPI either didn't compile Highs.cpp (unlikely) or something is not linked correctly (more likely) -- might be due to how conda is handling this wheel? Try installing with pip to see if that resolves it.

If it doesn't, as a hacky workaround, try adding the path to libhighs: sys.path.append("/Users/e3-work/opt/anaconda3/envs/nmt-pins/lib/python3.10/site-packages/highspy/.dylibs/") (also make sure there is indeed a library named libhighs.1.5.0.dylib somewhere in /Users/e3-work/opt/anaconda3/envs/nmt-pins/lib/python3.10/site-packages/highspy)

[jajhall]

We've heard of people installing HiGHS via (ana)conda, but we only support installation via pip.

[chkwon]

I installed via pip and obtained the same error:

macbook: ~  % pip install highspy
Collecting highspy
  Using cached highspy-1.5.0.dev0-cp310-cp310-macosx_10_9_x86_64.whl (236 kB)
Collecting pybind11
  Using cached pybind11-2.10.3-py3-none-any.whl (222 kB)
Collecting pyomo
  Using cached Pyomo-6.4.4-cp310-cp310-macosx_10_15_x86_64.whl (4.8 MB)
Collecting numpy
  Using cached numpy-1.24.2-cp310-cp310-macosx_10_9_x86_64.whl (19.8 MB)
Collecting ply
  Using cached ply-3.11-py2.py3-none-any.whl (49 kB)
Installing collected packages: ply, pyomo, pybind11, numpy, highspy
Successfully installed highspy-1.5.0.dev0 numpy-1.24.2 ply-3.11 pybind11-2.10.3 pyomo-6.4.4
macbook: ~  % python3
Python 3.10.9 (main, Dec 15 2022, 18:25:35) [Clang 14.0.0 (clang-1400.0.29.202)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import highspy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.10/site-packages/highspy/__init__.py", line 1, in <module>
    from .highs import (
  File "/usr/local/lib/python3.10/site-packages/highspy/highs.py", line 1, in <module>
    from .highs_bindings import (
ImportError: dlopen(/usr/local/lib/python3.10/site-packages/highspy/highs_bindings.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN5Highs10clearModelEv'
>>>

I am on Intel macOS 13.

[jajhall]

@goroderickgo Does the recent push to latest fix your issue?

[chkwon]

@goroderickgo You will need to compile and install manually

git clone https://github.com/ERGO-Code/HiGHS.git
cd HiGHS
git checkout latest
mkdir build 
cd build 
cmake .. 
make
make install 
pip install ../src/interfaces/highspy/

[FlorianWilhelm]

I encountered the same problem on MacOS Ventura, Python 3.8.16 on an old Intel Mac.

Following the manual compilation & installation instructions from @chkwon solved the problem. Thank you! Will there be a new pip release soon?

[galabovaa]

There will be a new pip release soon, but we can not say exact timing. We need to finish some features we are currently implementing and some features we are adding to the python interface.

[jajhall]

@germannp writes (in former issue #1189)

After > pip install highspy I get the following problem:

> python
Python 3.9.7 (default, Sep 16 2021, 16:59:28) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32

Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated.  Libraries may fail to load.  To activate this environment
please see https://conda.io/activation

@germannp writes (in former issue #1189)

Type "help", "copyright", "credits" or "license" for more information.
>>> import highspy
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\u228039\Anaconda3\lib\site-packages\highspy\__init__.py", line 30, in <module>
    from .highs import (
  File "C:\Users\u228039\Anaconda3\lib\site-packages\highspy\highs.py", line 1, in <module>
    from .highs_bindings import (
ImportError: DLL load failed while importing highs_bindings: The specified procedure could not be found.
>>>

Any ideas?

[rschwarz]

I see yet another, similar error:

AttributeError: function/symbol 'Highs_destroy' not found in library 'C:\hostedtoolcache\windows\Python\3.10.9\x64\lib\site-packages\highspy\highs_bindings.cp310-win_amd64.pyd': error 0x7f

This is for Python 3.10.9, highspy 1.5.0.dev0 on windows-2022 in a GitHub action.

[jajhall]

@galabovaa is working on the Python build (and other things) at the moment: thanks for your patience

[jjcherian]

I noticed there was a release a week ago - is this issue supposed to have been resolved in that one? I'm unfortunately still having the same problem when importing highspy as @chkwon - thanks!

[jajhall]

That release was only on the 'latest' branch. We expect to push a release to 'master' by the end of the week.

[jjcherian]

Just as an FYI, this problem isn't present (for me at least) on the master branch if I compile from source - can you point me to the commit that fixed this issue on the 'latest' branch?

Also apologies if this is available somewhere, but what changed in the 1.5.3 release? There's no source files on pip and I don't see a version here on GitHub.

[jajhall]

Sorry, I don't know when this was fixed. Keeping track of such changes is too time-consuming for us.

[galabovaa]

We intend to keep track of such changes but at a later stage, once we have finished implementing some features.

[galabovaa]

Please check out the latest version of HiGHS 1.5.3 and open a new issue if you run into any further problems! Version 1.5.3 introduces major changes to the build system and the structure of the python interface.

[rschwarz]

After updating highspy to 1.5.3, I do still see the same error as mentioned above (https://github.com/ERGO-Code/HiGHS/issues/1119#issuecomment-1470288576).

See for example in this GitHub action run.

[jjcherian]

I suspect that #1328 explains this issue - taking a look on PyPI, it looks like the MacOS wheel for every version after 1.1.2dev3 has this problem (and that's also the only release that doesn't trigger this ImportError).

[alspadoni]

Hello, I too have the same issues with version 1.5.3 on MacOS.

I understand this may not be a priority for the dev team right now. My team is currently evaluating the adoption of highs for our next project and this is the only blocking issue as of now. Would you be able to tell me a general idea of when one can expect this issue to be addressed/solved?

Thank you.

[galabovaa]

Hi @alspadoni, this actually is with high priority. I can not give you an exact time, but we expect to have it resolved sooner rather than later.

[jajhall]

Closed by #1343

[robingirard]

I still have the same error ( symbol not found in flat namespace '__ZN5Highs10clearModelEv') on mac with pip install of version 1.5.3.

[jajhall]

Sorry, @galabovaa has been working on the PyPI build. Hopefully she can advise you

[rschwarz]

Version 1.5.3 is from May and (I think) does not include the latest attempts to fix things, so I guess this is still a matter of waiting for the next release.

[ZedongPeng]

@goroderickgo You will need to compile and install manually

git clone https://github.com/ERGO-Code/HiGHS.git
cd HiGHS
git checkout latest
mkdir build 
cd build 
cmake .. 
make
make install 
pip install ../src/interfaces/highspy/

Hi @chkwon . Thanks for your answer and I followed your instruction. After installing HIGHS, it failed when trying pip install ../src/interfaces/highspy/. Here is the files I have in ../src/interfaces/.

-rw-r--r--  1 zedongpeng  staff    55K Aug  9 15:26 highs_c_api.cpp
-rw-r--r--  1 zedongpeng  staff    85K Aug  9 15:26 highs_c_api.h
-rw-r--r--  1 zedongpeng  staff    31K Aug  9 15:25 highs_csharp_api.cs
-rw-r--r--  1 zedongpeng  staff    33K Aug  9 15:25 highs_fortran_api.f90
-rw-r--r--@ 1 zedongpeng  staff   8.0K Aug  9 15:25 highs_python_api.py

No folder names highspy. Looking forward to your response.

[chkwon]

@ZedongPeng I think they changed the directory structure.

Try pip install ..

[jajhall]

We'll publish this as soon as the process is fixed for all architectures. It's a major priority

[richardmahuze]

Hi,

I have the same error:

ImportError: dlopen(…opt/anaconda3/envs/test/lib/python3.9/site-packages/highspy/highs_bindings.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '__ZN5Highs10clearModelEv'

I’m using Apple M2 chip and installed highspy using pip.

[IvoW10]

Hi,

I have the same error:

ImportError: dlopen(…opt/anaconda3/envs/test/lib/python3.9/site-packages/highspy/highs_bindings.cpython-39-darwin.so, 0x0002): symbol not found in flat namespace '__ZN5Highs10clearModelEv'

I’m using Apple M2 chip and installed highspy using pip.

For me the following did the trick:

git clone https://github.com/ERGO-Code/HiGHS.git
cd HiGHS
git checkout master
mkdir build 
cd build 
cmake .. 
make
make install 
pip install .. 

[IvoW10]

I faced issues with this again. No installation that we tried works for Mac (also with friends not), the version that does work is highspy==1.1.2.dev3.

[galabovaa]

Our latest pre-release 1.7.1.dev1 should be stable now.

https://pypi.org/project/highspy/#history

[ZedongPeng]

Thank you. @galabovaa

[IvoW10]

Yes this works perfectly! It installed highs as expected on my Mac now as well. Thanks for the effort!