search

jdber1 / opendrop

OpenDrop pendant drop tensiometry software
GNU General Public License v3.0
31 stars 20 forks source link

error using pip to install Opendrop (step 4) #43

Open ranit-mukherjee opened 2 years ago

ranit-mukherjee commented 2 years ago

Hi,

I am trying to install Opendrop in my Mac (M1 chip). I have gone through step 4 (installing Boost and Sundials) successfully. But I keep getting this error (full error file attached):

Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [27 lines of output] Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError:

Can you please guide me here? Opendrop error.pdf

eugenhu commented 2 years ago

Hmm, I'm not exactly sure why it's not finding the 'SCons' module. Can you try manually installing scons first, with:

python3 -m pip install SCons

And then try installing opendrop again, this time from the development branch:

python3 -m pip install git+https://github.com/jdber1/opendrop.git@development
ranit-mukherjee commented 2 years ago

Hi Eugen,

I tried the following. The error changed but still fails to install Opendrop. Pasting the error here-

Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [28 lines of output] Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: /opt/homebrew/Cellar/python@3.10/3.10.6_2/bin/python3.10 -m cython -3 --cplus -o opendrop/features/.checkpoints/colorize.cpp opendrop/features/colorize.pyx Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: /opt/homebrew/Cellar/python@3.10/3.10.6_2/bin/python3.10: No module named cython scons: *** [opendrop/features/.checkpoints/colorize.cpp] Error 1 Traceback (most recent call last): File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 156, in prepare_metadata_for_build_wheel hook = backend.prepare_metadata_for_build_wheel AttributeError: module 'buildapi' has no attribute 'prepare_metadata_for_build_wheel'

  During handling of the above exception, another exception occurred:

  Traceback (most recent call last):
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in <module>
      main()
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 160, in prepare_metadata_for_build_wheel
      whl_basename = backend.build_wheel(metadata_directory, config_settings)
    File "/private/var/folders/zf/kkc7k0zx10v_68gm9lws56pr0000gn/T/pip-req-build-zmtcb_5i/build_backend/buildapi.py", line 12, in build_wheel
      subprocess.run(['scons', '-Q', 'bdist_wheel', '--build-dir=' + wheel_directory], check=True, env=env)
    File "/opt/homebrew/Cellar/python@3.10/3.10.6_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 524, in run
      raise CalledProcessError(retcode, process.args,
  subprocess.CalledProcessError: Command '['scons', '-Q', 'bdist_wheel', '--build-dir=/private/var/folders/zf/kkc7k0zx10v_68gm9lws56pr0000gn/T/pip-modern-metadata-6cgojln9']' returned non-zero exit status 2.
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: metadata-generation-failed

× Encountered error while generating package metadata. ╰─> See above for output.

note: This is an issue with the package mentioned above, not pip. hint: See above for details.

ranit-mukherjee commented 2 years ago

I installed the cython module as stated in the above error, but it still failed. The new error message is this:

g++ -o opendrop/fit/younglaplace/.checkpoints/shape.os -c -O3 -std=c++14 -fPIC -Iinclude -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 opendrop/fit/younglaplace/.checkpoints/shape.cpp In file included from opendrop/fit/younglaplace/.checkpoints/shape.cpp:763: include/opendrop/younglaplace.hpp:9:10: fatal error: 'arkode/arkode_erkstep.h' file not found

include <arkode/arkode_erkstep.h>

           ^~~~~~~~~~~~~~~~~~~~~~~~~
  1 error generated.
  scons: *** [opendrop/fit/younglaplace/.checkpoints/shape.os] Error 1
  Traceback (most recent call last):
    File "/opt/homebrew/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 156, in prepare_metadata_for_build_wheel
      hook = backend.prepare_metadata_for_build_wheel
  AttributeError: module 'buildapi' has no attribute 'prepare_metadata_for_build_wheel'
eugenhu commented 2 years ago

Do you know where the sundials headers were installed? E.g. on my linux system, "arkode_erkstep.h" is in "/usr/include/arkode/arkode_erkstep.h". Could yours be in "/usr/local/include/arkode/arkode_erkstep.h"?

eugenhu commented 2 years ago

Possibly, it's in "/opt/homebrew/include/arkode/arkode_erkstep.h". It looks like Homebrew uses the "/opt/homebrew" prefix now for Apple Silicon platforms.

ranit-mukherjee commented 2 years ago

it is in "/opt/homebrew/Cellar/sundials/6.3.0/include/arkode/arkode_erkstep.h".

eugenhu commented 2 years ago

Interesting, could you try running:

CPATH=/opt/homebrew/include python3 -m pip install git+https://github.com/jdber1/opendrop.git@development

And if that fails, try:

CPATH=/opt/homebrew/Cellar/sundials/6.3.0/include python3 -m pip install git+https://github.com/jdber1/opendrop.git@development

CPATH will tell gcc where to look for header files.

ranit-mukherjee commented 2 years ago

Ran both, but failed both of the times. The error message is still similar-

(base) ranit@Ranits-MacBook-Air ~ % C_INCLUDE_PATH=/opt/homebrew/include python3 -m pip install git+https://github.com/jdber1/opendrop.git@development Collecting git+https://github.com/jdber1/opendrop.git@development Cloning https://github.com/jdber1/opendrop.git (to revision development) to /private/var/folders/zf/kkc7k0zx10v_68gm9lws56pr0000gn/T/pip-req-build-lr9er6bu Running command git clone --filter=blob:none --quiet https://github.com/jdber1/opendrop.git /private/var/folders/zf/kkc7k0zx10v_68gm9lws56pr0000gn/T/pip-req-build-lr9er6bu Running command git checkout -b development --track origin/development Switched to a new branch 'development' branch 'development' set up to track 'origin/development'. Resolved https://github.com/jdber1/opendrop.git to commit e18fbf3d7d6f94d02f4846400be518a56b86e73c Installing build dependencies ... done Getting requirements to build wheel ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error

× Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [43 lines of output] Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: /opt/homebrew/Cellar/python@3.10/3.10.6_2/bin/python3.10 -m cython -3 --cplus -o opendrop/features/.checkpoints/colorize.cpp opendrop/features/colorize.pyx Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: g++ -o opendrop/features/.checkpoints/colorize.os -c -O3 -std=c++14 -fPIC -Iinclude -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 opendrop/features/.checkpoints/colorize.cpp g++ -o opendrop/features/colorize.cpython-310-darwin.so -dynamiclib opendrop/features/.checkpoints/colorize.os -L/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib -lpython3.10 /opt/homebrew/Cellar/python@3.10/3.10.6_2/bin/python3.10 -m cython -3 --cplus -o opendrop/fit/needle/.checkpoints/hough.cpp opendrop/fit/needle/hough.pyx Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: g++ -o opendrop/fit/needle/.checkpoints/hough.os -c -O3 -std=c++14 -fPIC -Iinclude -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 opendrop/fit/needle/.checkpoints/hough.cpp g++ -o opendrop/fit/needle/hough.cpython-310-darwin.so -dynamiclib opendrop/fit/needle/.checkpoints/hough.os -L/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib -lpython3.10 -lm /opt/homebrew/Cellar/python@3.10/3.10.6_2/bin/python3.10 -m cython -3 --cplus -o opendrop/fit/younglaplace/.checkpoints/shape.cpp opendrop/fit/younglaplace/shape.pyx Error in sitecustomize; set PYTHONVERBOSE for traceback: AssertionError: g++ -o opendrop/fit/younglaplace/.checkpoints/shape.os -c -O3 -std=c++14 -fPIC -Iinclude -I/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/include/python3.10 opendrop/fit/younglaplace/.checkpoints/shape.cpp In file included from opendrop/fit/younglaplace/.checkpoints/shape.cpp:763: include/opendrop/younglaplace.hpp:9:10: fatal error: 'arkode/arkode_erkstep.h' file not found

include <arkode/arkode_erkstep.h>

eugenhu commented 2 years ago

I see, can you try using CPATH instead of C_INCLUDE_PATH. I've edited my previous comment, I think C_INCLUDE_PATH is for C only and not C++, apologies.

ranit-mukherjee commented 2 years ago

Thank you for your help! I changed it to CPATH. But the same problem of arkode_erkstep not being found still persists.

eugenhu commented 2 years ago

I've patched the build script to pass "/opt/homebrew/include" directly to gcc as a command line argument. Can you try installing from the new branch:

python3 -m pip install git+https://github.com/jdber1/opendrop.git@issue43
ranit-mukherjee commented 2 years ago

This solved the previous problem, but looks like can not access the sundials_arkode library:

g++ -o opendrop/fit/younglaplace/shape.cpython-310-darwin.so -dynamiclib opendrop/fit/younglaplace/.checkpoints/shape.os -L/opt/homebrew/opt/python@3.10/Frameworks/Python.framework/Versions/3.10/lib -lpython3.10 -lsundials_arkode -lsundials_nvecserial ld: library not found for -lsundials_arkode clang: error: linker command failed with exit code 1 (use -v to see invocation)

eugenhu commented 2 years ago

Apologies for the late reply, I think the best way to compile these C extension modules (in macOS) that depend on other shared libs is to hardcode "/opt/homebrew/lib" into the binaries' rpath. I've added another patch to the issue43 branch, could you try installing again?:

python3 -m pip install git+https://github.com/jdber1/opendrop.git@issue43
ranit-mukherjee commented 2 years ago

No worries! You have been super helpful. I am very new to python, so still learning the basics.

I tried the new patch, it shows the same problem of not finding sundials_arkode library

eugenhu commented 2 years ago

Sorry again for the back and forth. It might take a few more trys and errors before we reach a successful build, so thanks for your patience. I've made another patch, when you're free can you try installing again:

python3 -m pip install git+https://github.com/jdber1/opendrop.git@issue43

Hopefully the linker is able to find the libraries and finish compiling; but it may not be able to run successfully yet since one might need to add "/opt/homebrew/lib" to Homebrew's python executable's rpaths. If the compilation succeeds but running the app (with python -m opendrop) fails, try running with the command DYLD_LIBRARY_PATH=/opt/homebrew/lib python -m opendrop instead.

I will be away for the next 2.5 weeks so my further replies may be delayed until then.

ranit-mukherjee commented 2 years ago

This one compiled perfectly! Thank you so much. But running the app is now crashing python with the following error (Screenshot). I tried adding the LIBRARY_PATH command also, but similar error.

Screen Shot 2022-10-06 at 8 56 24 PM
eugenhu commented 2 years ago

Thanks for your patience again. Can you try running

DYLD_PRINT_LIBRARIES=1 python -m opendrop

This will fail, but it should print some diagnostic information into the terminal, could you copy and paste that here?

I don't have an easy solution in mind to get this working on Homebrew on Apple Silicon. I'd have to go through a lot more trial-and-error and I don't want to burden you with that if running opendrop on your M1 mac is not super important to you. There are pre-built windows binaries you can try if you need to use opendrop and have access to a windows machine. Thanks for the cooperation so far though, you've been very helpful. But I'm happy to leave this issue open for now until someone else with an apple silicon mac is able to troubleshoot this.

ranit-mukherjee commented 2 years ago

Hi eugenhu,

Sorry for the late comment. Thanks again for all the help. Even though I could not make it work, I got to learn some stuff from your comments. I ran the command you mentioned. It showed the same "zsh segmentation fault" error with the python stopped working window. I Don't think that is what you were hoping for. Anyway, hopefully I can figure something out in the windows system.

ToriLi1210 commented 3 months ago

Hi,

I encountered a similar issue as @ranit-mukherjee and followed the steps provided by @eugenhu. After running python3 -m pip install git+https://github.com/jdber1/opendrop.git@issue43, I managed to resolve the problem with the sundials_arkode library. However, I am now facing a new error. Could someone please assist me in addressing this new issue? I would appreciate any guidance or suggestions.

Thank you! Terminal Error.pdf

eugenhu commented 3 months ago

What operating system are you using? Also, what version is your SUNDIALS library?

ToriLi1210 commented 3 months ago

What operating system are you using? Also, what version is your SUNDIALS library?

macOS, And the current version is sundials 7.1.1

eugenhu commented 3 months ago

Ok, I think the problem is with an API change in sundials. I've updated the issue43 branch, try installing it again with:

python3 -m pip install git+https://github.com/jdber1/opendrop.git@issue43
ToriLi1210 commented 3 months ago

Thanks! The issue mentioned now solved. However, I’ve encountered another issue with Sundials: the function ERKStepSetStopTime is explicitly marked as deprecated. It seems like there might be a dependency issue with Sundials that’s affecting OpenDrop. OpenDropError.pdf

eugenhu commented 3 months ago

The deprecation errors don't seem to be a problem now. It seems like this is is causing the error:

Undefined symbols for architecture arm64: "_ompi_mpi_comm_null", referenced from
opendrop::younglaplace::YoungLaplaceShape<double>::volume(double) in shape.os

I don't have an Apple silicon laptop to try building opendrop on so I'm not really sure how to help you proceed from here unfortunately. Apologies.

Potentially you need to install Open MPI (perhaps open-mpi) or you may not have the correct SUNDIALS for your architecture (Apple silicon).

ToriLi1210 commented 3 months ago

Thank you for all your support so far. Could you let me know which version of SUNDIALS you are currently working with?

eugenhu commented 3 months ago

You have the correct SUNDIALS version, I'm also using version 7. I'm just not sure if you've installed the version for Intel or Apple silicon. But if you've installed it from homebrew it's probably ok.

ToriLi1210 commented 3 months ago

I installed the opendrop package using homebrew outlined in the documentation provided but I ran into a linker error indicating undefined symbols related to MPI

Screenshot 2024-08-11 at 9 49 14 pm
eugenhu commented 3 months ago

Sorry but I can't really help you further with installing opendrop on an Apple silicon mac since I don't have one to test and troubleshoot with. It seems to be complaining about some '_ompi_mpi_comm_null' not being found, but I really don't know what to suggest as a next step for you to try. Apologies again.