isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.53k stars 2.32k forks source link

Import fails with segmentation fault on MacOS within Anaconda environment #6082

Open flolu opened 1 year ago

flolu commented 1 year ago

Checklist

Steps to reproduce the issue

I first cloned Open3D by:

conda env create -n open3d-test
conda activate opend3d-test
pip install open3d==0.17.0
python
import open3d

Error message

[5]    35067 segmentation fault  python

### Open3D, Python and System information

```markdown
- Operating system: macOS Ventura 13.3.1
- Python version: Python 3.10.10
- System architecture: apple-silicon
- Is this a remote workstation?: no
- How did you install Open3D?: pip within conda

Additional information

It works fine when running outside Anaconda like this:

pip install open3d==0.17.0
python
import open3d
cwreynolds commented 1 year ago

This is not an answer to your issue, but just as a point of reference: I recently installed Open3D 0.17.0, with pip into a conda env, on my macOS 12.6.3 M1 laptop. See: https://github.com/isl-org/Open3D/discussions/6032

ferranOCT commented 1 year ago

I made some test with this problem. It seems som incompatibility with other libraries. In a clean environment, only installing python=3.10 and open3D=0.17, all works perfectly. Then, for my project requeriments I need to install pandas=2.0.3. After the installation, the segmentation fault appears when "import open3d" is called. These are the libraries installed alongside with pandas:

libblas conda-forge/osx-arm64::libblas-3.9.0-17_osxarm64_openblas libcblas conda-forge/osx-arm64::libcblas-3.9.0-17_osxarm64_openblas libcxx conda-forge/osx-arm64::libcxx-16.0.6-h4653b0c_0 libgfortran conda-forge/osx-arm64::libgfortran-5.0.0-12_2_0_hd922786_31 libgfortran5 conda-forge/osx-arm64::libgfortran5-12.2.0-h0eea778_31 liblapack conda-forge/osx-arm64::liblapack-3.9.0-17_osxarm64_openblas libopenblas conda-forge/osx-arm64::libopenblas-0.3.23-openmp_hc731615_0 llvm-openmp conda-forge/osx-arm64::llvm-openmp-16.0.6-h1c12783_0 numpy conda-forge/osx-arm64::numpy-1.25.0-py310haa1e00c_0 pandas conda-forge/osx-arm64::pandas-2.0.3-py310h1cdf563_0 python-dateutil conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0 python-tzdata conda-forge/noarch::python-tzdata-2023.3-pyhd8ed1ab_0 python_abi conda-forge/osx-arm64::python_abi-3.10-3_cp310 pytz conda-forge/noarch::pytz-2023.3-pyhd8ed1ab_0 six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0

any idea? It's a very annoying problem!

Thanks!

ferranOCT commented 1 year ago

I was not able to find the package responsable of the segmentation fault. However, I used Rosetta2 to run all the code in order to execute them with a intel_x86 virtualised chipset. To do this, is mandatory to remove all libraries, both from Homebrew and Anaconda and also remove both frameworks. Then, install Rosetta2 and execute the terminal with the flag "use with Rosetta" activated. Then, I re-installed all in this situation, which automatically take the sources not from arm64 but from x86_64. Following these steps, finally I'm able to run the code with only a warning about some deprecated compatibility.

Here some references:

Install Rosetta2 -> https://osxdaily.com/2020/12/04/how-install-rosetta-2-apple-silicon-mac/ Install Miniconda with x86_64 support -> https://towardsdatascience.com/how-to-install-miniconda-x86-64-apple-m1-side-by-side-on-mac-book-m1-a476936bfaf0

I hope this could help.

Crear12 commented 1 year ago

open3d

Thank you so much! Solved this issue by creating a brand new conda environment with python 3.10, and installing open3d first!

brentonmallen1 commented 1 year ago

I'm running into this issue now on an M2 machine running Ventura. I was able to install open3d successfully and then once I installed numpy, I started getting the seg fault error. Does using Rosetta2 impact the performance of the library due to the emulation? What if I didn't want to convert my entire miniforge setup to use rosetta?

ferranOCT commented 1 year ago

I don't have any problem more than some warnings appearing in the console during the execution. About the performance I don't have a trustable comparison, but I was working in Ubuntu with a Intel i7 + 16gb RAM few months ago with the same code and I didn't notice substantial differences in the execution time. Regarding the question about converting your entire setup, you always can have two different console (i.e. Alacritty and Terminal), one running with Rosseta and the other one without.

brentonmallen1 commented 1 year ago

@ferranOCT were you installing anything along side Open3D, like numpy, pandas, etc?

ferranOCT commented 1 year ago

Yes, a lot of libraries :P. Although is not 100% sure, the error rises when I install numpy after open3D.

brentonmallen1 commented 1 year ago

Okay, that's validating :) I was trying to build from source but there doesn't seem to be any documentation about how to set the dependency versions for things like numpy. :( I'll keep trying though. thank you!

cemoke commented 11 months ago

I encountered the same issue. If I only use pip-installed libraries and not any conda-installed ones in the venv, I don't have any issues.

Unfortunately, I have to use other conda-installed libraries. Any suggestions?

Crear12 commented 11 months ago

I encountered the same issue. If I only use pip-installed libraries and not any conda-installed ones in the venv, I don't have any issues.

Unfortunately, I have to use other conda-installed libraries. Any suggestions?

Unfortunately no...I have multiple computers, both Mac and Windows, and this happens too frequently. I simply added a platform detector to decide if the Open3D-related functions need to be imported. I quite like what Open3D has brought, but can't waste more time on installing/configuring the environment.

Crear12 commented 9 months ago

Okay, that's validating :) I was trying to build from source but there doesn't seem to be any documentation about how to set the dependency versions for things like numpy. :( I'll keep trying though. thank you!

I tried to build from source based on this: https://www.open3d.org/docs/release/arm.html But it doesn't help at all. It even asks me to use "apt-get" on my Mac.

richardrl commented 6 months ago

Still happening!

Fraer commented 4 months ago

Just had the same issue with python 3.11:

% python3 -c "import open3d as o3d; print(o3d.__version__)"
0.18.0

% python3 -c "import open3d as o3d; \                     
           mesh = o3d.geometry.TriangleMesh.create_sphere(); \
           mesh.compute_vertex_normals(); \
           o3d.visualization.draw(mesh, raw_mode=True)"
FEngine (64 bits) created at 0x7f80e6100000 (threading is enabled)
FEngine resolved backend: OpenGL
zsh: segmentation fault  python3 -c

Same with python3.10 and open3d v0.17:

% python --version                                         
Python 3.10.14
% python -c "import open3d as o3d; print(o3d.__version__)" 
0.17.0
% python -c "import open3d as o3d; \                      
           mesh = o3d.geometry.TriangleMesh.create_sphere(); \
           mesh.compute_vertex_normals(); \
           o3d.visualization.draw(mesh, raw_mode=True)"
FEngine (64 bits) created at 0x7fe682900000 (threading is enabled)
FEngine resolved backend: OpenGL
zsh: segmentation fault  python3 -c 
JackmanYuan commented 3 months ago

Hi everyone, I encountered a similar issue on my MacBook with the M3 chip.

The default Open3D installation via pip install open3d caused segmentation faults when I tried to run any Open3D functions.

I was testing with the following simple script:

import open3d as o3d

def main():
    print("Open3D version:", o3d.__version__)

    # Create a simple point cloud
    point_cloud = o3d.geometry.PointCloud()
    points = [[0, 0, 0], [1, 0, 0], [0, 1, 0]]
    point_cloud.points = o3d.utility.Vector3dVector(points)

    # Visualize the point cloud
    o3d.visualization.draw_geometries([point_cloud])

if __name__ == "__main__":
    main()

The segmentation fault occurred when calling o3d.utility.Vector3dVector(points).

I solved this issue by following this helpful comment and manually installing a compatible version of Open3D (0.18.0) as follows:

  1. Downloaded the open3d-0.18.0-cp38-cp38-macosx_11_0_arm64.whl file from the Open3D releases page.
  2. Installed the wheel file using the command:
    pip install open3d-0.18.0-cp38-cp38-macosx_11_0_arm64.whl
  3. This resolved the segmentation fault issue, and Open3D now works perfectly on my M3 chip MacBook.

I hope this helps!

Mastergalen commented 3 months ago

Thanks @JackmanYuan

I couldn't find open3d-0.18.0-cp38-cp38-macosx_11_0_arm64.whl on the releases page.

I tried to run pip install https://github.com/isl-org/Open3D/releases/download/v0.18.0/open3d-0.18.0-cp310-cp310-macosx_13_0_arm64.whl but still get the same issue.

OMP: Error #179: Function pthread_mutex_init failed:
OMP: System error #22: Invalid argument

I found that if I import open3d before numpy, there is no segfault. However, if it is imported after numpy, it segfaults.

# ✅ 
import open3d
import numpy

# ❌ Segfault
import numpy
import open3d

My environment:

Mastergalen commented 3 months ago

I can confirm that installing the latest dev build does fix the issue.

When will the next version be officially released? The download link below will stop working when a new dev version is released.

https://github.com/isl-org/Open3D/releases/download/main-devel/open3d-0.18.0+6cb32f6-cp310-cp310-macosx_12_0_universal2.whl
stefansommer commented 3 months ago

I believe the download link has stopped working. Do you know a way to obtain a working version?

JackmanYuan commented 3 months ago

Hi!

I just revisited the Open3D release page, and I can confirm that the wheel files are still available. If you don't see them immediately, try expanding the "Assets" section under the specific release version.

I've attached a screenshot for reference.

Screenshot 2024-08-22 at 8 26 03 PM

Maybe you can try downloading the wheels from there?

Hope this will help!

stefansommer commented 3 months ago

Thanks! As far as I can tell, the crash still occurs with those files, but I found the latest dev build and that works.

roelofvandijkO commented 2 months ago

Also encountering this with the latest version. M3 Mac, Python 3.11.10, in a conda env, open3d install via poetry. Numpy is 1.26.4


  File ", /opt/homebrew/Caskroom/miniconda/base/envs/s2p-pipelines/lib/python3.11/site-packages/open3d/__init__.pycharset_normalizer.md", , , line charset_normalizer.mdrequests.packages.charset_normalizer.md93,  in , requests.packages.chardet.md<module>requests.packages.charset_normalizer.md
,   File , numpy.core._multiarray_umath"<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 940 in exec_module
  File "<frozen importlib._bootstrap>", line 690 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1147 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1176 in _find_and_load```
cemoke commented 1 month ago

Thanks! As far as I can tell, the crash still occurs with those files, but I found the latest dev build and that works. @stefansommer I am encountering the same issue. Can you please share more about where you found it, how you installed it, and whether your machine is arm64?

vandamd commented 1 week ago

Couldn't find a compatible dev build today - can confirm installing a .whl built from source works