search

isl-org / Open3D

Open3D: A Modern Library for 3D Data Processing
http://www.open3d.org
Other
11.55k stars 2.32k forks source link

Request Conda support #5272

Open ellisbrown opened 2 years ago

ellisbrown commented 2 years ago

Checklist

Proposed new feature or change

Support conda installation of open3d in order to better integrate with the broader python ecosystem.

References

Conda is the preferred package and environment manager for a significant portion of the python community. When working on a project using conda, it is best to install all packages via conda and only use pip if absolutely necessary in order to ensure the best environment resolution. Mixing the two can lead to package conflicts and unsolvable environments.

https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-pkgs.html#:~:text=You%20can%20install%20pip%20in,current%20conda%20environment%20is%20used.

Additional information

I tried using the below conda package, but was disappointed to see the deprecation warning

Warning] Since Open3D 0.15, installing Open3D via conda is deprecated. Please re-install Open3D via: `pip install open3d -U`.

https://anaconda.org/open3d-admin/open3d

pietakio commented 2 years ago

100%. conda is the Python package manager of choice for the computational community – and for good reason.

conda "just works" out-of-the-box. conda seamlessly integrates with non-trivial hardware solutions like CUDA-accelerated GPU compute and the emerging landscape of Tensor Processing Units (e.g., Apple Neural Engine (ANE) in Apple M1 and M2 Silicon), all of which are mandatory prerequisites for modern data science pipelines.

Pip != Sanity

The same cannot be said for pip. pip integrates poorly with CUDA and anything else requiring low-level drivers. Because pip lacks core support for installing system libraries like CUDA sanely (by which we mean robustly, reusably, and portably), every Python package requiring CUDA support is forced to ship pip-friendly binary wheels that literally contain a duplicate CUDA installation inside of themselves. This duplicate CUDA installation is then duplicated across every version bump of every CUDA-capable Python package.

This is the height of insanity. Mass duplication of CUDA everywhere means that hosting of pip-friendly binary wheels on PyPI is fundamentally unsustainable and guaranteed to blow up in the community's face. That is now happening. Even polite requests for PyPI quota increases from popular large-scale projects are now increasingly rejected.

If anything, it's pip that should be deprecated. Antiquated packaging technologies that catastrophically fail to deduplicate heavyweight dependencies like CUDA should be relegated to the historical dustbin of Wikipedia changelogs.

Conda == Sanity

conda is the future. pip is the past. If maintaining an Open3D-specific open3d-admin conda channel is beyond the volunteer scope of Intelligent Systems, the appropriate solution is to migrate Open3D's conda package to a community-maintained channel like conda-forge. Let cloud-hosted automation do the heavy lifting of packaging, distribution, and testing for you.

But don't abandon the only forward path for Python packaging in 2022.