search

colmap / pycolmap

Python bindings for COLMAP
BSD 3-Clause "New" or "Revised" License
858 stars 125 forks source link

Script to export our own wheel #275

Closed jytime closed 1 week ago

jytime commented 1 month ago

Hi Paul,

Hope this question is within the scope. I have modified the source code of colmap and binded it by pycolmap. It works well in my local machine.

Now I was hoping to export the compiled pycolmap as a python wheel and use it in a remote cluster. I did this by pip wheel . -w dist under the path of colmap/pycolmap. The file tree structure is shown below. 

colmap
├── pycolmap
│   ├── .eggs
│   ├── .gitignore
│   ├── CMakeLists.txt
│   ├── README.md
│   ├── ci
│   ├── dist
│   ├── example.py
│   ├── pycolmaplib
│   ├── pyproject.toml
│   └── setup.py
├── src
│   ├── colmap
│   ├── pycolmap
│   └── thirdparty

However, after installation, whenever I import pycolmap in the new environment, it tells me

ImportError                               Traceback (most recent call last)
Cell In[5], line 1
----> 1 import pycolmap

ImportError: libboost_program_options.so.1.84.0: cannot open shared object file: No such file or directory

This still happens even if I copied the original .so file and export it in LD_LIBRARY_PATH. So I guess I did something wrong when exporting the wheel. May I ask any suggestion on how to export our own python wheel? Did I miss anything?

sarlinpe commented 1 month ago

This will only work if your remote cluster has the same disto as your local machine. If not, you need to build a manylinux wheel. For PyCOLMAP, we build manylinux2014 inside a Docker container with CentOS (this is managed by https://cibuildwheel.pypa.io). Your issue relates to Python packaging and isn't specific to PyCOLMAP.

jytime commented 1 month ago

Thanks for your prompt reply! I will check if cibuildwheel can work without docker, as docker is not allowed in my cluster. I would share it when I have an answer.

sarlinpe commented 1 month ago

To clarify, Docker is required to build the wheel locally, not to install it on other machines.

jytime commented 1 month ago

Yes, unfortunately I think both my working cluster (AWS) and the remote machine (kaggle notebook) do not allow docker :(

jytime commented 1 week ago

The simplest solution we found is using github workflow, which is provided in colmap repository:

https://github.com/colmap/colmap/tree/main/.github/workflows

Basically people need to have their own colmap repository and a docker, then letting github compile all the stuffs in its cluster. This will require some costs but overall affordable.