search

mapillary / OpenSfM

Open source Structure-from-Motion pipeline
https://www.opensfm.org/
BSD 2-Clause "Simplified" License
3.42k stars 861 forks source link

Docker Installation Fails #1080

Open FullMetalNicky opened 1 week ago

FullMetalNicky commented 1 week ago

I tried to build the library with Docker, after pulling the Github code. git clone --recursive https://github.com/mapillary/OpenSfM Looks like despite pulling the submodules, the ThirdParty files for gtest are missing, resulting in the following error

44.28 -- Performing Test HAS_FLTO 44.36 -- Performing Test HAS_FLTO - Success 44.36 -- LTO enabled 44.37 -- Configuring done 44.37 CMake Error at CMakeLists.txt:112 (add_library): 44.37 Cannot find source file: 44.37 44.37 testing_main.cc 44.37 44.37 Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm 44.37 .hpp .hxx .in .txx 44.37 44.37 44.37 CMake Error at CMakeLists.txt:106 (add_library): 44.37 Cannot find source file: 44.37 44.37 third_party/gtest/gmock_gtest_all.cc 44.37 44.37 Tried extensions .c .C .c++ .cc .cpp .cxx .cu .m .M .mm .h .hh .h++ .hm 44.37 .hpp .hxx .in .txx 44.37 44.37 44.37 CMake Error at CMakeLists.txt:112 (add_library): 44.37 No SOURCES given to target: test_main 44.37 44.37 44.37 CMake Error at CMakeLists.txt:106 (add_library): 44.37 No SOURCES given to target: gtest 44.37 44.37 44.37 CMake Error at third_party/vlfeat/CMakeLists.txt:11 (add_library): 44.37 No SOURCES given to target: vl 44.37 44.37 44.37 CMake Generate step failed. Build files cannot be regenerated correctly. 44.38 Configuring for python 3.8... 44.38 Traceback (most recent call last): 44.38 File "setup.py", line 59, in 44.38 configure_c_extension() 44.38 File "setup.py", line 43, in configure_c_extension 44.38 subprocess.check_call(cmake_command, cwd="cmake_build") 44.38 File "/usr/lib/python3.8/subprocess.py", line 364, in check_call 44.38 raise CalledProcessError(retcode, cmd) 44.38 subprocess.CalledProcessError: Command '['cmake', '../opensfm/src', '-DPYTHON_EXECUTABLE=/usr/bin/python3']' returned non-zero exit status 1.

Dockerfile:30

29 |
30 | >>> RUN pip3 install -r requirements.txt && \ 31 | >>> python3 setup.py build 32 |

ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt && python3 setup.py build" did not complete successfully: exit code: 1 make: *** [Makefile:2: build] Error 1

Please advise on how to solve this. The whole point of providing a Docker is to have a smooth and painless installation...

MysteryHS commented 1 week ago

I'm experiencing the same issue even though I'm not using docker to build the tool.

DiegoHerrera1890 commented 1 week ago

Is there any update in this matter?

alessandro-pistola-me commented 1 week ago

The most frequent errors encountered are:

  1. pyyaml installation error (cython error with pyyaml version 5.4.1)
  2. pybundle error
  3. third_party/gtest error

By reviewing the commit history from August 9 (commit ef872b2399cf1cc036d91e950f210a7be33c2745) onward, you’ll notice that all workflows for subsequent commits, like "Docker CI / Build docker and run tests (push)" and "Docker CI / Build docker installing CeresSolver 2 and run tests (push)," fail.

Below is a step-by-step guide explaining how I resolved these issues.

The missing files should be taken from the release, but only those for the third_party folder (missing gtest files), as the release uses outdated dependencies and likely Ceres 2.0.

To begin, create two clones of the latest repository:

In the root of the main repo, run: `git submodule update --init --recursive This will load the removed submodules.

In the main repo, remove pyyaml==5.4.1 from requirements.txt and install it manually with the following command: pip install "cython<3.0.0" && pip install --no-build-isolation pyyaml==5.1

In the second repo, go to the August 9 commit: git checkout ef872b2399cf1cc036d91e950f210a7be33c2745

From this commit, copy the missing files (gmock_gtest_all.cc, gmock_main.cc, testing_main.cc, and any other required files) into the main repository. Now you should be able to build the image from the modified dockerfile in which you should install ceres 2.2 insted ceres 2.0 RUN \ curl -L http://ceres-solver.org/ceres-solver-2.2.0.tar.gz | tar xz && \ mkdir ceres-bin && cd ceres-bin && \ cmake ../ceres-solver-2.2.0 && \ make -j3 && \ make install

DiegoHerrera1890 commented 1 week ago

I did exactly what you said but now I have this error:

42.59 -- Configuring done
42.60 CMake Error at third_party/vlfeat/CMakeLists.txt:11 (add_library):
42.60   No SOURCES given to target: vl
42.60 
42.60 
42.60 CMake Generate step failed.  Build files cannot be regenerated correctly.
42.60 Configuring for python 3.8...
42.60 Traceback (most recent call last):
42.60   File "setup.py", line 59, in <module>
42.60     configure_c_extension()
42.60   File "setup.py", line 43, in configure_c_extension
42.60     subprocess.check_call(cmake_command, cwd="cmake_build")
42.60   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
42.60     raise CalledProcessError(retcode, cmd)
42.60 subprocess.CalledProcessError: Command '['cmake', '../opensfm/src', '-DPYTHON_EXECUTABLE=/usr/bin/python3']' returned non-zero exit status 1.
------
Dockerfile.ceres2:41
--------------------
  40 |     
  41 | >>> RUN pip3 install -r requirements.txt && \
  42 | >>>     python3 setup.py build
  43 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt &&     python3 setup.py build" did not complete successfully: exit code: 1

It seems to be related to vlfeat... Do you have any idea how to solve this?

alessandro-pistola-me commented 1 week ago

I did exactly what you said but now I have this error:

42.59 -- Configuring done
42.60 CMake Error at third_party/vlfeat/CMakeLists.txt:11 (add_library):
42.60   No SOURCES given to target: vl
42.60 
42.60 
42.60 CMake Generate step failed.  Build files cannot be regenerated correctly.
42.60 Configuring for python 3.8...
42.60 Traceback (most recent call last):
42.60   File "setup.py", line 59, in <module>
42.60     configure_c_extension()
42.60   File "setup.py", line 43, in configure_c_extension
42.60     subprocess.check_call(cmake_command, cwd="cmake_build")
42.60   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
42.60     raise CalledProcessError(retcode, cmd)
42.60 subprocess.CalledProcessError: Command '['cmake', '../opensfm/src', '-DPYTHON_EXECUTABLE=/usr/bin/python3']' returned non-zero exit status 1.
------
Dockerfile.ceres2:41
--------------------
  40 |     
  41 | >>> RUN pip3 install -r requirements.txt && \
  42 | >>>     python3 setup.py build
  43 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt &&     python3 setup.py build" did not complete successfully: exit code: 1

It seems to be related to vlfeat... Do you have any idea how to solve this?

did you import OpenSfM/opensfm/src/third_party/vlfeat/vl from the "release" repository?

NyxAither commented 1 week ago

I'm having the same issue on windows, not using docker.

DiegoHerrera1890 commented 1 week ago

I did exactly what you said but now I have this error:

42.59 -- Configuring done
42.60 CMake Error at third_party/vlfeat/CMakeLists.txt:11 (add_library):
42.60   No SOURCES given to target: vl
42.60 
42.60 
42.60 CMake Generate step failed.  Build files cannot be regenerated correctly.
42.60 Configuring for python 3.8...
42.60 Traceback (most recent call last):
42.60   File "setup.py", line 59, in <module>
42.60     configure_c_extension()
42.60   File "setup.py", line 43, in configure_c_extension
42.60     subprocess.check_call(cmake_command, cwd="cmake_build")
42.60   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
42.60     raise CalledProcessError(retcode, cmd)
42.60 subprocess.CalledProcessError: Command '['cmake', '../opensfm/src', '-DPYTHON_EXECUTABLE=/usr/bin/python3']' returned non-zero exit status 1.
------
Dockerfile.ceres2:41
--------------------
  40 |     
  41 | >>> RUN pip3 install -r requirements.txt && \
  42 | >>>     python3 setup.py build
  43 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt &&     python3 setup.py build" did not complete successfully: exit code: 1

It seems to be related to vlfeat... Do you have any idea how to solve this?

did you import OpenSfM/opensfm/src/third_party/vlfeat/vl from the "release" repository?

I run git submodule update --init --recursive and the OpenSfM/opensfm/src/third_party/vlfeat/vl folder only has three items:

(base) diego@diego:~/OpenSfM/opensfm/src/third_party/vlfeat/vl$ ls
ikmeans_elkan.tc  ikmeans_init.tc  ikmeans_lloyd.tc
alessandro-pistola-me commented 1 week ago

I did exactly what you said but now I have this error:

42.59 -- Configuring done
42.60 CMake Error at third_party/vlfeat/CMakeLists.txt:11 (add_library):
42.60   No SOURCES given to target: vl
42.60 
42.60 
42.60 CMake Generate step failed.  Build files cannot be regenerated correctly.
42.60 Configuring for python 3.8...
42.60 Traceback (most recent call last):
42.60   File "setup.py", line 59, in <module>
42.60     configure_c_extension()
42.60   File "setup.py", line 43, in configure_c_extension
42.60     subprocess.check_call(cmake_command, cwd="cmake_build")
42.60   File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
42.60     raise CalledProcessError(retcode, cmd)
42.60 subprocess.CalledProcessError: Command '['cmake', '../opensfm/src', '-DPYTHON_EXECUTABLE=/usr/bin/python3']' returned non-zero exit status 1.
------
Dockerfile.ceres2:41
--------------------
  40 |     
  41 | >>> RUN pip3 install -r requirements.txt && \
  42 | >>>     python3 setup.py build
  43 |     
--------------------
ERROR: failed to solve: process "/bin/sh -c pip3 install -r requirements.txt &&     python3 setup.py build" did not complete successfully: exit code: 1

It seems to be related to vlfeat... Do you have any idea how to solve this?

did you import OpenSfM/opensfm/src/third_party/vlfeat/vl from the "release" repository?

I run git submodule update --init --recursive and the OpenSfM/opensfm/src/third_party/vlfeat/vl folder only has three items:

(base) diego@diego:~/OpenSfM/opensfm/src/third_party/vlfeat/vl$ ls
ikmeans_elkan.tc  ikmeans_init.tc  ikmeans_lloyd.tc

Try to import the folder content from the release. Btw I managed to build the docker image but OpenSfM is still not running, this is the error: Traceback (most recent call last): File "/code/OpenSfM/bin/opensfm_main.py", line 10, in <module> from opensfm import commands File "/code/OpenSfM/opensfm/__init__.py", line 2, in <module> from opensfm import pybundle ImportError: cannot import name 'pybundle' from partially initialized module 'opensfm' (most likely due to a circular import) (/code/OpenSfM/opensfm/__init__.py)

DiegoHerrera1890 commented 1 week ago

You mean copying the files from the release folder? I did that, but a bunch of new errors arose.

alessandro-pistola-me commented 1 week ago

You mean copying the files from the release folder? I did that, but a bunch of new errors arose.

Then you can try from this commit: ef872b2399cf1cc036d91e950f210a7be33c2745 But as I told you previously It seems the docker build fails anyway (also in their github workflows)