search

yufu-wang / tram

TRAM: Global Trajectory and Motion of 3D Humans from in-the-wild Videos
MIT License
179 stars 13 forks source link

Has Droid-slam changed its source code? #17

Open 2022yingjie opened 1 month ago

2022yingjie commented 1 month ago

Hi, thank you for your impressive work, when I tried to make a inference, there was an wired error showing as follows: Traceback (most recent call last): File "/mnt/fast_nas/xiyingjie/code/tram/scripts/estimate_camera.py", line 46, in <module> cam_int, is_static = calibrate_intrinsics(img_folder, masks, is_static=args.static_camera) File "/mnt/fast_nas/xiyingjie/code/tram/scripts/../lib/camera/masked_droid_slam.py", line 169, in calibrate_intrinsics best_err, is_static = test_slam(img_folder, input_msks, File "/mnt/fast_nas/xiyingjie/code/tram/scripts/../lib/camera/masked_droid_slam.py", line 206, in test_slam droid.track(t, image, intrinsics=intrinsics, mask=conf_msk) TypeError: Droid.track() got an unexpected keyword argument 'mask'

I check the droid's source code, it seems that its author changed the code, the corresponding code is: `def track(self, tstamp, image, depth=None, intrinsics=None): """ main thread - update map """

    with torch.no_grad():
        # check there is enough motion
        self.filterx.track(tstamp, image, depth, intrinsics)

        # local bundle adjustment
        self.frontend()

        # global bundle adjustment
        # self.backend()`

is "masks" parameters important to the inference?

yufu-wang commented 1 month ago

Hi. We provided our own version of droid-slam here. If you clone our repo, you will get this one. The mask is important because we need to mask out the dynamic humans during slam.

HugoXANA commented 2 weeks ago

@yufu-wang hello,

My question

I reffered to the original repo of Droid-slam, and the original installation method indecates another conda env is needed for Droid-slam to be compiled. And thirdparty/DROID-SLAM/environment.yaml shows that it needs pytorch=1.10 which is not compatible with main repo install script install.sh(uses pytorch==2.0.0).
So, how can we compile DROID-SLAM under conda env tram? Thank you!

Detail

plus, I tried to compile DROID-SLAM under conda env tram(pytorch==2.0.0), and while it shows success at the end, 

~/Documents/tram/thirdparty/DROID-SLAM$ python setup.py install
running install
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()
running bdist_egg
running egg_info
writing droid_backends.egg-info/PKG-INFO
writing dependency_links to droid_backends.egg-info/dependency_links.txt
writing top-level names to droid_backends.egg-info/top_level.txt
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'droid_backends.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'droid_backends.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/torch/utils/cpp_extension.py:398: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
creating build/bdist.linux-x86_64/egg
copying build/lib.linux-x86_64-cpython-310/droid_backends.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/__init__.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/run_tests.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/group_ops.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/broadcasting.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/groups.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/gradcheck.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch_backends.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/lietorch/__init__.py to __init__.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/run_tests.py to run_tests.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/group_ops.py to group_ops.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/broadcasting.py to broadcasting.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/groups.py to groups.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/gradcheck.py to gradcheck.cpython-310.pyc
creating stub loader for droid_backends.cpython-310-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/droid_backends.py to droid_backends.cpython-310.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying droid_backends.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying droid_backends.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying droid_backends.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying droid_backends.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.droid_backends.cpython-310: module references __file__
creating 'dist/droid_backends-0.0.0-py3.10-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing droid_backends-0.0.0-py3.10-linux-x86_64.egg
removing '/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg' (and everything under it)
creating /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg
Extracting droid_backends-0.0.0-py3.10-linux-x86_64.egg to /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages
Adding droid-backends 0.0.0 to easy-install.pth file

Installed /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg
Processing dependencies for droid-backends==0.0.0
Finished processing dependencies for droid-backends==0.0.0
running install
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: SetuptoolsDeprecationWarning: setup.py install is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` directly.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
        ********************************************************************************

!!
  self.initialize_options()
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/setuptools/_distutils/cmd.py:66: EasyInstallDeprecationWarning: easy_install command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``easy_install``.
        Instead, use pypa/build, pypa/installer or other
        standards-based tools.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()
running bdist_egg
running egg_info
writing thirdparty/lietorch/lietorch.egg-info/PKG-INFO
writing dependency_links to thirdparty/lietorch/lietorch.egg-info/dependency_links.txt
writing top-level names to thirdparty/lietorch/lietorch.egg-info/top_level.txt
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/torch/utils/cpp_extension.py:476: UserWarning: Attempted to use ninja as the BuildExtension backend but we could not find ninja.. Falling back to using the slow distutils backend.
  warnings.warn(msg.format('we could not find ninja.'))
reading manifest file 'thirdparty/lietorch/lietorch.egg-info/SOURCES.txt'
adding license file 'LICENSE'
writing manifest file 'thirdparty/lietorch/lietorch.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
copying thirdparty/lietorch/lietorch/__init__.py -> build/lib.linux-x86_64-cpython-310/lietorch
copying thirdparty/lietorch/lietorch/run_tests.py -> build/lib.linux-x86_64-cpython-310/lietorch
copying thirdparty/lietorch/lietorch/group_ops.py -> build/lib.linux-x86_64-cpython-310/lietorch
copying thirdparty/lietorch/lietorch/broadcasting.py -> build/lib.linux-x86_64-cpython-310/lietorch
copying thirdparty/lietorch/lietorch/groups.py -> build/lib.linux-x86_64-cpython-310/lietorch
copying thirdparty/lietorch/lietorch/gradcheck.py -> build/lib.linux-x86_64-cpython-310/lietorch
running build_ext
/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/torch/utils/cpp_extension.py:398: UserWarning: There are no g++ version bounds defined for CUDA version 11.8
  warnings.warn(f'There are no {compiler_name} version bounds defined for CUDA version {cuda_str_version}')
creating build/bdist.linux-x86_64/egg
copying build/lib.linux-x86_64-cpython-310/droid_backends.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/__init__.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/run_tests.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/group_ops.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/broadcasting.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/groups.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch/gradcheck.py -> build/bdist.linux-x86_64/egg/lietorch
copying build/lib.linux-x86_64-cpython-310/lietorch_backends.cpython-310-x86_64-linux-gnu.so -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/lietorch/__init__.py to __init__.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/run_tests.py to run_tests.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/group_ops.py to group_ops.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/broadcasting.py to broadcasting.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/groups.py to groups.cpython-310.pyc
byte-compiling build/bdist.linux-x86_64/egg/lietorch/gradcheck.py to gradcheck.cpython-310.pyc
creating stub loader for lietorch_backends.cpython-310-x86_64-linux-gnu.so
byte-compiling build/bdist.linux-x86_64/egg/lietorch_backends.py to lietorch_backends.cpython-310.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying thirdparty/lietorch/lietorch.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying thirdparty/lietorch/lietorch.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying thirdparty/lietorch/lietorch.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying thirdparty/lietorch/lietorch.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
writing build/bdist.linux-x86_64/egg/EGG-INFO/native_libs.txt
zip_safe flag not set; analyzing archive contents...
__pycache__.lietorch_backends.cpython-310: module references __file__
creating 'dist/lietorch-0.2-py3.10-linux-x86_64.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing lietorch-0.2-py3.10-linux-x86_64.egg
removing '/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/lietorch-0.2-py3.10-linux-x86_64.egg' (and everything under it)
creating /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/lietorch-0.2-py3.10-linux-x86_64.egg
Extracting lietorch-0.2-py3.10-linux-x86_64.egg to /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages
Adding lietorch 0.2 to easy-install.pth file

Installed /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/lietorch-0.2-py3.10-linux-x86_64.egg
Processing dependencies for lietorch==0.2
Finished processing dependencies for lietorch==0.2

errror occurs when running demo 

python scripts/estimate_camera.py --video "./example_video.mov"

error info:

~/Documents/tram$ python scripts/estimate_camera.py --video "./example_video.mov"
Traceback (most recent call last):
  File "/home/hugoxana/Documents/tram/scripts/estimate_camera.py", line 12, in <module>
    from lib.camera import run_metric_slam, calibrate_intrinsics
  File "/home/hugoxana/Documents/tram/scripts/../lib/camera/__init__.py", line 1, in <module>
    from .masked_droid_slam import run_metric_slam, calibrate_intrinsics
  File "/home/hugoxana/Documents/tram/scripts/../lib/camera/masked_droid_slam.py", line 13, in <module>
    from droid import Droid
  File "/home/hugoxana/Documents/tram/thirdparty/DROID-SLAM/droid_slam/droid.py", line 2, in <module>
    import lietorch
  File "/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg/lietorch/__init__.py", line 2, in <module>
    from .groups import LieGroupParameter, SO3, RxSO3, SE3, Sim3, cat, stack
  File "/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg/lietorch/groups.py", line 5, in <module>
    from .group_ops import Exp, Log, Inv, Mul, Adj, AdjT, Jinv, Act3, Act4, ToMatrix, ToVec, FromVec
  File "/home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg/lietorch/group_ops.py", line 1, in <module>
    import lietorch_backends
ImportError: /home/hugoxana/anaconda3/envs/tram/lib/python3.10/site-packages/droid_backends-0.0.0-py3.10-linux-x86_64.egg/lietorch_backends.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN2at4_ops10zeros_like4callERKNS_6TensorESt8optionalIN3c1010ScalarTypeEES5_INS6_6LayoutEES5_INS6_6DeviceEES5_IbES5_INS6_12MemoryFormatEE