search

MIT-SPARK / Kimera-VIO

Visual Inertial Odometry with SLAM capabilities and 3D Mesh generation.
BSD 2-Clause "Simplified" License
1.53k stars 416 forks source link

OpenCV + other errors with 2021 M1 Mac Pro #200

Closed alex-t-hu closed 10 months ago

alex-t-hu commented 1 year ago

Description: After building the image with a slightly modified Dockerfile, I used XQuartz to make the GUI work in the Docker container. Then, I get warnings/errors such as "Missing landmark error to build 3DMesh" and "Delauney triangle out of range," but these don't seem to be critical. I also get errors with libgl, which I then fix with changing the LIBGL_ALWAYS_INDIRECT environment variable. Finally, I get an OpenCV error which I can't resolve.

Command: Dockerfile:

# Use an official Python runtime as a parent image
FROM ubuntu:18.04

LABEL maintainer='Antoni Rosinol "arosinol@mit.edu"'

# To avoid tzdata asking for geographic location...
ENV DEBIAN_frontend noninteractive

# Set the working directory to /root
ENV DIRPATH /root
WORKDIR $DIRPATH
ENV DEBIAN_FRONTEND=noninteractive

#Install build dependencies
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils
RUN apt-get update && apt-get install -y git cmake

# Install xvfb to provide a display to container for GUI related testing.
RUN apt-get update && apt-get install -y xvfb

# Install GTSAM
# RUN apt-get update && apt-get install -y libboost-all-dev
# ADD https://api.github.com/repos/borglab/gtsam/git/refs/heads/master version.json
# RUN git clone https://github.com/borglab/gtsam.git
# RUN cd gtsam && \
#     git fetch && \
#     mkdir build && \
#     cd build && \
#     cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DGTSAM_BUILD_TESTS=OFF -DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF -DCMAKE_BUILD_TYPE=Release -DGTSAM_BUILD_UNSTABLE=ON -DGTSAM_TANGENT_PREINTEGRATION=OFF .. && \
# #     make -j$(nproc) install
#       make -j4 install
# Install GTSAM
RUN apt-get update && apt-get install -y libboost-all-dev
# FROM dev-boost as gtsam-code
ADD https://api.github.com/repos/borglab/gtsam/git/refs/heads/master version.json
RUN git clone https://github.com/borglab/gtsam.git
RUN cd gtsam && \
    git fetch && \
    git reset --hard e5866799dff48239573cdd84964180867e50edd2
# FROM gtsam-code as gtsam
RUN cd gtsam && \
    mkdir build && \
    cd build && \
    cmake -DCMAKE_INSTALL_PREFIX=/usr/local -DGTSAM_BUILD_TESTS=OFF -DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF -DCMAKE_BUILD_TYPE=Release -DGTSAM_BUILD_UNSTABLE=ON -DGTSAM_TANGENT_PREINTEGRATION=OFF .. && \
    make -j$(nproc) install

# Install OpenCV for Ubuntu 18.04
RUN apt-get update && apt-get install -y \
      build-essential cmake unzip pkg-config \
      libjpeg-dev libpng-dev libtiff-dev \
      libvtk6-dev \
      libgtk-3-dev \
      libatlas-base-dev gfortran

RUN git clone https://github.com/opencv/opencv.git
RUN cd opencv && \
      git checkout tags/3.3.1 && \
      mkdir build

RUN git clone https://github.com/opencv/opencv_contrib.git
RUN cd opencv_contrib && \
      git checkout tags/3.3.1

RUN cd opencv/build && \
      cmake -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX=/usr/local \
      -D BUILD_opencv_python=OFF \
      -D BUILD_opencv_python2=OFF \
      -D BUILD_opencv_python3=OFF \
      -DOPENCV_EXTRA_MODULES_PATH=$DIRPATH/opencv_contrib/modules .. && \
      make -j$(nproc) install
RUN echo $(nproc)
# Install Open_GV
RUN git clone https://github.com/laurentkneip/opengv
RUN cd opengv && \
      mkdir build
RUN cd opengv/build && \
      cmake -DCMAKE_BUILD_TYPE=Release \
      -DCMAKE_INSTALL_PREFIX=/usr/local \
      -DEIGEN_INCLUDE_DIRS=$DIRPATH/gtsam/gtsam/3rdparty/Eigen \
      -DEIGEN_INCLUDE_DIR=$DIRPATH/gtsam/gtsam/3rdparty/Eigen .. && \
      make -j$(nproc) install

# Install DBoW2
RUN git clone https://github.com/dorian3d/DBoW2.git
RUN cd DBoW2 && \
      mkdir build && \
      cd build && \
      cmake .. && \
      make -j$(nproc) install

# Install RobustPGO
ADD https://api.github.com/repos/MIT-SPARK/Kimera-RPGO/git/refs/heads/master version.json
RUN git clone https://github.com/MIT-SPARK/Kimera-RPGO.git
RUN cd Kimera-RPGO && \
      mkdir build && \
      cd build && \
      cmake .. && \
      make -j$(nproc)

## [Optional] Install Kimera-VIO-Evaluation from PyPI
RUN apt-get update && \
    apt-get install software-properties-common -y
# Get python3
RUN apt-get update && \
    add-apt-repository ppa:deadsnakes/ppa
# RUN apt-get update && \
#     apt-get install -y python3.5 python3.5-dev python-pip python3-pip python-tk python3-tk
# RUN python3.5 -m pip install PyQt5==5.14
RUN apt-get update && \
    apt-get install -y python3.9 python3.9-dev python-pip python3-pip  python3.9-distutils 
RUN python3.9 -m pip install --upgrade pip wheel setuptools requests
# FROM python3 as python3-ui
RUN apt-get update && \
    apt-get install -y python-tk python3-tk qt5-default qtbase5-dev python3-pyqt5 qttools5-dev-tools
RUN qmake --version
# RUN python3.9 -m pip install PyQt5==5.15.7
RUN echo | pip --version
# RUN brew install pyqt@5
# RUN python3.9 -m pip install PyQt5
# FROM python3-ui as evo1
# Install evo-1 for evaluation
# Hack to avoid Docker's cache when evo-1 master branch is updated.
ADD https://api.github.com/repos/ToniRV/evo-1/git/refs/heads/master version.json
RUN git clone https://github.com/ToniRV/evo-1.git
RUN cd evo-1 && python3.9 $(which pip3) install .

# Install evo-1 for evaluation
# Hack to avoid Docker's cache when evo-1 master branch is updated.
# ADD https://api.github.com/repos/ToniRV/evo-1/git/refs/heads/master version.json
# RUN git clone https://github.com/ToniRV/evo-1.git
# RUN cd evo-1 && python3.5 $(which pip3) install .

# Install spark_vio_evaluation
# RUN python3.5 $(which pip3) install ipython prompt_toolkit
RUN python3.9 $(which pip3) install ipython prompt_toolkit
# Hack to avoid Docker's cache when spark_vio_evaluation master branch is updated.
ADD https://api.github.com/repos/ToniRV/Kimera-VIO-Evaluation/git/refs/heads/master version.json
RUN git clone https://github.com/ToniRV/Kimera-VIO-Evaluation.git
# We use `pip3 install -e .` so that Jinja2 has access to the webiste template...
# RUN cd Kimera-VIO-Evaluation && python3.5 $(which pip3) install -e .
RUN cd Kimera-VIO-Evaluation && python3.9 $(which pip3) install -e .

# Add credentials on build
ARG SSH_PRIVATE_KEY
RUN mkdir /root/.ssh/
RUN echo "${SSH_PRIVATE_KEY}" > /root/.ssh/id_rsa
RUN chmod 600 ~/.ssh/id_rsa

# Make sure your domain is accepted
RUN touch /root/.ssh/known_hosts
RUN ssh-keyscan -t rsa github.com >> /root/.ssh/known_hosts

# Install glog, gflags
RUN apt-get update && apt-get install -y libgflags2.2 libgflags-dev libgoogle-glog0v5 libgoogle-glog-dev

# Install Kimera-VIO
RUN git clone https://github.com/MIT-SPARK/Kimera-VIO.git
RUN cd Kimera-VIO && mkdir build && cd build && cmake .. && make -j2

# Download and extract EuRoC dataset.
RUN apt-get update && apt-get install -y wget
RUN wget http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_01_easy/V1_01_easy.zip
RUN mkdir -p $DIRPATH/euroc && unzip V1_01_easy.zip -d $DIRPATH/euroc

# Yamelize euroc dataset
RUN bash $DIRPATH/Kimera-VIO/scripts/euroc/yamelize.bash -p $DIRPATH/euroc

#CMD xvfb-run $DIRPATH/Kimera-VIO/scripts/stereoVIOEuroc.bash -p $DIRPATH/euroc -r
# CMD $DIRPATH/Kimera-VIO/scripts/stereoVIOEuroc.bash -p $DIRPATH/euroc -r

Run the container (the --privileged doesn't make a difference). This is the kimera_vio_docker.bash file.

xhost +local:root
docker run -it --rm \
    --env="DISPLAY={my address}:0" \
    --volume="/tmp/.X11-unix:/tmp/.X11-unix:rw" \
   --privileged kimera 
xhost -local:root

Run the stereoVIOEuroc.bash script inside the container

./Kimera-VIO/scripts/stereoVIOEuroc.bash -p /root/euroc
export LIBGL_ALWAYS_INDIRECT=1

Console output:

Using dataset at path: /root/euroc
 Launching:

            ██╗  ██╗██╗███╗   ███╗███████╗██████╗  █████╗
            ██║ ██╔╝██║████╗ ████║██╔════╝██╔══██╗██╔══██╗
            █████╔╝ ██║██╔████╔██║█████╗  ██████╔╝███████║
            ██╔═██╗ ██║██║╚██╔╝██║██╔══╝  ██╔══██╗██╔══██║
            ██║  ██╗██║██║ ╚═╝ ██║███████╗██║  ██║██║  ██║
            ╚═╝  ╚═╝╚═╝╚═╝     ╚═╝╚══════╝╚═╝  ╚═╝╚═╝  ╚═╝

I1202 23:57:34.616614   111 EurocDataProvider.cpp:89] Parsing Euroc dataset...
W1202 23:57:34.689756   111 EurocDataProvider.cpp:741] Value for final_k, 10000 is larger than total number of frames in dataset 2912
W1202 23:57:34.689790   111 EurocDataProvider.cpp:744] Using final_k = 2912
I1202 23:57:34.702877   111 RegularVioBackend.cpp:114] Using Regular VIO Backend.
I1202 23:57:34.708120   111 PipelineModule.h:449] MISO Pipeline Module: Display has no output queue registered.
I1202 23:57:34.708556   111 Pipeline.cpp:296] Pipeline Modules launched (parallel_run set to 1).
I1202 23:57:34.708905   117 Pipeline.cpp:89] Spinning Kimera-VIO.
I1202 23:57:34.708945   118 Pipeline.cpp:146] Shutting down VIO pipeline once processing has finished.
I1202 23:57:34.708958   118 Pipeline.cpp:159] Statistics
-----------                                  #  Log Hz  {avg     +- std    }    [min,max]
Display [ms]                                        0
Mesher [ms]                                         0
Stereo Data Provider [ms]                           0
VioBackend [ms]                                     0
VioFrontend [ms]                                    0
Visualizer [ms]                                     0
backend_input_queue Size [#]                        0
data_provider_left_frame_queue Size [#]             0
data_provider_right_frame_queue Size [#]            0
display_input_queue Size [#]                        0
frontend_input_queue Size [#]                       0
mesher_backend Size [#]                             0
mesher_frontend Size [#]                            0
visualizer_backend_queue Size [#]                   0
visualizer_frontend_queue Size [#]                  0
visualizer_mesher_queue Size [#]                    0
I1202 23:57:34.715790   116 EurocDataProvider.cpp:126] Running dataset between frame 50 and frame 2912
W1202 23:57:34.749269   114 Mesher.cpp:1377] Missing landmark information to build 3D Mesh.
E1202 23:57:34.749577   114 Mesher.cpp:1729] Delaunay Triangle out of image (size: x: 0, y: 0, height: 480, width 752
 Triangle: x, y: 
2256, 0
0, 2256
591.777, 435.712
W1202 23:57:34.751188   115 OpenCvVisualizer3D.cpp:1130] No landmark information for Visualizer. Not displaying 3D points.
W1202 23:57:34.754253   111 OpenCvDisplay.cpp:143] Missing Mesh in visualization's 3D widgets.
X Error of failed request:  GLXBadContext
  Major opcode of failed request:  149 (GLX)
  Minor opcode of failed request:  6 (X_GLXIsDirect)
  Serial number of failed request:  40
  Current serial number in output stream:  39
OpenCV Error: Assertion failed (key_ != -1 && "Can't fetch data from terminated TLS container.") in getData, file /root/opencv/modules/core/src/system.cpp, line 1532
./Kimera-VIO/scripts/stereoVIOEuroc.bash: line 110:   111 Segmentation fault      $BUILD_PATH/stereoVIOEuroc --dataset_type="$DATASET_TYPE" --dataset_path="$DATASET_PATH" --initial_k=50 --final_k=10000 --params_folder_path="$PARAMS_PATH" --use_lcd="$USE_LCD" --vocabulary_path="$VOCABULARY_PATH/ORBvoc.yml" --flagfile="$PARAMS_PATH/flags/stereoVIOEuroc.flags" --flagfile="$PARAMS_PATH/flags/Mesher.flags" --flagfile="$PARAMS_PATH/flags/VioBackend.flags" --flagfile="$PARAMS_PATH/flags/RegularVioBackend.flags" --flagfile="$PARAMS_PATH/flags/Visualizer3D.flags" --logtostderr=1 --colorlogtostderr=1 --log_prefix=1 --v=0 --vmodule=Pipeline*=00 --log_output="$LOG_OUTPUT" --log_euroc_gt_data="$LOG_OUTPUT" --save_frontend_images=1 --visualize_frontend_images=1 --output_path="$OUTPUT_PATH"

Please give also the following information:

marcusabate commented 10 months ago

I unfortunately have no way to test with macosx. If people want to reopen this as it's a common problem, I can attempt to investigate. OpenCV issues will hopefully be fixed in a later update, we plan on upgrading to a newer version of OpenCV.