search

MIT-SPARK / Kimera-VIO

Visual Inertial Odometry with SLAM capabilities and 3D Mesh generation.
BSD 2-Clause "Simplified" License
1.53k stars 416 forks source link

90 degree rotated camera setup #222

Open Deok-min opened 8 months ago

Deok-min commented 8 months ago

Hi, I'm trying to run Kimera-VIO with a 90 degree rotated d435i camera. But the constructed 3d mesh is not fine. It's like below video( It is 5 times speed up). Could you give some advice to construct good meshes? kimera

I run the below commands under the non-parallel configuration.

roslaunch kimera_vio_ros kimera_vio_ros_realsense_IR.launch
rviz -d $(rospack find kimera_vio_ros)/rviz/kimera_vio_tesse.rviz

and during the runtime, it says,

I1221 02:29:35.219280 22445 StereoVisionImuFrontend.cpp:113] ------------------- Processing frame k = 4--------------------
I1221 02:29:35.219341 22445 StereoVisionImuFrontend.cpp:138] Current IMU Preintegration frequency: 172414 Hz. (58 us).
I1221 02:29:35.221277 22445 Tracker.cpp:147] Optical Flow Timing [ms]: 1
I1221 02:29:35.221923 22445 ThreadsafeQueue.h:305] Queue with id: mesher_frontend is getting full, size: 3
W1221 02:29:35.221935 22445 PipelineModule.h:399] Module: VioBackend - Input queue: backend_input_queue didn't return an output.
I1221 02:29:35.221944 22445 MesherModule.cpp:38] Module: Mesher - Backend queue is empty or down
I1221 02:29:35.221951 22445 Visualizer3DModule.cpp:62] Module: Visualizer - visualizer_backend_queue queue is down
I1221 02:29:35.221958 22445 KimeraVioRos.cpp:244] Statistics
-----------                                        #    log hz  {avg +- std}    [min, max]  last
Display [ms]                                         1  9.2 {0.0 +- 0.0}    [0.0, 0.0]  0.0
Mesher [ms]                                          1  657.0   {1.0 +- 0.0}    [1.0, 1.0]  1.0
VioBackend [ms]                                      1  1342.4  {0.0 +- 0.0}    [0.0, 0.0]  0.0
VioFrontend Frame Rate [ms]                          3  56.0    {2.7 +- 1.2}    [2.0, 4.0]  2.0
VioFrontend [ms]                                     4  39.3    {13.2 +- 21.2}  [2.0, 45.0] 2.0
Visualizer [ms]                                      1  4690.6  {0.0 +- 0.0}    [0.0, 0.0]  0.0
backend_input_queue Size [#]                         1  4.1 {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_left_frame_queue Size [#]              5  22.0    {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_right_frame_queue Size [#]             5  22.0    {1.2 +- 0.4}    [1.0, 2.0]  1.0
display_input_queue Size [#]                         1  4.1 {1.0 +- 0.0}    [1.0, 1.0]  1.0
frontend_input_queue Size [#]                        4  13.5    {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_backend Size [#]                              1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_frontend Size [#]                             4  17.5    {1.8 +- 1.0}    [1.0, 3.0]  3.0
visualizer_backend_queue Size [#]                    1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_frontend_queue Size [#]                   1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_mesher_queue Size [#]                     1  5.7 {1.0 +- 0.0}    [1.0, 1.0]  1.0
I1221 02:29:35.229018 22445 Pipeline.cpp:102] Spinning Kimera-VIO.
I1221 02:29:35.229038 22445 MonoDataProviderModule.cpp:135] Module: Stereo Data Provider - queue is empty or down
I1221 02:29:35.229043 22445 KimeraVioRos.cpp:244] Statistics
-----------                                        #    log hz  {avg +- std}    [min, max]  last
Display [ms]                                         1  9.2 {0.0 +- 0.0}    [0.0, 0.0]  0.0
Mesher [ms]                                          1  657.0   {1.0 +- 0.0}    [1.0, 1.0]  1.0
VioBackend [ms]                                      1  1342.4  {0.0 +- 0.0}    [0.0, 0.0]  0.0
VioFrontend Frame Rate [ms]                          3  56.0    {2.7 +- 1.2}    [2.0, 4.0]  2.0
VioFrontend [ms]                                     4  39.3    {13.2 +- 21.2}  [2.0, 45.0] 2.0
Visualizer [ms]                                      1  4690.6  {0.0 +- 0.0}    [0.0, 0.0]  0.0
backend_input_queue Size [#]                         1  4.1 {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_left_frame_queue Size [#]              5  22.0    {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_right_frame_queue Size [#]             5  22.0    {1.2 +- 0.4}    [1.0, 2.0]  1.0
display_input_queue Size [#]                         1  4.1 {1.0 +- 0.0}    [1.0, 1.0]  1.0
frontend_input_queue Size [#]                        4  13.5    {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_backend Size [#]                              1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_frontend Size [#]                             4  17.5    {1.8 +- 1.0}    [1.0, 3.0]  3.0
visualizer_backend_queue Size [#]                    1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_frontend_queue Size [#]                   1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_mesher_queue Size [#]                     1  5.7 {1.0 +- 0.0}    [1.0, 1.0]  1.0
I1221 02:29:35.238714 22445 Pipeline.cpp:102] Spinning Kimera-VIO.
I1221 02:29:35.238734 22445 MonoDataProviderModule.cpp:135] Module: Stereo Data Provider - queue is empty or down
I1221 02:29:35.238740 22445 KimeraVioRos.cpp:244] Statistics
-----------                                        #    log hz  {avg +- std}    [min, max]  last
Display [ms]                                         1  9.2 {0.0 +- 0.0}    [0.0, 0.0]  0.0
Mesher [ms]                                          1  657.0   {1.0 +- 0.0}    [1.0, 1.0]  1.0
VioBackend [ms]                                      1  1342.4  {0.0 +- 0.0}    [0.0, 0.0]  0.0
VioFrontend Frame Rate [ms]                          3  56.0    {2.7 +- 1.2}    [2.0, 4.0]  2.0
VioFrontend [ms]                                     4  39.3    {13.2 +- 21.2}  [2.0, 45.0] 2.0
Visualizer [ms]                                      1  4690.6  {0.0 +- 0.0}    [0.0, 0.0]  0.0
backend_input_queue Size [#]                         1  4.1 {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_left_frame_queue Size [#]              5  22.0    {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_right_frame_queue Size [#]             5  22.0    {1.2 +- 0.4}    [1.0, 2.0]  1.0
display_input_queue Size [#]                         1  4.1 {1.0 +- 0.0}    [1.0, 1.0]  1.0
frontend_input_queue Size [#]                        4  13.5    {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_backend Size [#]                              1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_frontend Size [#]                             4  17.5    {1.8 +- 1.0}    [1.0, 3.0]  3.0
visualizer_backend_queue Size [#]                    1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_frontend_queue Size [#]                   1  5.8 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_mesher_queue Size [#]                     1  5.7 {1.0 +- 0.0}    [1.0, 1.0]  1.0
I1221 02:29:35.248896 22445 Pipeline.cpp:102] Spinning Kimera-VIO.
I1221 02:29:35.248929 22445 MonoDataProviderModule.cpp:135] Module: Stereo Data Provider - queue is empty or down
I1221 02:29:35.248939 22445 StereoVisionImuFrontend.cpp:113] ------------------- Processing frame k = 5--------------------
I1221 02:29:35.248968 22445 StereoVisionImuFrontend.cpp:138] Current IMU Preintegration frequency: 416667 Hz. (24 us).
I1221 02:29:35.250912 22445 Tracker.cpp:147] Optical Flow Timing [ms]: 1
I1221 02:29:35.252046 22445 Tracker.cpp:367] Low mono disparity.
I1221 02:29:35.275588 22445 FeatureDetector.cpp:208] Number of points detected : 1128
I1221 02:29:35.275607 22445 FeatureDetector.cpp:222] Need n corners: 75
I1221 02:29:35.275732 22445 NonMaximumSuppression.cpp:102] Running Binning: 6
I1221 02:29:35.275751 22445 NonMaximumSuppression.cpp:119] Non Maximum Suppression Timing [ms]: 0
I1221 02:29:35.280579 22445 FeatureDetector.cpp:293] Corner Sub Pixel Refinement Timing [ms]: 4
I1221 02:29:35.280735 22445 StereoMatcher.cpp:130] sparseStereoMatching: StereoFrame is already rectified!
I1221 02:29:35.280742 22445 StereoCamera.cpp:273] Rectifying already rectified stereo frame ...
I1221 02:29:35.294643 22445 StereoVisionImuFrontend.cpp:180] Keyframe 5 with: 295 smart measurements
I1221 02:29:35.294698 22445 ThreadsafeQueue.h:305] Queue with id: mesher_frontend is getting full, size: 4
I1221 02:29:35.294895 22445 RegularVioBackend.cpp:214] Tracker has a LOW_DISPARITY status.
I1221 02:29:35.295295 22445 VioBackend.cpp:1375] Backend: Update IMU Bias.
W1221 02:29:35.295351 22445 Mesher.cpp:1455] Missing landmark information to build 3D Mesh.
I1221 02:29:35.296854 22445 ThreadsafeQueue.h:305] Queue with id: display_input_queue is getting full, size: 2
I1221 02:29:35.326288 22445 KimeraVioRos.cpp:244] Statistics
-----------                                        #    log hz  {avg +- std}    [min, max]  last
Display [ms]                                         2  7.5 {14.5 +- 20.5}  [0.0, 29.0] 29.0
Mesher [ms]                                          2  15.3    {1.0 +- 0.0}    [1.0, 1.0]  1.0
VioBackend [ms]                                      2  15.4    {0.0 +- 0.0}    [0.0, 0.0]  0.0
VioFrontend Frame Rate [ms]                          3  56.0    {2.7 +- 1.2}    [2.0, 4.0]  2.0
VioFrontend Keyframe Rate [ms]                       1  7.9 {45.0 +- 0.0}   [45.0, 45.0]    45.0
VioFrontend [ms]                                     5  28.6    {19.6 +- 23.2}  [2.0, 45.0] 45.0
Visualizer [ms]                                      2  15.5    {0.0 +- 0.0}    [0.0, 0.0]  0.0
backend_input_queue Size [#]                         2  5.4 {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_left_frame_queue Size [#]              6  23.4    {1.0 +- 0.0}    [1.0, 1.0]  1.0
data_provider_right_frame_queue Size [#]             6  23.4    {1.2 +- 0.4}    [1.0, 2.0]  1.0
display_input_queue Size [#]                         3  8.0 {1.3 +- 0.6}    [1.0, 2.0]  2.0
frontend_input_queue Size [#]                        5  15.4    {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_backend Size [#]                              2  6.6 {1.0 +- 0.0}    [1.0, 1.0]  1.0
mesher_frontend Size [#]                             5  16.6    {2.2 +- 1.3}    [1.0, 4.0]  4.0
visualizer_backend_queue Size [#]                    2  6.6 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_frontend_queue Size [#]                   2  6.6 {1.0 +- 0.0}    [1.0, 1.0]  1.0
visualizer_mesher_queue Size [#]                     2  6.6 {1.0 +- 0.0}    [1.0, 1.0]  1.0
I1221 02:29:35.326620 22445 Pipeline.cpp:102] Spinning Kimera-VIO.
I1221 02:29:35.326651 22445 MonoDataProviderModule.cpp:135] Module: Stereo Data Provider - queue is empty or down
I1221 02:29:35.326666 22445 StereoVisionImuFrontend.cpp:113] ------------------- Processing frame k = 6--------------------

thank you for reading.