Error trying to use slam on a video

[pouyaAB]

I calibrated my camera using OpenCV. I extracted fx, cx, fy, cy, k1, k2, p1, p2, k3 and add them to my camera config file like this.

# perspective model for Pixel 2

#==============#
# Camera Model #
#==============#
Camera.name: "Pixel 2"
Camera.setup: "monocular"
Camera.model: "perspective"

Camera.fps: 30.0
Camera.cols: 360
Camera.rows: 640

Camera.fx: 610.25
Camera.cx: 180.0
Camera.fy: 610.25
Camera.cy: 320.0

Camera.k1: 0.060448
Camera.k2: -1.0096
Camera.p1: 0.0
Camera.p2: 0.0
Camera.k3: 2.86

Camera.color_order: "RGB"

I used my phone camera and record a video. I tried to run run_image_slam like this:

./run_video_slam -v ../orb_vocab.dbow2 -m ../map.mp4 -c ../config.yaml

but I get this error when I run the command:

[2019-09-26 13:23:27.514] [D] CONSTRUCT: config
[2019-09-26 13:23:27.514] [I] config file loaded: ../config.yaml
[2019-09-26 13:23:27.514] [D] load camera model type
[2019-09-26 13:23:27.514] [D] load camera model parameters
[2019-09-26 13:23:27.515] [D] CONSTRUCT: camera::base
[2019-09-26 13:23:27.515] [D] CONSTRUCT: camera::perspective
[2019-09-26 13:23:27.515] [D] compute image bounds
[2019-09-26 13:23:27.515] [D] load ORB parameters
[2019-09-26 13:23:27.515] [D] load tracking parameters
[2019-09-26 13:23:27.515] [D] load depth threshold
[2019-09-26 13:23:27.515] [D] load depthmap factor
[2019-09-26 13:23:27.517] [D] CONSTRUCT: system
  ___               __   _____ _      _   __  __ 
 / _ \ _ __  ___ _ _\ \ / / __| |    /_\ |  \/  |
| (_) | '_ \/ -_) ' \\ V /\__ \ |__ / _ \| |\/| |
 \___/| .__/\___|_||_|\_/ |___/____/_/ \_\_|  |_|
      |_|                                        

Copyright (C) 2019,
National Institute of Advanced Industrial Science and Technology (AIST)
All rights reserved.

This is free software,
and you are welcome to redistribute it under certain conditions.
See the LICENSE file.

Camera Configuration:
- name: Pixel 2
- setup: Monocular
- fps: 30
- cols: 360
- rows: 640
- color: RGB
- model: Perspective
  - fx: 610.25
  - fy: 610.25
  - cx: 180
  - cy: 320
  - k1: 0.060448
  - k2: -1.0096
  - p1: 0
  - p2: 0
  - k3: 2.86
  - min x: 3.42977
  - max x: 356.57
  - min y: 6.09736
  - max y: 633.903
ORB Configuration:
- number of keypoints: 2000
- scale factor: 1.2
- number of levels: 8
- initial fast threshold: 20
- minimum fast threshold: 7
- mask rectangles:
  - [0, 1, 0, 0.1]
  - [0, 1, 0.84, 1]
Tracking Configuration:

[2019-09-26 13:23:27.517] [I] loading ORB vocabulary: ../orb_vocab.dbow2
[2019-09-26 13:23:27.800] [D] CONSTRUCT: data::camera_database
[2019-09-26 13:23:27.800] [D] CONSTRUCT: data::map_database
[2019-09-26 13:23:27.800] [D] CONSTRUCT: data::bow_database
[2019-09-26 13:23:27.800] [D] CONSTRUCT: publish::frame_publisher
[2019-09-26 13:23:27.800] [D] CONSTRUCT: publish::map_publisher
[2019-09-26 13:23:27.800] [D] CONSTRUCT: module::initializer
[2019-09-26 13:23:27.800] [D] CONSTRUCT: module::relocalizer
[2019-09-26 13:23:27.800] [D] CONSTRUCT: tracking_module
[2019-09-26 13:23:27.800] [D] CONSTRUCT: mapping_module
[2019-09-26 13:23:27.800] [D] CONSTRUCT: global_optimization_module
[2019-09-26 13:23:27.800] [I] startup SLAM system
[2019-09-26 13:23:27.800] [I] start global optimization module
[2019-09-26 13:23:27.800] [I] start mapping module
[2019-09-26 13:23:27] [connect] Successful connection
[2019-09-26 13:23:27] [connect] WebSocket Connection 127.0.0.1:3000 v-2 "WebSocket++/0.8.1" /socket.io/?EIO=4&transport=websocket&t=1569518607 101
[2019-09-26 13:23:27.801] [I] connected to server
terminate called recursively
terminate called after throwing an instance of 'std::out_of_range'
*** Aborted at 1569518607 (unix time) try "date -d @1569518607" if you are using GNU date ***
  what():  vector::_M_range_check: __n (which is 0) >= this->size() (which is 0)
terminate called recursively
PC: @     0x7f7fb7c74428 gsignal
*** SIGABRT (@0x3e8000032bc) received by PID 12988 (TID 0x7f7f957fa700) from PID 12988; stack trace: ***
    @     0x7f7fb85b2390 (unknown)
    @     0x7f7fb7c74428 gsignal
    @     0x7f7fb7c7602a abort
    @     0x7f7fb82ae7dd __gnu_cxx::__verbose_terminate_handler()
    @     0x7f7fb82ac6b6 (unknown)
    @     0x7f7fb82ab6a9 (unknown)
    @     0x7f7fb82ac005 __gxx_personality_v0
    @     0x7f7fb8018f83 (unknown)
    @     0x7f7fb8019487 _Unwind_Resume
    @     0x7f7fbaae315a _ZNK9openvslam7feature13orb_extractor16initialize_nodesB5cxx11ERKSt6vectorIN2cv8KeyPointESaIS4_EEiiii
    @     0x7f7fbaae4987 openvslam::feature::orb_extractor::distribute_keypoints_via_tree()
    @     0x7f7fbaae51e6 _ZNK9openvslam7feature13orb_extractor22compute_fast_keypointsERSt6vectorIS2_IN2cv8KeyPointESaIS4_EESaIS6_EERKNS3_3MatE._omp_fn.0
    @     0x7f7fafae043e (unknown)
    @     0x7f7fb85a86ba start_thread
    @     0x7f7fb7d4641d clone
    @                0x0 (unknown)
Aborted (core dumped)

I also tried to use the 'run_slam_images' example after converting my video to a sequence of images but it didn't help and I got the same error.

Honestly, I don't know what is the vocabulary file, and if I can use the one provided here for my purpose. I would appreciate any help so that I can understand the error.

[pouyaAB]

I ended up changing the scale_factor in config file from 1.2 to 1.0 and works without any error now. Can somebody explains what does this scale factor mean? Even though the code is running without errors now, the algorithm is working properly.

[kyawakyawa]

Hi, the error you encountered seems to be caused by openvslam's assumption that input images are landscape when feature points are extracted. We're dealing with this issue at the moment.

[shinsumicco]

@pouyaAB Could you try the PR at #132 ?

[pouyaAB]

@pouyaAB Could you try the PR at #132 ?

The PR #132 is working for me. Thanks.