Rovio rejects image measurements with 'Latency is too large' message

[shprotello]

Hi,

It is my first experience in MAPLAB use. I installed ROS/MAPLAB on Ubuntu 16.04 64bit according with installation guides.

As a sample data I used a video captured in the office corridor by GoPro Hero 5 Black camera (preliminarily calibrated using kalibr tool).

Raw images and IMU data were extracted from the video file and written to a ROS bag using kalibr_bagcreater utility:

• 24fps for images in JPEG format (gray-scale PNG images were tried too)
• ~60Hz for IMU data

Before running ROVIOLI in VIO mode I set ROVIO_CONFIG_DIR variable to the folder were all data files are located (including rovio_default_config.info one).

After running the following command

rosrun rovioli run_rovioli "" output_map dataset.bag

MAPLAB fails with the error and prints:

I1009 13:58:47.755957 26882 rovio-factory.cc:169] Loading ROVIO configuration template: /dn23/ROBINS/snn/maplab_sandbox/office/rovio_default_config.info
Set distortion parameters (fov) to: w(0.856368)
Registered filter state update callback.
I1009 13:58:48.356084 26882 map-builder-flow.cc:31] Set VIMap folder to: map-24-all_0
I1009 13:58:48.365288 26882 rovioli-node.cc:105] Starting data source...
I1009 13:58:48.365571 26882 rovioli-app.cc:136] 
Message delivery queues:
subscriber-node               queue-topic                   queue-id                      num elements                  
MapBuilderFlow                ROVIO_ESTIMATES               0af1..0000                    0                             
MapBuilderFlow                VIO_UPDATES                   af7e..0000                    0                             
DataPublisherFlow             VIO_UPDATES                   63ff..0000                    0                             
ImuCameraSynchronizerFlow     IMAGE_MEASUREMENTS            29ee..0000                    0                             
MapBuilderFlow                TRACKED_NFRAMES_AND_IMU       a7cc..0000                    0                             
RovioFlow                     IMU_MEASUREMENTS              3abf..0000                    0                             
RovioFlow                     IMAGE_MEASUREMENTS            b913..0000                    0                             
FeatureTrackingFlow           SYNCED_NFRAMES_AND_IMU        7839..0000                    0                             
RovioFlow                     LOCALIZATION_RESULT           cf30..0000                    0                             
ImuCameraSynchronizerFlow     IMU_MEASUREMENTS              b1fc..0000                    0                             
FeatureTrackingFlow           ROVIO_ESTIMATES               2a5c..0000                    0                             
MapBuilderFlow                LOCALIZATION_RESULT           ec0d..0000                    0                             
DataPublisherFlow             RAW_VIMAP                     af1f..0000                    0                             
DataPublisherFlow             ROVIO_ESTIMATES               76b2..0000                    0                             
SyncedNFrameThrottlerFlow     TRACKED_NFRAMES_AND_IMU       1096..0000                    0                             
DataPublisherFlow             LOCALIZATION_RESULT           508e..0000                    0                             
-- Filter: Initializing using accel. measurement ...
-- Filter: Initialized at t = 1539074970.02
[FilterBase::addUpdateMeas] Warning: included measurements at time 1539074970.02 before safeTime 1539074970.02
W1009 13:58:48.417889 26899 rovio-flow.cc:136] ROVIO rejected image measurement. Latency is too large.
*** Aborted at 1539082728 (unix time) try "date -d @1539082728" if you are using GNU date ***
PC: @     0x7f295b33fbc2 aslam::GyroTwoFrameMatcher::matchKeypoint()
*** SIGILL (@0x7f295b33fbc2) received by PID 26882 (TID 0x7f292da5c700) from PID 1530133442; stack trace: ***
    @     0x7f2969398390 (unknown)
    @     0x7f295b33fbc2 aslam::GyroTwoFrameMatcher::matchKeypoint()
    @     0x7f295b34127d aslam::GyroTwoFrameMatcher::match()
    @     0x7f295d53bfff aslam::GyroTracker::track()
    @     0x7f2962afa162 feature_tracking::VOFeatureTrackingPipeline::trackFeaturesSingleCamera()
    @     0x7f2962afcf62 std::_Function_handler<>::_M_invoke()
    @           0x40cdb9 std::__future_base::_State_baseV2::_M_do_set()
    @     0x7f2969395a99 __pthread_once_slow
    @     0x7f2962afd682 _ZNSt17_Function_handlerIFvvEZN5aslam10ThreadPool14enqueueOrderedIRMN16feature_tracking25VOFeatureTrackingPipelineEFvRKN5kindr7minimal26RotationQuaternionTemplateIdEEmPNS1_11VisualFrameESD_PSt6vectorINS1_17FrameToFrameMatchEN5Eigen17aligned_allocatorISF_EEESK_EJPS5_SB_RmRSD_SQ_SK_SK_EEESt6futureINSt9result_ofIFT_DpT0_EE4typeEEmOST_DpOSU_EUlvE_E9_M_invokeERKSt9_Any_data
    @     0x7f2969e89628 aslam::ThreadPool::run()
    @     0x7f2968c0cc80 (unknown)
    @     0x7f296938e6ba start_thread
    @     0x7f296867b41d clone

Could anyone explain me what is wrong with my data?

All files can be found in the archive.

[dymczykm]

Hi @shprotello, thanks for your question and sorry for a late response.

The issue seems t be related to the timestamping and your sensor data. Apparently, your image arrived too late to be taken into account.

I think your issues might come from several sources: 1) The topic names are wrong so ROVIOLI is not getting the sensor data at all 2) The bag is badly timestamped (e.g. missing data or not ordered properly) 3) The IMU frequency of 60 Hz is generally not enough to perform good estimation and I think you might be hitting one of the hard we have set for the IMU data timerange. 4) You are not indicating how did you synchronize the IMU and the camera. While that shouldn't lead to crashes, it probably won't work without proper time calibration. Furthermore, gopro is a global shutter camera which will cause issues with rapid motion.