FPS problem in ros version

[konszy]

Hi, great job! Thanks for sharing dm-vio.

I tested the slides 1 sequence of TUMVI datasets, and got 40-60 fps by python tool, the result is quite good. But it only had 10-20 track fps in ROS version and the frame loss caused serious drift, which made the test impossible to go on (usually about 10 seconds will lose tracking). Any idea what I got wrong?

By the way, the live test using ROS is ok, the tracking frame rate is maintained at 29-30 frames. The euroc dataset works also fine in ROS version, 19-20fps.

[lukasvst]

Hi, thanks!

For me TUM-VI slides 1 works well with the ROS version. Which command do you use for running it? Note that the Rosbag does not have exposure times, hence you need to switch to mode 3 (as mode 0 expects perfect calibration and mode 1 does not use the vignette at all).

Can you share the timings.txt file that DM-VIO saves (ideally for both, the python-tool version of TUM-VI (in realtime mode so it's comparable) and for the Rosbag one which is slow)? It should give insights into which part is slow.

One thing I found: Have you disabled \ enabled Pangolin GUI for both commands? If the correct graphics driver is not installed, this can sometimes impact performance.

[konszy]

Hi, thanks for your feedback.

I was using mode1 before and I switched to mode3 as suggested.

The current frame rate problem has not improved, and I still don't quite understand the meaning of the four parameters in the time file. I uploaded two versions of the time file, sinde the ros version will crash due to the tracking cannot be restored, so the time file of the ros version is generated after running for about 10 seconds.

timings (ros).txt timings(python-tool).txt

I tried both nogui=0 and nogui=1, there is not much difference since they both have low fps in ros version. Here are the commands I used for running ros version.

roscore

source devel/setup.bash rosrun dmvio_ros node calib=/omit/slam/dm-vio/configs/tumvi_calib/camera02.txt gamma=/omit/slam/dm-vio//configs/tumvi_calib/pcalib.txt settingsFile=/omit/slam/dm-vio/configs/tumvi.yaml
imuCalib=/omit/slam/dm-vio/configs/tumvi_calib/camchain.yaml mode=3 nogui=0 nolog=1 preset=1 useimu=1 quiet=1 init_requestFullResetNormalizedErrorThreshold=0.8 init_pgba_skipFirstKFs=1

rosbag play /omit/slam/dataset/TumVi/dataset-slides1_512_16.bag

Thanks for the help again！

[lukasvst]

One thing I see in the command is that you have not passed the vignette (it's in any of the dso format dataset files). vignette=/PATH/TO/VI_DATASET/dataset-corridor1_512_16/dso/cam0/vignette.png But this should not impact the times, just the tracking quality.

The 3 values in each row in the timings.txt file are:

• mean time of this component \ method
• variance of the time.
• maximum time
• number of samples. E.g. if addActiveFrame is called 692 times, the mean, variance and maximum time it took are reported.

For both of the times files you uploaded, the main system seems to be very fast, addActiveFrame takes just 13ms so it should be capable of over 70FPS. This suggests, that the problem is in the ROS driver, or there is a problem with the rosbag replay \ loading the images from your hard drive. I would suggest to try to profile the application, or you can add TimeMeasurements in the ROS driver. There must be some bottleneck before calling addActiveFrame in https://github.com/lukasvst/dm-vio-ros/blob/master/src/main.cpp#L122

[konszy]

Thank for the helping!

After testing many times, I found that if the dmvio system is stable, the frame rate is controlled at around 20 fps, because the frame rate of the dataset is 20 fps, and if you use the command "rosbag play -r 2", the frame rate goes up to 40 fps (as much as the computing device allows).

Each time the dmvio system is re-initialized, the frame rate drops, but this occurs relatively infrequently, for example, in the slide scenario in the slide1 dataset.

The problem now is that when I use "rosbag play" on my computer (m1pro macbook using Parallels Desktop), it lags(attached with screenshot) (cpu is abundant), which causes the lost of information.

line4 1520530310.648636 to line5 1520530311.125953

If I run “rosbag play” directly in the system, the above problem does not occur, is there any idea how to solve it???

Euroc dataset does not have similar problems, I guess the file size still affects the rosbag reading. So, I tried mh01(size of 2.1g) and room01(size of 3g), euroc works without problem and tumvi had same problem.

[konszy]

Problem solved, I used "rosbag compress --lz4 dataset-slides1_512_16.bag", the size became 5.6g instead of 5.9g.

I didn't found any difference between lz4 and bz2 compress when running the dmvio. It should be ok.

[lukasvst]

I'm glad you where able to resolve the issue.

For future reference: As of today, the ROS driver also supports running directly on rosbags in non-realtime mode, by providing the argument rosbag (see https://github.com/lukasvst/dm-vio-ros#new-now-you-can-also-run-in-non-realtime-mode-on-rosbags) This will process a prerecorded dataset as fast as possible.