What thread is the slowest in ORB_SLAM2?

[DigitalGabriele]

I am trying to figure out which thread takes the longest time to do its job and which module of that thread is the slow one.

[AlejandroSilvestri]

@DigitalGabriele ,

The main thread, where Tracking runs on, is the slowest one.

Slowest algorithm, in order:

• FAST feature detector
• ORB (BRIEF descriptor extractor)

[DigitalGabriele]

@AlejandroSilvestri, I timed the amount of seconds that are needed to execute ComputeKeyPointsOctTree() in OrbExtractor. On average it takes around 0.03 seconds. I wouldnt say it's very slow. Or am I looking at the wrong FAST?

Because I have only found cv::FAST being called in ORBextractor.cc

[AlejandroSilvestri]

@DigitalGabriele ,

That's where FAST is.

Of course, time depends on resolution and hardware.

If you want 30 fps you only has 33 ms per frame, so the 30 ms you measured used almost all the time available.

But keep in mind that SLAM is actually PTAM (parallel tracking and mapping), parallel instead of simultaneous, because tracking happens on every frame and mapping only on each keyframe.

Mapping one keyframe is very slow comparing with tracking one frame. The idea on using keyframes is to avoid worthless work on every frame. There are times where keyframes pile up in the queue to be processed, and times where local mapping is iddle.

So, tracking has a lot of permanent work, mapping has intermittent work.

[DigitalGabriele]

@AlejandroSilvestri, Thank you for your quick and good answer. I was wondering if I could replace FAST with the FAST using machine learning (based on Rosten paper et al). When I ran a few examples of just FAST and timed it, I received on average 0.00000007 seconds. It does not seem that a FAST using machine learning could improve much.

[AlejandroSilvestri]

@DigitalGabriele , you are welcome.

I don't believe it can improve from 30 ms to 70 ns. It would be nice to compare both on same image or stream. FAST is available on OpenCV.

Now I'm very interested in your results. Please let me know.

[DigitalGabriele]

@AlejandroSilvestri Hah! Thanks. But I am not very wishful here to improve from 70 ns to anything less than that.

[buemi]

@DigitalGabriele How did you measured the single methods time requirement?

[AlejandroSilvestri]

@DigitalGabriele

I recommend using OpenCV's handy object TickMeter,:

#include <opencv2/core/utility.hpp> // or #include <opencv2/core.hpp>
TickMeter tm;
tm.start();
// execute function of interest
tm.stop();
double duration =  tm.getTimeMilli();  // or getTimeMicro()
tm.reset();  // If you want to prepare the same object for another run without destroying it

You can put the code inside the method (at beginning and ending) or outside wrapping its call. Keep in mind their execution time vary highly with the scene. You can register peak time, or (more challenging) mean of 10% slowest execution.

And this would be a lot better if you can track down the context where you get slow execution (for example, rotation with a lot of new mappoints), and even better if you can repeat the whole orb-slam2 with the same exact initialization.

One way to achieve repeatability is loading an initial map (orb-slam2 doesn't save not load maps, you need a mod).

If you can make the effort, I recommend measuring every execution of every function of interest (FOI), and send the result to a "timing object" that can register this data (let's say a vector per FOI) and save it to a ascii or yaml file at the end, expressing the time in us or something readable (not in ticks). Then you paste the table to a spreadsheet (like Google Sheets, Calc, Excel) to analyse and graph.

For every FOI execution let's register duration and timestamp. This last one can be useful to relate to other FOI registers.

[buemi]

Thanks a lot!