[BUG] Performance of the spatial detection and object tracker nodes

[Thomas-C258]

Check if issue already exists Created at the suggestion of @luxonis-eric

Describe the bug

I'm trying to optimize the speed at which I can track objects in 3D using an Oak-D-lite. This is a benchmark of the fps loss at each step toward a tracking pipeline: detection / spatial detection / object tracker. I also checked the impact of activating subpixel mode.

In short, I get these results:

• detection node only: 32 fps
• spatial detection only: 27 fps
• spatial detection with subpixel: 18 fps
• spatial tracker: 18 fps

Minimal Reproducible Example speedtestMRE.zip

Expected behavior The spatial detection node is from the documentation just a spatial locator node added to the detection node. From what I understand, it should not be a too taxing operation, yet I see a 5 fps drop.

Then I'm surprised by the loss when activating the subpixel mode (either in default mode or at maximum fractional bits. I get that the depth frame contains more data, but still...

And finally the object tracker, even using the zero-term-imageless option which is documented as the fastest. Object tracking should be faster that NN-based detection, but resources on the oak are probably too limited.

Screenshots speedtests

Additional context

I'm using a custom trained Tiny yolo v7. Preview resolution is 480*480, mono resolutions are 400p. I maximize the framerate by lowering progressively the set fps on mono and color cameras until no further fps gain is achieved.

[SzabolcsGergely]

Subpixel uses 3 "memory slices" vs 1 for standard mode. See here for details. That means there are less resources available for NN. In the output console, there is a warning printed:

[SpatialDetectionNetwork(1)] [warning] Network compiled for 6 shaves, maximum available 11, compiling for 5 shaves likely will yield in better performance

That's why you get 18 fps.

[Thomas-C258]

Thanks for the quick answer ! I recompiled the model for 5 shaves, and I'm indeed back up to 26 fps with subpixel activated for the spatialdetector, so that's great.

About the drop between detection and spatial detection, is that 5 fps drop to be expected, or any mistake I made there too ? One workaround I see is doing the spatial calculation on host, since I'm able to send the depth frame to the host faster than the spatial locator node is working.

[SzabolcsGergely]

The drop shouldn't happen due to spatial detections. How many detections you have? @Thomas-C258 Using latest develop you can measure the runtime for spatial location calculator by setting logger level to trace. export DEPTHAI_LEVEL=trace You can also speed up spatial location calculator if needed, by the newly added APIs. See here setSpatialCalculationStepSize

[Thomas-C258]

Mostly zero when I'm doing the speed testing (seems like the spatialdetector sends an empty detections message when it detects nothing, perfect for a speed test), but framerate stays the same when I have 1-5 detected objects in the frame.

I'll check that out, thanks !