Realsense vs oak D Pro

[antithing]

Hi, thanks again for this code. I currently have realsense running, but am looking to switch to OAK-D Pro PoE, to avoid USB cable length restrictions.

Is this oakd model supported?

Are there any performance differences between this sensor and the D455?

Can I run with the callback approach in the mapping example, with oakd? Thanks!

[oseiskar]

We have sometimes had issues with the PoE cameras, which are somewhat hard to reproduce (see, e.g., here https://github.com/SpectacularAI/sdk-examples/issues/35). This may also depend on your network HW, settings, OS and your version of the Luxonis software / firmware. They are related to data not arriving fast enough through the link, which is also more likely to occur with larger resolutions and when not using the built-in accelerated feature tracking. In earlier versions of the device, the active IR depth quality was not as good as in RealSense.

To summarize: use at your own risk; the full functionality of the Spectacular AI SDK may not be available, e.g., due to PoE-related issues, and troubleshooting these issues is currently not a high priority to us. If you really need to get it to work, you can contact us for commercial support.

[antithing]

Thank you! One more question, is the frame latency different with different camera models?

With the D455 I am seeing fluctuating frame times from 25ms through to 65ms. Is this the expected rate?

Are there any variables that can be changed to improve this? I am aiming for a stable 30 frames per second, without the spikes. (Windows, i9 ten core, rtx 3090)

I am timing the frames by using std chrono functions either side of the vio->waitForOutput. Thanks!

[oseiskar]

There is a way to address this, but it's currently an undocumented feature. The spectacularAI::rsPlugin::Configuration has an map<string, string> internalParameters member. Add this there:

config.internalParameters["imuFrontEnd"] = "true"; // note: quotes around "true" are necessary

This should reduce the latency of VIO outputs closer to IMU latency. However, the result also depends on internal latencies in the RS pipeline.

[antithing]

Thanks! That has it running at 32ms per frame, much better, but seems synced to rgb frame rate still. Can I speed it up further by changing the D455 frame rate to 60?

[oseiskar]

You can increase the output rate like this (also currently undocumented):

config.internalParameters["imuFrontEnd"] = "true";
config.internalParameters["outputOnImu"] = "true";
config.internalParameters["outputOnFrames"] = "false";

It should give you > 50Hz with a low latency, with VIO outputs no longer synced to frames. I don't recommend increasing the input frame rate from the D455 to over 30 FPS since several things in the pipeline will start to have trouble keeping up.

[antithing]

Thanks! That shows a frame time of between 0.5ms and 4ms.