Closed antithing closed 8 months ago
We have sometimes had issues with the PoE cameras, which are somewhat hard to reproduce (see, e.g., here https://github.com/SpectacularAI/sdk-examples/issues/35). This may also depend on your network HW, settings, OS and your version of the Luxonis software / firmware. They are related to data not arriving fast enough through the link, which is also more likely to occur with larger resolutions and when not using the built-in accelerated feature tracking. In earlier versions of the device, the active IR depth quality was not as good as in RealSense.
To summarize: use at your own risk; the full functionality of the Spectacular AI SDK may not be available, e.g., due to PoE-related issues, and troubleshooting these issues is currently not a high priority to us. If you really need to get it to work, you can contact us for commercial support.
Thank you! One more question, is the frame latency different with different camera models?
With the D455 I am seeing fluctuating frame times from 25ms through to 65ms. Is this the expected rate?
Are there any variables that can be changed to improve this? I am aiming for a stable 30 frames per second, without the spikes. (Windows, i9 ten core, rtx 3090)
I am timing the frames by using std chrono functions either side of the vio->waitForOutput. Thanks!
There is a way to address this, but it's currently an undocumented feature. The spectacularAI::rsPlugin::Configuration
has an map<string, string> internalParameters
member. Add this there:
config.internalParameters["imuFrontEnd"] = "true"; // note: quotes around "true" are necessary
This should reduce the latency of VIO outputs closer to IMU latency. However, the result also depends on internal latencies in the RS pipeline.
Thanks! That has it running at 32ms per frame, much better, but seems synced to rgb frame rate still. Can I speed it up further by changing the D455 frame rate to 60?
You can increase the output rate like this (also currently undocumented):
config.internalParameters["imuFrontEnd"] = "true";
config.internalParameters["outputOnImu"] = "true";
config.internalParameters["outputOnFrames"] = "false";
It should give you > 50Hz with a low latency, with VIO outputs no longer synced to frames. I don't recommend increasing the input frame rate from the D455 to over 30 FPS since several things in the pipeline will start to have trouble keeping up.
Thanks! That shows a frame time of between 0.5ms and 4ms.
Hi, thanks again for this code. I currently have realsense running, but am looking to switch to OAK-D Pro PoE, to avoid USB cable length restrictions.
Is this oakd model supported?
Are there any performance differences between this sensor and the D455?
Can I run with the callback approach in the mapping example, with oakd? Thanks!