request rpc or syncNode->controlInput so can dynamically change the sync threshold

[diablodale]

Why

I implemented host-side cross-frame synchronization 2 years ago to workaround #440. Recently tried dai::node::Sync. Unfortunately...dai::node::Sync is 15% slower (fps from 23 -> 20fps) with no other changes to code. Before I stop using it, a clearer understanding in documentation is needed. And, better tuning capabilities in the node itself.

What

1. clear implementation documentation, including flowcharts, of the logic implemented in dai::node::Sync. This is needed so insightful tuning can be done. Random guessing at threshold values without clearly understanding the logic on how it is used it not productive.
2. much better tuning capabilities for dai::node::Sync including, but not limited to, the ability to dynamically at-runtime change the dai::node::Sync "sync threshold" so it can be dynamically tuned for the specific computer, usb ports, OAK device, OAK device pipeline configuration, etc. Having a single fixed value that can only be set before the pipeline is started forbids any productive tuning.

How

1. do an RPC to the sync node, make an API.
2. Or setup something similar to how the color camera and depth nodes have control inputs and dai::CameraControl

[moratom]

Thanks for the request.

Would you be willing to share a MRE that showcases this slowdown using sync node? In general the sync node should be really light weight in terms of CPU power so the FPS drop should be explainable with more dropped frames.

In short the syncs node works run loop looks like this (will be expanded in the documentation CC: @jakaskerl):

1. Get all messages that are connected to the sync node
2. Check if messages are synced - meaning minimum and maximum timestamp differ less than the set threshold
3. Branchout
   • If the messages are synced:
     • compile the MessageGroup and send it out
   • If the messages are not synced:
     • get the message that had the oldest timestamp
     • go back to 2.

[moratom]

much better tuning capabilities for dai::node::Sync including, but not limited to, the ability to dynamically at-runtime change the dai::node::Sync "sync threshold" so it can be dynamically tuned for the specific computer, usb ports, OAK device, OAK device pipeline configuration, etc. Having a single fixed value that can only be set before the pipeline is started forbids any productive tuning

Added to our task list.

[diablodale]

No MRE. I don't provide code or detailed bug reports to Luxonis anymore. Your team didn't move on my high-quality PRs and bugs so I retracted/closed many of them and not doing that anymore. themarpe can bring you up-to-speed if you need details.

My mistake, I wasn't clear on the 15% fps slowdown.

• The reference is my host-side sync code...it achieves ~23 fps in my specific test scenario.
• Removed my sync code and added depthai Sync node. That Sync test was ~20fps aka 15% less.

The OAK gen2 devices are often resource limited (when producing color+depth with my checks/aligns/resolution). I suspect that either: 1) the test +node::Sync is further stressing the OAK device, or 2) the sync/bucketing methodology used by node::Sync results in fewer sync'd groups/second compared to my host-side sync code. (2) is why I request the methodology documentation and tuning.

Of course, there is more detail in node::Sync methodology. How it interacts with input queues. How it internally caches frames to find matches. How it drops frames that have no matches (frame rates are not perfectly equal across inputs, and may also slightly vary). How many frames can it cache in device memory and how does that resource use affect other activities on the device. I acknowledge some of that is implementation/feature spec. There is likely a good-middle-detail level that combined with real-time tuning can assist.