Question: How to process moving frames?

[onthegrid007]

Hello devs,

I am working with a DepthAI camera from Luxonis and this is by far the fastest and most accurate April tag implementation I have found so far. My problem is with even the slightest camera movement I assume the frames are blurry and fails to detect even the most obvious less than 6 inches away tag. I have tried the built in depthai-core april tag tracker and it is much worse than this implementation. My question is: Is there a way to almost overlay the frames to reduce blurriness to track the tags during movement or by some other implementation of possibly keeping track of the last April tags found with their location and rotation and possibly use an algorithm to track it over frames? There is also a built in 3axis accelerometer built into the camera and raw xyz and quat if this helps at all. I'm looking at this for a personal project any help would be appreciated...

[mkrogius]

Can you post an example image that fails to be detected, please?

[onthegrid007]

It's a live video feed. Let me make some modifications and see if I can't grab a few stills

[onthegrid007]

3 Frames in order

[mkrogius]

The middle frame has too much motion blur for the apriltag to be detected. I'd suggest either putting the apriltag closer to the camera or reducing the exposure in order to get a detection. I don't think frame averaging will be helpful.

This repo is only focused on single frame detection, not tracking, so it can't help with implementing a tracking solution. That said, using the IMU to continue tracking the camera when the apriltag is blurry, would probably work well

[onthegrid007]

Thank you