thaytan
commented
4 years ago I am not sure this issue will ever be closed, as there's probably always scope to keep improving the CV. I've implemented the main suggestion above though - the code now projects LEDs to their expected x/y position, and then for each blob looks for an LED that it can match against. That avoids degenerate cases where all the LEDs match against 1 blob, and allows multiple blobs to match against 1 LED.
Closing this issue. I'll open others for future improvement ideas.
One challenging problem in the optical tracking is in deciding which pose hypothesis is the right one and best matches the observed blobs in a frame.
At the moment, I'm using some heuristics that change pretty often, based around taking the hypothesised pose, back projecting the visible LEDs, and then seeing if they land within a visible blob. That mostly works, but doesn't require that LEDs uniquely map to a blob. It's also not desirable to require that they uniquely map to a blob - because LEDs around the edge of the HMD/device can be collinear / overlapping in the projected view.
Mapping projected LED positions into blobs non-uniquely is that there are degenerate cases where every LED lands inside 1 large blob and is considered a strong match.
It would be better to reverse the matching - project LEDs and generate a list of x/y positions, then iterate over each blob and count how many map onto any LED, and calculate the reprojection error of the matching blobs.
Beyond that, we need to develop a solid heuristic for evaluating what constitutes the 'best' match among hypotheses, and if possible to reject bad hypotheses as early as possible as a performance optimisation (because full pose projection and checks are expensive)