search

david-cattermole / mayaMatchMoveSolver

A Bundle Adjustment solver for MatchMove related tasks.
https://david-cattermole.github.io/mayaMatchMoveSolver/
Other
101 stars 29 forks source link

Solver UI - Calculate Optimal Root Frame Numbers #160

Closed david-cattermole closed 1 year ago

david-cattermole commented 3 years ago

Feature

Add the ability to automatically calculate the optimal Root Frame numbers based on the markers in the Solver.

The "Standard" Solver UI tab contains a user-settable Root Frames option which currently needs to be set manually. For the artist, it's not very intuitive because most people don't know which frames to set.

Ideally, the Root Frame values should be computed automatically and always produce the best result. However realistically, this might not be possible.

Therefore, an option to automatically calculate the root frames and set the values will be helpful. Artists can then change the calculated values manually.

Example Usage Steps

  1. Create Collection, and add Markers and Attributes.
  2. Run tool
    • Root Frames are calculated and set.
  3. Artist adds a new Marker.
  4. Run tool
    • Root Frames are calculated (with new Marker) and set.
  5. Click "Solve" button.
david-cattermole commented 3 years ago

For now, I have added a basic "Set Auto" button which will set the Root Frames for the Standard Solver. These root frames are not yet "optimal", however the results on the unit tests in mmSolver have dramatically reduced deviation, in some cases from ~11 pixels to ~2 pixels, which is better than nothing.

Future work is needed, but the "Set Auto" currently committed should be OK for general use, until we can implement something better.

Future work...

There appears to be some interesting research papers on this topic: https://www.researchgate.net/publication/260616120_Optimal_key-frame_selection_for_video-based_structure-from-motion

The paper uses the error level of homography and fundamental matrices of two frames to detect "degenerate" frames (pairs of frames that will not compute an accurate camera - for example because there is not enough parallax).

Using homography and fundamental matrices requires the maths (or library) to calculate the matrices, such as in libmv (#144), which is not planned until at least v0.4.0.

Further, there are use-cases in mmSolver where the Markers may not be produce valid fundamental/homography matrices as the solver is not constrained to only solving cameras, or only solving objects. Such fundamental/homography matrices also assume only Markers that solve all three Bundle axes; X, Y and Z, but mmSolver is not constrained by this and therefore these techniques will not work.

david-cattermole commented 1 year ago

This has been implemented as the "Per-Marker" and "Span" check-boxes in the Solver UI. https://david-cattermole.github.io/mayaMatchMoveSolver/tools_solver_ui.html#user-and-root-frames

Closing.