epfl-cs358 / 2024sp-helping-hand

2 stars 0 forks source link

📦 NEW: filter output of sam and tansform to expected output #57

Closed violoncelloCH closed 1 month ago

violoncelloCH commented 1 month ago

this is based on #55 , so #55 needs to be merged first

Using the the SAM model, this applies more or less sophisticated filtering to get rid of masks which are not buttons. This is done in a few steps:

  1. Construct a tree based on the bounding boxes of the masks.
  2. In this tree search masks which could plausibly represent the remote (or a bit more than the remote) based on a few criteria:
    • a fixed point close to the lower clamp which always is on the remote
    • an assumed minimum size for any remote
    • an assumed maximum size for any remote (based on the space constraints we have anyway
    • the assumption that a remote has at least 3 buttons
  3. Out of the masks which possibly represent remotes, the mask with the highest stability is selected.
  4. All childs of that masks are put back in a flat list.
  5. Masks which are too big compared to the remote are considered not to be buttons and are ignored.