search

mbari-org / vars-localize

Tool for creating localizations within the VARS database.
MIT License
0 stars 0 forks source link

VARS localize doesn't pull localizations in made from VARS Annotation #5

Open kwalz opened 1 month ago

kwalz commented 1 month ago

We need vars localize to pull all localizations from VARS db regardless of where they were made. We suspect it is also not pulling in localizations generated from our machine learning proposals either.

lonnylundsten commented 1 month ago

It appears as if vars-localize is only showing users localizations that were created with vars-localize. Giovanna has been localizing marine debris images and unfortunately approximately 2000 duplicate localizations were added to VARS because she could not see that there was already a localization created in some other way.

We'll need to try to do the following:

  1. Add the ability to see all existing localizations in VARS within vars-localize.
  2. Remove the duplicate localizations that Giovanna created by using the IoU method to find and remove them.
kwalz commented 1 month ago

Kyra and I also used vars-localize on midwater concepts and may have created duplicate localizations in the same way.

kevinsbarnard commented 1 month ago

Hi @kwalz @lonnylundsten, VARS Localize only deals with framegrabs, so you will not see any bounding boxes made in VARS Annotation that are made on the video, since they are not tied to a framegrab. This is the reason we needed the dfg-tool, in order to capture framegrabs so that VARS Localize could be used. VARS Localize will indeed show localizations made in other apps as well, so long as they are tied to a framegrab (image reference).

We need vars localize to pull all localizations from VARS db regardless of where they were made.

Implementing this in VARS Localize would mean integrating Beholder to capture framegrabs on the fly from the video, similar to VARS GridView. This will take some effort since VARS Localize was not intended to do this; I'd estimate 1-2 weeks of time.

We suspect it is also not pulling in localizations generated from our machine learning proposals either.

This is correct, since again there is no framegrab for VARS Localize to use for these.

Giovanna has been localizing marine debris images and unfortunately approximately 2000 duplicate localizations were added to VARS because she could not see that there was already a localization created in some other way. ... Kyra and I also used vars-localize on midwater concepts and may have created duplicate localizations in the same way. ...

  1. Remove the duplicate localizations that Giovanna created by using the IoU method to find and remove them.

We can do this, but I'll need some more information about these annotation efforts so I can identify the relevant data. Tagging @hohonuuli so he's aware too, as this would involve a bulk deletion of data in VARS. Once I know what to look for, I'd propose:

  1. I do some analysis (e.g. the IoU approach) to identify potential sets of duplicate boxes
  2. We discuss what to do with them in the next VARS meeting

I'm going to create a separate issue for this in our vars-feedback issue tracker. As a reminder, this repository is the public/external version of VARS Localize -- the up-to-date internal VARS Localize lives at https://bitbucket.org/mbari/vars-localize.