neuroinformatics-unit / movement

Python tools for analysing body movements across space and time
http://movement.neuroinformatics.dev
BSD 3-Clause "New" or "Revised" License
104 stars 8 forks source link

Add support for tracked segmentation masks? #301

Open sfmig opened 1 month ago

sfmig commented 1 month ago

With tools like SAM2 that track segmentation masks in videos, this may be a nice addition to our set of accepted input data.

Segmentation masks are usually represented as RLE (Run length encoding):

Maybe we can consider to link masks to bounding boxes too?

We may want to check Annolid - an annotation and tracking tool centred around instance segmentation

niksirbi commented 3 days ago

@gchindemi has also expressed strong interest in movement supporting this type of tracking data. Given the growing popularity of models like SAM, I think the appetite for this sort of thing will keep growing, which means we should probably implement this.