Choose object-detector model.

[samcunliffe]

• Select a few models for ID tracking.
  • keypoint tracking (DLC, SLEAP)
  • object detection (TensorFlow... old? transformer is probably better)
  • cell-tracking uses optic flow (is this useful?)
• Minimum 3, maximum 5.
• Produce a spreadsheet or table or something.

---

Related considerations

The number of simultaneously tracked animals.

Is there a limit imposed by the model?

---

2023-05-03

@sfmig's thoughts

Zoom is more promising. Unzoomed might be hard.

• Supervised expect to perform better.
• Unsupervised tools exist: btrack, and DINOv2.
• Compare top-down and bottom-up approaches in scenarios where there is partial occlusion with animals.
• Pose-estimation seems not that useful

@harvey

Far away video is "going to be super hard":

• Downsampling.
• Camouflage.
• Patch-based approach might be better?
• Particle filters?

Harvey's friend:

• benchmark for multiobject tracking
• he used ByteTrack: this handles tracking but needs a separate detector (YOLO?).
  • use ReID algorithm

Tracking reappearance.

Keeping the identity of the animal after complete occlusion.

• Second bullet point is really hard.
• There are some ways to do this if we were to have ground truth data.
  • Paint the crabs?

[sfmig]

Current thoughts on our options for detecting crabs:

• context-RCNN seems a good temporal-approach option -- I contacted the main developer, but also the framework where it is hosted is deprecated as of 7 months ago :/
• cellfinder approach (aka combining classic computer vision methods with a binary classifier to determine if a crop contains a crab or not)
• "off-the-shelf" multiobject video detector: from tensorflow or meta?

[samcunliffe]

Stacked inputs. So recommends Sara.