search

YttriLab / A-SOID

An active learning platform for expert-guided, data efficient discovery of behavior.
Other
55 stars 7 forks source link

Question: Number of classes to manually label for A-SOID; including landmark keypoints #80

Closed eacrummy closed 7 months ago

eacrummy commented 8 months ago

We are trying to use SLEAP with A-SOID to automate scoring of freezing behaviors and time on spent on a platform in mice during an operant platform mediated avoidance task. We’ve built a model in SLEAP, but wanted to get an idea of the best way to manually score the videos for training A-SOID. A few questions we had are:

  1. Is it best to manually score additional behaviors to help with A-SOID classification (i.e. labeling freezing, rearing, and/or grooming)?
  2. We want to make sure that we aren’t overlapping scores for freezing and staying on the platform. Will it be possible to differentiate these behaviors using A-SOID if we have coordinates predicted for the platform itself?

We appreciate any feedback. Thank you!

JensBlack commented 7 months ago
  1. I don't think that additional classes are required to increase classification performance. Be aware that all unlabeled bouts will be put into "other" by default, so you will end up with 1 big class ("other") and 2 small ones. 2a. During annotation you will need to label discrete and exclusive events. This means, that you will have to pick one behavior that is present. A-SOiD is not able to identify multiple behaviors happening at the same time without a workaround. 2b. If your behavior can be identified using coordinates directly it might be a beneficial pre/postprocessing step to directly use the pose information, yes. However, you cannot influence the extracted features in a direct manner, given the architecture of our app, which will result in any absolute coordinates in the image to be lost.

An alternative solution is to include keypoints that identify important landmarks in the arena, so that the relative position to the platform/object is included in the feature extraction.

Let me know if this helps. Feel free to close this issue.