search

rly / ndx-pose

NWB extension to store pose estimation data
BSD 3-Clause "New" or "Revised" License
13 stars 11 forks source link

Add multi-animal pose functionality #3

Closed talmo closed 3 months ago

talmo commented 2 years ago

Adds PoseGroupingSeries, AnimalIdentitySeries to enable storage of metadata for multi-animal pose tracking, such as part grouping and tracking scores.

bendichter commented 2 years ago

@talmo We were scoping this for 1-subject for now, but I suppose it doesn't hurt to think about how to extend this to multi-subject. I think we already go some way to handling this use-case.

In the example code, we show the PoseEstimation object holding only a single PoseEstimationSeries, but it is designed to store 1+ such objects, and can be used to group together joint trackings associated with a specific animal. Then we could put multiple PoseEstimation objects in the behavior module to handle multiple animals. So I don't think we need the PoseGroupingSeries but I may be missing something. I had originally thought of this as a hard assignment where we could have multiple Subject objects and hard assign a PoseEstimation to a subject, but it looks here like this should be a soft assignment over time that handles uncertainty in the assignment. Is that right?

AlexEMG commented 2 years ago

I thought we had discussed that for NWB each animal has its own file (e.g. to store neural data) and thus also only one PoseEstimationSeries. So, our plan was to export a DLC output into multiple PoseEstimationSeries.