moved body points

[ZelikowskyLab]

Describe the bug body points are being moved to the top left corner of the video (same location in multiple videos). Videos were tracked in SLEAP, and the body points are never in this location. This is still present, even with outlier correction.

To Reproduce Steps to reproduce the behavior:

1. I updated SimBA with: pip install simba-uw-tf-dev --upgrade
2. I set up a project, imported all videos and .h5 files from sleap (interpolation: linear; smoothing: none), set distances etc
3. Tried both with and without outlier correction (location and movement set to 1.5, nose to tail base on each animal)
4. analyzed directing to animals & visualized

Expected behavior I expected that the video would show body points on each mice (like when tracked in sleap) & when they are directed toward the other mouse. This was mostly the case except for long periods of time where a body point is placed in the top left corner of the video (same spot on multiple different videos - maybe the numbers are just being set to 0?) and counted in the direction data.

Desktop (please complete the following information):

• OS: windows 11
• Python Version [e.g. 3.6.0]
• Are you using anaconda? yes

[sronilsson]

Hi @ZelikowskyLab !

TLDR: If you run Body-part: Nearest interpolation on the data, does it look better?

The 0,0 coordinate (top left corner) data is where SimBA places body-parts when the pose-estimation package doesn't provide any location for a given frame.

There are two ways to get rid of these missing data points: (i) train a better pose estimation model which provides good body parts predictions for all or nearly all frames or (ii) interpolate the missing data.

There are several interpolation methods in SimBA. They can be specified to be performed only when entire animals are missing, or when individual body-parts are missing, and can be quadratic, linear or nearest. I see that you use linear interpolation but I don't know if it is animal or body-part based. If using animal based interpolation, missing data will still be retained unless all of the bodyparts of an animal is missing. Linear interpolation is likely to be a little bit off if data is missing for extended periods of time relative to the other methods. That said, all interpolation methods will be inaccurate if data is missing for many consecutive seconds. Best method, if possible, is always to train a pose-estimation model that provides better data without missing frames so interpolation is rarely used.

[ZelikowskyLab]

That worked. Thanks!

[sronilsson]

Thanks for letting me know @ZelikowskyLab !