Debug problems with composite video generation

[anayapouget]

Until recently I was focusing on composite videos of Aeon 3 social 2 which have minimal issues since I believe these are the ones Orsi was testing her code with. However, I started noting some issues when I moved on to Aeon 4 social 02, and then with social 03 and 04 it gets worse... Here's a list of the problems I have noticed so far:

1. Aeon 4 social 03, social period 2024-06-19T14-00-00: there are some frames where no mouse is in frame
2. Aeon 3 social 04, presocial period 2024-08-20T11-00-00 + Aeon 4 social 02, presocial period 2024-01-31_11-00-00 and 2024-02-01T15-00-00: the generated composite videos are shorter than expected, frames must have been dropped
3. Aeon 4 social 04, presocial period 2024-08-16T17-00-00: all frames are sampled from the same quadrant camera

Possible explanations:

• Issues with Bonsai centroid tracking or DJ SLEAP full pose ID data i.e., some frames where it is missing, and/or some frames where it is inaccurate?
• Blind spots i.e., some spots in the arena are perhaps not covered by any of the quad cameras? I think this is unlikely though
• Innacurate homography, though the pixel mapping results plots look pretty decent so I would be surprised this is causing massive issues? Check if it's the case in particular for Aeon 4 social 03, social period 2024-06-19T14-00-00 because there is one camera angle where I am getting weird SLEAP labels (but not in individual sessions which is weird, so maybe the DJ SLEAP full pose ID data is to blame?)
• Bugs in Orsi's code that assigns quadrant camera frames to top camera frames, causing some frames to be dropped or wrong assignments

[anayapouget]

1. Cause of problem: There are discrepancies between top camera and quadrant camera timestamps causing the wrong quadrant camera frames to be pulled. Something went wrong at the start of the epoch causing a 110 frame delay between certain cameras. Solution: Just don't use that epoch!
2. Cause of problem: some epochs are missing quadrant camera data (i.e., the folder for a quadrant camera is missing). Solution: after running the new quadrant camera bonsai sleap workflow, we will have to run a second piece of code that completes frames missing from the composite videos with CameraTop pose data.
3. Cause of problem: the bonsai centroid tracking is stuck on the top of the nest and not actually tracking the mouse. I believe @NeuroThom made a new bonsai centroid tracking workflow that fixes this issue? Solution: either we run @NeuroThom 's fix on the data, or we wait until I manage to generate SLEAP position data for these chunks. I am currently struggling to do so though because I need to find a way to restrict the number of detected instances when running sleap predictions through Bonsai. @glopesdev do you know if that is possible to do?