Programatic arena definition, video use, number of videos for training

[neuraldino]

Hi, I have few questions:

• Is there a way to define the arena programmatically without the GUI? I am using large quantities of videos with the same orientation and it is a very lengthy process to setup polygon arenas on all videos. The alternative would be not to have any arena, but "none" option does not seem to work. This would also allow to run the pipeline without GUI in a console.
• Are videos used in training or just keypoints? I have videos from multiple cameras and so there is not 1:1 overlap between keypoints and videos (keypoints are all in the same top-down view, but come from multiple cameras to limit occlusion).
• How many videos should be used for (unsupervised) training? Is there any upper limit? Thanks for help.

[lucasmiranda42]

Dear Dino,

apologies for the delay! You caught me on holidays. There is no easy way to programmatically define the arena without the GUI at the moment, although we're working on it, as it seems that arena detection improvements are a widely requested feature.

Even though you still can't define the arena programmatically, and minimal GUI interaction is still required, we added automatic detection of polygonal arenas to the development branch (hosted in a separate repository). You can find the details in this issue.

Answering your second question, DeepOF only uses key points when training unsupervised models. Arena detection is only used for some rule-based assessments (such as climbing or interacting with the walls). These unfortunately only support top-down (hence single-camera) videos at the moment, but you could use these videos only for DeepOF, and provide the multi-camera tracks which limit occlusion anyway. Let me know if you need help in this regard!

Regarding the number of videos, the upper limit would be determined by your hardware. I would strongly suggest you to try to use them all and play with the batch size if you're having issues with GPU memory.

Best, and hope you find the package useful! I'll tag the issue as an enhancement, related to the arena detection improvements mentioned above, and let you know when updates are up.

Cheers, Lucas

[lucasmiranda42]

Dear @neuraldino,

We just released a new version of the package with several new features, including easier (either fully automatic or manual) arena detection. Importantly, if accurate arenas are not important or all videos should use the same parameters, you can now copy the last manually defined arena to all remaining videos.

Best, and hope this helps! Lucas