search

backprop64 / DAMM

A codebase for tracking laboratory mice in videos
9 stars 0 forks source link

Tracking at different fps #4

Closed matiasandina closed 4 days ago

matiasandina commented 6 months ago

We film long videos (24 h), which means DAMM would need to iterate over a really long time. Ideally we would like to quantify faster than acquisition time (hopefully a few hours for each 24 h video?). We have not been able to get DAMM to predict much faster than acquisition rate (we are ~20fps right now on an NVIDIA GeForce RTX 3080 Ti). We would really welcome input in order to improve this.

What we tried to speed things up was to reduce the number of frames. We did a tests at 15, 10, and 5 fps.

This is 15 fps

https://github.com/backprop64/DAMM/assets/7494967/ae7ffbe8-f68a-4ee9-9549-65096a266fdc

This is 10 fps

https://github.com/backprop64/DAMM/assets/7494967/8fc27534-f2ea-45f9-a99c-413fca8bb9d5

This is 5 fps

https://github.com/backprop64/DAMM/assets/7494967/889732c5-4d6a-4c11-b5c0-46d14a5bbd78

Predictably, the time it took to predict these files scaled with the number of frames (eyeballing it, it seems quite linear so there isn't much of optimization for large files? We might be wrong here). The not so nice surprise was that the tracking at 5 fps is much worse. Is this due to post-processing or due to the nature of the network expecting certain amount of pixel to pixel movement?

backprop64 commented 4 days ago

This is a good point, a of people have really long recordings at high FPSs, I will add some functionality that lets the user specify the tracking FPS when using DAMM so they dont need to adjust their video.

Unfortuanatly the speed of the video analysis will depend on your GPU, model size, and video FPS. Ill add some additional tips for speed/compute efficency under the usage examples.