search

denisecailab / ezTrack

Free, platform independent, behavior tracking software.
GNU General Public License v3.0
118 stars 41 forks source link

Frames calculated by ezTrack #27

Closed h1408 closed 2 years ago

h1408 commented 3 years ago

Hello, I've been trying to run behavioral videos on ezTrack and have been using the frame rate to calculate the start/end frames. One thing I've noticed is that when I set the end frame to None and then run step 3, the total frames calculated is much higher than what I'm seeing it is by getting the total frames of the video in Matlab. Also the 'nominal fps' is different from the frame rate I see on my computer of the video. What would this difference mean in terms of analyzing the video? If I put an end frame of 15000 but ezTrack is calculated the total frames to be 600,000+ would this mean it's not actually analyzing the full video? I would appreciate any insight you have on this, thank you!

ZachPenn commented 3 years ago

Could you please provide more information on the filetype being used? The nominal fps is taken directly from the video codec metadata so I'm not sure why there would be a discrepancy. As to the number of frames, some video file types seem to have a large number of irretrievable or blank frames, and ezTrack simply reports the total, including these 'blank' frames. If the blank frames are at the end of the video, after all the frames you actually want to process, there's no issue. However, if they are intermixed, you will likely encounter issues.

I might try converting to another filetype.

h1408 commented 3 years ago

Hello, this is a .wmv filetype. When I open it on a windows pc, the frame rate shows as 25 frames/second. In addition, I have run the video file through MATLAB to assess the properties of the file and it also shows it as 25 frames/second, along with a total number of frames as 16000 as in the attached photo. However, in ezTrack the frames per second is shown as 1000 and the total number of frames is over 600000 (photo of this is also attached). I thought there weren't any blank frames since I was able to find the total number of frames of the video which is around 16000. Would converting potentially lead to loss of information? Thanks so much! Screenshot (4)

Screenshot (3)

Screen Shot 2021-05-10 at 4 30 23 PM

ZachPenn commented 3 years ago

Looking more into the frame issue, I don't think it's so much that you are missing frames, but that you have incomplete frames. This is a typical of many video compression algorithms to support support smaller file sizes (see https://en.wikipedia.org/wiki/Video_compression_picture_types for a description of 'p frames'). Unfortunately, ezTrack does not currently possess a good way to handle videos with these incomplete, 'p frames', and simply skips them. While all full frames can be processed, frame indexing gets really weird (because key frames 1,2,3,4,5 might have indices 1,11,21,31,41, or something like this).

It may be possible for me to implement something to handle this, but I don't have any videos like yours so I can't do any testing. If you would like to post a video here or reach out to me via email, please do. I would like to start dealing with this.

With respect to the frame rate issue, I really am not sure why ezTrack isn't read the metadata properly. Again, if you'd be willing to post an actual video, it'd be great to check it out.

h1408 commented 3 years ago

Hi Zach,

Thank you so much for your reply and help with this! I can definitely share some videos with you via email! Where should I send them to?

From: Zach Pennington @.> Sent: Tuesday, May 25, 2021 10:36 AM To: denisecailab/ezTrack @.> Cc: Haniyyah Sardar @.>; Author @.> Subject: Re: [denisecailab/ezTrack] Frames calculated by ezTrack (#27)

Looking more into the frame issue, I don't think it's so much that you are missing frames, but that you have incomplete frames. This is a typical of many video compression algorithms to support support smaller file sizes (see https://en.wikipedia.org/wiki/Video_compression_picture_types for a description of 'p frames'). Unfortunately, ezTrack does not currently possess a good way to handle videos with these incomplete, 'p frames', and simply skips them. While all full frames can be processed, frame indexing gets really weird (because key frames 1,2,3,4,5 might have indices 1,11,21,31,41, or something like this).

It may be possible for me to implement something to handle this, but I don't have any videos like yours so I can't do any testing. If you would like to post a video here or reach out to me via email, please do. I would like to start dealing with this.

With respect to the frame rate issue, I really am not sure why ezTrack isn't read the metadata properly. Again, if you'd be willing to post an actual video, it'd be great to check it out.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/denisecailab/ezTrack/issues/27#issuecomment-848076400, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ATFFDAI5D3SQE3OD6JMRT4LTPPNZDANCNFSM43M6WDOQ.

ZachPenn commented 2 years ago

I have implemented fixes to help test for these issues (commit 0fec6e4), and have updated instructions to specify that ezTrack should not be used with temporal compression algorithms. I am closing this issue for now.