search

cosanlab / py-feat

Facial Expression Analysis Toolbox
https://py-feat.org/
Other
262 stars 71 forks source link

Getting slow in processing tiem #182

Open keijiota opened 1 year ago

keijiota commented 1 year ago

Hello, Thanks for developing this amazing toolbox. I have started using this with the aim of detecting facial motion units in a single person face of video recordings. It looks like the processing time per each frame is getting slow as the time in the video goes to the end. Do you see why and how to avoid this? I use a video with about 3minutes long. I use a default detector, detector = Detector(). Thanks! Screenshot 2023-09-19 223238

ljchang commented 1 year ago

hi @keijiota , thanks for posting an issue. This is a known issue when detecting features with video. We fixed an issue where detection would crash if your computer ran out of RAM from loading a large video file. As an interim solution, we now only load a subset of frames, which keeps the RAM costs low. However, an unfortunate consequence of this decision is that the detection time linearly increases with the number of frames run. Also, we have found that the overall detection runs quite a bit slower especially on GPUs.

Here are a couple of kludgy temporary workarounds:

  1. Cut your videos smaller and then run detection in parallel.
  2. Skip frames to run less frames

We are aware that pytorch is working on their own solution to loading video frames and will switch over to that once it is completely implemented, though we don't know when that will be finalized yet.

keijiota commented 1 year ago

Hello @ljchang, many thanks for the reply! I understand the issue. I think a serial computing on a set of smaller videos will help for me. Is there any model of detector that you recommend for facial motion units analysis? There may be a trade-off between great precision and slower speed though.