The mask FPS returned from API is not accurate. The number seems to be rounded down. When we assume the FPS is 29 while it's actually 30, the delay for the mask is exaggerated when the video plays on.
mask frames returned from the API don't match perfectly with the video frames, so if we just calculate the frame index, there's still a noticeable 1-2 seconds mismatch.
The fix is to store the mask frames ordered by timestamp, and when we need to show the mask, we use binary search to find the latest mask frame.
Also, a minor improvement is to start using the mask before the processing is finished, which decreases the loading time when we play the video from the beginning.
yichengchen
commented
1 year ago
This PR is to fix https://github.com/yichengchen/ATV-Bilibili-demo/issues/45
There are two issues:
The fix is to store the mask frames ordered by timestamp, and when we need to show the mask, we use binary search to find the latest mask frame. Also, a minor improvement is to start using the mask before the processing is finished, which decreases the loading time when we play the video from the beginning.