FPS should not be slow motion

DaWe35 commented 1 year ago

I'm submitting a ... [x] bug report [ ] feature request [ ] question about the decisions made in the repository [ ] question about how to use this project
Summary

I recorded the same canvas twice with 25 and 60 FPS (60 is the native FPS of the canvas)

With the 25 fps option, the recorder doesn't reduce the recording FPS, it only stretches the video out so it is like a slow motion video:

25 fps:

https://user-images.githubusercontent.com/10603822/225377667-77eca13b-8227-4c31-a37b-6e77e3740d7a.mp4

60 fps:

https://user-images.githubusercontent.com/10603822/225377378-45ff7735-a27d-4115-8c6f-af9fa8716feb.mp4

Other information (e.g. detailed explanation, stack traces, related issues, suggestions how to fix, links for us to have context, eg. StackOverflow, personal fork, etc.)
Running on Ubuntu 22.04, Google cloud instance with one K80 GPU and 8 cpu cores (so plenty of resources for this)

lherman-cs commented 9 months ago

I'm also experiencing a similar issue. After collecting some data, it appears that the inter-frame latency from pageVideoStreamCollector is unstable:

Note: avg here is weighted moving average with beta = 0.5

Inter-frame delay: 27ms (avg: 20.751578919743174ms)
Inter-frame delay: 16ms (avg: 18.375789459871587ms)
Inter-frame delay: 18ms (avg: 18.187894729935792ms)
Inter-frame delay: 19ms (avg: 18.593947364967896ms)
Inter-frame delay: 16ms (avg: 17.296973682483948ms)
Inter-frame delay: 25ms (avg: 21.148486841241976ms)
Inter-frame delay: 6ms (avg: 13.574243420620988ms)
Inter-frame delay: 34ms (avg: 23.787121710310494ms)
Inter-frame delay: 15ms (avg: 19.393560855155247ms)
Inter-frame delay: 18ms (avg: 18.69678042757762ms)
Inter-frame delay: 16ms (avg: 17.34839021378881ms)
Inter-frame delay: 17ms (avg: 17.174195106894405ms)
Inter-frame delay: 33ms (avg: 25.0870975534472ms)
Inter-frame delay: 35ms (avg: 30.0435487767236ms)
Inter-frame delay: 47ms (avg: 38.5217743883618ms)
Inter-frame delay: 38ms (avg: 38.260887194180896ms)
Inter-frame delay: 46ms (avg: 42.13044359709045ms)
Inter-frame delay: 51ms (avg: 46.565221798545224ms)
Inter-frame delay: 36ms (avg: 41.28261089927261ms)
Inter-frame delay: 42ms (avg: 41.641305449636306ms)
Inter-frame delay: 56ms (avg: 48.82065272481815ms)
Inter-frame delay: 34ms (avg: 41.41032636240908ms)
Inter-frame delay: 47ms (avg: 44.20516318120454ms)
Inter-frame delay: 52ms (avg: 48.10258159060227ms)
Inter-frame delay: 33ms (avg: 40.551290795301135ms)
Inter-frame delay: 45ms (avg: 42.775645397650564ms)
Inter-frame delay: 92ms (avg: 67.38782269882529ms)
Inter-frame delay: 38ms (avg: 52.693911349412645ms)
Inter-frame delay: 41ms (avg: 46.84695567470632ms)
Inter-frame delay: 52ms (avg: 49.42347783735316ms)
Inter-frame delay: 39ms (avg: 44.21173891867658ms)
Inter-frame delay: 96ms (avg: 70.1058694593383ms)
Inter-frame delay: 33ms (avg: 51.55293472966915ms)
Inter-frame delay: 42ms (avg: 46.77646736483457ms)
Inter-frame delay: 55ms (avg: 50.88823368241729ms)
Inter-frame delay: 34ms (avg: 42.44411684120864ms)
Inter-frame delay: 48ms (avg: 45.22205842060432ms)

I'm suspicious that either the collector or Chrome dropped some frames implicitly, but the writer still assumes 25fps. Thus, future frames end up being played back for previous dropped frames, causing the delay.

We can check if there's a way to get a more consistent source or interpolate the missing frame to match the output fps by duplicating frames to make up the dropped ones.

lherman-cs commented 9 months ago

I tested this again against a blank website, basically no animation at all. The result is surprising:

> node index.js https://www.blank.org/

Recorder start delay: 7ms
Going to https://www.blank.org/
Inter-frame delay: 207ms (avg: 103.5ms)
Inter-frame delay: 149ms (avg: 126.25ms)
<EOF>

I couldn't find the details about the API regarding the expected inter-frame delay from the Page.screencastFrame API. But, this test tells me that Chrome will only send changes in animation and it will be the best effort. So, in the case of animation-heavy websites, some frames can potentially be dropped.

I think the writer should be rewritten to assume unstable inter-frame delays. It's probably much cleaner to have its clock instead of relying on the upstream.

lherman-cs commented 9 months ago

If anybody is interested, I made a fix in my fork, https://github.com/lherman-cs/puppeteer-screen-recorder/commit/6e8b6dff22987770ec062b2ae43b196212bb13b4.

Note: it currently only handles dropped frames. So, expect some skipping frames or flashing.

prasanaworld / puppeteer-screen-recorder

FPS should not be slow motion #67