schickling / chromeless

🖥 Chrome automation made simple. Runs locally or headless on AWS Lambda.
https://chromeless.netlify.com
MIT License
13.25k stars 574 forks source link

Add screencast functionality #168

Open timsuchanek opened 7 years ago

timsuchanek commented 7 years ago

For debugging purposes, it would be awesome to have .startScreencast(), that stops and gets saved automatically when an error is raised or could be stopped by invoking .stopScreencast().

CDP actually provides that feature. https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-startScreencast

Acceptance Criteria

shaeqahmed commented 7 years ago

A problem is that the method Page.startScreencast doesn't provide frames at a specific rate, it only triggers a Page.screencastFrame event when the screen has changed, so the when the "video" is put together it might be choppy. Im not sure how you would get around that though.

adieuadieu commented 7 years ago

There's a similar thread in https://github.com/adieuadieu/serverless-chrome/issues/17. It is possible to do this both locally and on Lambda. But as @shaeqahmed notes, it's not exactly straightforward. CDPs Page.startScreencast will trigger a Page.screencastFrame event only when the browser paints something new. In other words, you're provided only with the sort-of "key frames". This means that, rather than producing a frame at a specific frame-rate, Page.screencastFrame will only produce a frame (specifically, an PNG image) when the browser has rendered something new. In other words, a frame is only produced when there are visual changes in the viewport.

What we'll have to do to implement this is, once we start the screen cast, start capturing each frame and recording the time between each frame. The object returned by Page.screencastFrame includes timestamps which we can use. Once the screen cast ends, we'll have to stitch these frames together (with ffmpeg or some other encoder). The trick is that, for each frame we got from Page.screencastFrame, we have to repeat that frame in the rendered video for the duration between the frame and the next frame's timestamp, taking into account the video frame-rate we're rendering at. So far I haven't figured out how to do this via ffmpeg's CLI (other than actually producing an image for each frame—but that could get crazy real fast.) gifencoder has a useful delay feature, but I don't think we want to be able to only produce GIFs.. Also ideally we should stream the PNGs produced by Page.screencastFrame directly to the encoder as they're produced rather than writing them to disk first, then encoding later.

This feature is something I'd be interested to take on as I've already made some progress on it previously—both locally and in Lambda.

whatsdis commented 6 years ago

updates?