Capture movies

[blairlyons]

• [x] https://github.com/simularium/simularium-viewer/issues/240
• [x] https://github.com/simularium/simularium-website/issues/327

Use Case

It would be nice if users can capture and export movies while interacting with the Viewer.

Acceptance Criteria

Users can save high quality movies of interactive sessions in the Viewer.

Details

The UI will need a way to start and stop recording and to save the results to disk.

This needs to be implemented to minimize choppiness from dropped frames while rendering and capturing large scenes.

[toloudis]

a quick look at capturing from canvas shows me this:
https://developer.mozilla.org/en-US/docs/Web/API/HTMLCanvasElement/captureStream

It’s a new “experimental” feature but seems supported in nearly all major browsers. https://caniuse.com/mediacapture-fromelement

This can be tied to other code to potentially save out video, although transcoding the video to expected compression formats might be big can of worms. example usage: https://stackoverflow.com/questions/62863547/save-canvas-data-as-mp4-javascript

[toloudis]

We used to do this by capturing the canvas contents to an image and forcing the download of the image, once per frame.. That captureStream thing makes it potentially easier and less invasive to implement.

[schoinh]

I also found this library, ccapture.js... They have an awesome live demo. I'm not positive that our code operates how this library needs it to... but the demo is cool at least since it has UI elements similar to what we talked about (timestamp, start/stop recording, etc.). The big downside to using this, it seems like, is that while it's recording, it plays far far slower than normal.

[toloudis]

looks nice, although not actively maintained! I wonder if it's using the same captureStream api

[schoinh]

It looks like it does, yeah. https://github.com/spite/ccapture.js/blob/f6baa12526512d1385f23abf56401fb4f0fd2fe6/src/CCapture.js#L425

At this point, I feel like the next step would be to try prototyping using captureStream -- can borrow/learn from ccapture.js too?

[blairlyons]

blocked by https://github.com/simularium/simularium-viewer/issues/240

[schoinh]

Since embedding videos in PowerPoint slides is a main use case, we should probably aim to offer .mp4 files encoded with H.264 as the output format (at least to start with): https://support.microsoft.com/en-us/office/video-and-audio-file-formats-supported-in-powerpoint-d8b12450-26db-4c7b-a5c1-593d3418fb59

[schoinh]

@lynwilhelm I found an example of another app that lets you export canvas frames to an mp4 you can download, maybe we can borrow some UX ideas, demo here: https://www.youtube.com/watch?v=PAbwPaGQIyI&t=50s

[lynwilhelm]

Ticket was moved to "done" column, but I moved it back to "In review" after Na Hyung showed prototype. There were a few issues that will effect the design. When designs are updated, I will move back to "done". I will also update the acceptance criteria for the dev ticket #327

UX Items to Solve:

• Need to limit recording to a certain max time (or file size) and let user know what that is (maybe there is a warning when approaching max during recording session and recording stop automatically at that point)
• Should we allow video bits/second choice up front?
• Progress bar for after commit

[toloudis]

https://github.com/allen-cell-animated/nucmorph-colorizer/issues/33 has a solution that works without any shared memory requirements or workers. The solution is only Chrome-compatible (and must be used in a secure https session only) but it was straightforward to implement. It's using a web API which could be implemented in the future in other browsers.