Rendering and recording mirrored video stream from camera.

[stepancar]

Description Added a demo which demonstrates rendering and recording mirrored video stream from camera.

Purpose

Mirroring video is a common case in video processing. https://github.com/webrtc/samples/issues/1626 This demo shows advantage of processing video frame-by-frame in a worker https://stepancar.github.io/webrtc-samples/src/content/insertable-streams/video-recording/

[hthetiot]

@stepancar I was wondering if you could add the work around to make MediaRecorder recorded video seekable part the of sample. Doing so would inform developers that implement recording of the issue and potentially serve as reference to push Chromium team reconsider the Won't Fix state of the bug (I can always dream) if all other non chromium browsers proves not having the issue.

I would understand if you refuse, at least mentioning that recording file may not be seekable from this sample when using Chromium based browser can in my point of view go in the right direction and help the cause.

Chrome related WontFix bug:

• https://bugs.chromium.org/p/chromium/issues/detail?id=642012

Know working workaround:

• https://github.com/buynao/webm-duration-fix

[stepancar]

@hthetiot , I would be happy do that too, but when I get any comments from maintainers to make sure this sample makes sense

[stepancar]

@fippo , could you review it? Thank you

[stepancar]

Hello @fippo! May I ask you to review it/validate idea?

[stepancar]

@hthetiot , do you know how to get a feedback from maintainer in this project?

[fippo]

@stepancar apologies for the radio silence. The samples repository is effectively in maintenance mode and hasn't had a dedicated maintainer for a while.

The use-case for the sample is interesting, one normally does not think much about this as most mirroring is done with CSS but that does not apply to MediaRecorder. But mixing mirroring with MediaRecorder and then providing two modes makes this way more complicated than it would need to be. It would probably fit better as a processor in the video-processing sample but that one is not using a worker and the code makes me cry.

If you still see the slowdown filing a specific crbug for it would be better.

[stepancar]

@fippo , thank you for your answer!

I know that usually people mirror video with css. This works well when we need to implement video calls app. But here I'm trying to demonstrate a specific scenario - recording mirrored video. Which is a use case for video editors.

This demo helps to better understand limitations of MediaRecorder API / canvas.captureStream and demonstrates how https://github.com/w3c/mediacapture-transform can help here.

Yes, know about https://webrtc.github.io/samples/src/content/insertable-streams/video-processing/ I thought to add it there, but there is already and existing canvas 2d example. I would say it's a different kind of demo https://webrtc.github.io/samples/src/content/insertable-streams/video-processing/ - is saying if you want to transform your stream you can do different things like... . The problem with that - it does not demonstrate why should I do it. all transformations from that demo could be implemented without https://github.com/w3c/mediacapture-transform My demo demonstrates problem (frame drops in recorded video) and shows the solution.

I didn't get your point about crbug, could you elaborate on that? Media recorder behaviour follows specification. Thank you!

[fippo]

I get the use-case and it is great! But adding it to the simple recording demo is out of question because iirc Firefox does not support breakoutbox yet (and don't even try to get started with worker vs non-worker) and it brings in a ton of complexity.

Now if you actually look at the problem that needs solving it would be way more efficient just to use libyuv's I420Mirror method in-place rather than doing multiple thread hops etc. Works nicely but requires a JS API change to add a flip option.

Over to @guidou

[guidou]

@stepancar Can you file a spec bug at https://github.com/w3c/mediacapture-main/issues/ or https://github.com/w3c/mediacapture-record/issues explaining the use case and, if you have a proposal, a sketch of it and its rationale?

[stepancar]

@fippo , I totally agree with you that it would be nice to have simple flip option in media capture api, or, even in api where we get the stream https://developer.mozilla.org/en-US/docs/Web/API/MediaDevices/getUserMedia It would be more efficient and convenient for this specific use case

but the problem is that those api are not about transforming streams and introducing such option there is abstraction/responsibility leaking. So, not sure maintainers will want to support it (I will ask in another issue) The same time https://github.com/w3c/mediacapture-transform is designed for that, right? This is why I'm adding this demo here

Ideally, it should be another, non javascript api for making such transformations, without requiring to write js code and thinking about workers, etc. Something like

const mediaStream = new MediaStreamTransform(originalMediaStram, {
     transform: 'scaleY(-1)', // https://developer.mozilla.org/en-US/docs/Web/CSS/transform
     filter: 'blur(5px)'  // https://developer.mozilla.org/en-US/docs/Web/CSS/filter
})

Talking about firefox is not supported. I'm not sure how this can affect this sample. this sample is for https://github.com/w3c/mediacapture-transform section. If it's not supported by vendor - it's expected that it would not work. It means, existing demos which use breakoutbox also don't work in FF