Overlay mode

[BenjaminLindberg]

@JonnyBurger 👋🏻 It would be really good if there was a "OverlayMode" where the composition gets rendered over a video. This would mean you don't need to have a video element in the composition lowering the performance. It could be an argument when you render with lambda or with ssr, you pass the videos URL/path or something like that. If this would be possible it would be a great feature for sure! 😄

[JonnyBurger]

Good idea, I'm +1 on this!

[UmungoBungo]

Not sure I'm understanding - so this to improve the performance of the player component?

[JonnyBurger]

@UmungoBungo No, it is to make the render faster. A usecase that arises often (also in your app) is that you render something on top of a video, and instead of screenshotting a Video tag you would just render the overlay and then use FFMPEG to overlay it

[BenjaminLindberg]

Any news regarding this?😁

[JonnyBurger]

@B3nnjoe A bit reprioritized since one of the potential big customers has found a workaround and they don't necessarily need it anymore. If we would do it, we would implement it together with other features that require multiple FFMPEG passes, like #1276

[BenjaminLindberg]

@JonnyBurger sounds good! But one thing im wondering is the workaround. Would it be possible to get a rough explanation?

[JonnyBurger]

@B3nnjoe something similar can be achieved by rendering only a transparent overlay in ProRes format (https://www.remotion.dev/docs/transparent-videos/) and then use an FFMPEG command to combine both the original video and the overlay.

[JonnyBurger]

Coming in 2023

[BenjaminLindberg]

Would it be possible to invoke 2 lambda functions at once? One function that just renders the Remotion composition, and the other one is for the overlay. And they would get invoked at the same time so the overlaying function could get the time to load all the media from the s3 bucket, so when the Remotion composition is done rendering, the overlay function already have loaded everything and could basically just start overlaying directly.

Or would this just be unnecessary, or will the overlay and render be on the same lambda function?

Anyways here's a ffmpeg string that I'm using in my application to overlay a mp4 (pretty sure most if not all other video formats work) . ffmpeg -i background.mp4 -i prores-remotion-render.mov -filter_complex '[0:v][1:v]overlay[out]' -map '[out]' -map 0:a -y output.mp4 This works really good and I've had some errors with it, but I'm not thinking that this ffmpeg string causes those problems, but external factors like the VM. Still mostly works fine.

Might give you a bit of a head start at least, then starting from scratch 👍🏻

And also one question, are you planning on early-mid-late 2023?

[JonnyBurger]

@BenjaminLindberg

Kind of! For overlaying the two tracks, this will for sure happen in the master function. One function for rendering the Remotion part and the background ideally does not need to be preprocessed at all.

Thanks for the script, which is very helpful! I will indeed use it as a reference.

I don't set any targets or deadlines, but it is one of the higher priorities.

[Swarzox]

Hello,

Any news for this?