Closed Calinou closed 2 years ago
CPU-efficient compression algorithms are often patent-encumbered, unless you use something really old. By "CPU-efficient", I refer to algorithms that use little enough CPU to be usable in real-time manner, with a very low risk of frame dropping — VP9 and AV1 are too demanding for this.
Still, Godot could have ability to encode webm
: https://github.com/godotengine/godot/blob/b3c2281706778cf5311a68fe0071aa2740d01131/thirdparty/libvpx/vpx/vpx_encoder.h.
But when we talk about use cases like troubleshooting, for instance, GitHub does not support drag-n-dropping webm
into issues (only mp4
and mov
are supported as of now).
Some Goost users have asked me about ability to record videos as well for similar purposes. Due to complexity and patent-encumbered algorithms, all I could do is to implemented gif
saver: goostengine/goost#104. Users look satisfied now, the problem is that gif
is such a legacy format, and if we were to choose between apng
and webm/mp4
nowadays, that would be the latter, obviously.
Downsides: Does not allow for player input, and audio cannot be recorded. If you want to show gameplay sequences this way, you need to implement a demo/replay system in your game [emphasis mine], which is a lot of work if you only plan to use it for a trailer. When a game trailer relies on non-realtime rendering, audio is typically added in post-production. (There are ways around this, but it's not easy to implement.)
To decrease binary size, video recording functionality would likely be included in debug builds (editor and export templates), but not in release export templates.
Yeah, that's another use case that I see, which is being able to record game moments via replays. If Godot recognizes this use case as common, we shouldn't restrict video recording functionality to editor builds. If not, make it an optional component to be included in export templates (when you build Godot from source yourself).
This is a great idea.
@Calinou : For those that have these needs as of now, could you outline how non real-time simulation and ongoing frame output could be accomplished as a current workaround?
It's the second alternative scenario you mention.
For those that have these needs as of now, could you outline how non real-time simulation and ongoing frame output could be accomplished as a current workaround?
I have an example project here: https://github.com/Calinou/godot-video-rendering-demo
It's quite slow to save images due to the lack of a thread pool to save the PNG image sequence. This can be implemented in GDScript, but it'll require some fiddling to do correctly. (Saving frames out of order isn't a problem, as long as all frames are saved before the project quits.)
@Calinou thanks a lot for the example (should maybe be part of the engine?). I do not need it to be fast, just to be accurate (60 fps, never skipping a frame).
In short, I am working on an animation. Is there a similar way to also export the sound (then I can put together the two with a bash script using ffmpeg)? In the original post, in the non-realtime/PNG section, you mention that "there are ways around it, but not easy to implement". Could you hint at what is currently possible for sound in non-realtime?
Could you hint at what is currently possible for sound in non-realtime?
For non-realtime sound simulation to work, Godot would need to have a way to save its own audio output to a WAV file. I presume Godot would then fill in the PCM data as the AudioServer fills its buffer. It's quite a lot of work to do correctly and requires many core changes. Unfortunately, there's no way to work around this with a script.
Personally, I'd like to see a real-time encoder. While not every game could benefit from it, the resolution for old-school pixel art is around 256x240 pixels which is more feasible and would be pretty useful for everything from Dev snapshots, to trailers, to replays, speedruns, etc.
This is a lot of work for something that's always going to have better results (from a technical perspective) when done externally. While it can be a lot of work for one to set up external capture tools, it doesn't have to be much work for the end-users to set them up.
You'd be better off just providing a tutorial / explanation page or even an officially-supported self-created bundle. I don't have it on this computer, but I spent about half a day setting up a Git Bash script that ran FFmpeg to capture the Godot window losslessly at first then re-encode using x264 a very-high-quality lossy version. All using open-source tools and essentially needing no modifications outside of a double-click to start recording.
Easier to maintain, doesn't bulk up the engine, higher quality output (easier to update the toolchain), still fully open-source, and can be as simple as some double-clicks.
Another option, which could maximize speed, might be to save the video while recording in an invented format that's basically just a dump of the raw data from Godot, then process it to an actual video format, compressed or not, after the recording stops.
Another option, which could maximize speed, might be to save the video while recording in an invented format that's basically just a dump of the raw data from Godot, then process it to an actual video format, compressed or not, after the recording stops.
I don't think saving the AVI file is actually a bottleneck. It's a simple format to write and no compression scheme is involved.
Okay, that makes sense. I am easily intimidated by large amounts of code, like Red Eclipse's AVI recorder :)
I just want to echo everything @Gastronok said about it being a lot of work for something that OBS can already do.
I also would like to veto AVI as the container format for this (not that I have veto power! I just feel strongly haha). It is not an open format. I'd push for oggv/theora/vorbis, mostly because those are all fast and open. We could also do webm/vp8/opus but that's a heavy lift for real-time. As another option, we could require that ffmpeg be installed and then just call into it to do these things. This is a niche feature, I'm okay requiring extra steps to get it working if it means keeping the editor small.
As another option, we could require that ffmpeg be installed and then just call into it to do these things. This is a niche feature, I'm okay requiring extra steps to get it working if it means keeping the editor small.
In my experience, FFmpeg was never great at recording video. It's hard to get something smooth with few or no dropped frames, especially at high resolutions and framerates.
FFmpeg remains a good tool for encoding and post-processing, but it's a step that is best done after recording if you want to maximize performance during recording.
Is that an ffmpeg problem or a codec problem? I doubt ffmpeg is the bottleneck. ffmpeg is just a wrapper around a container format and a codec.
I've found ffmpeg -f gdigrab -framerate 60 -i title="Window Title (DEBUG)" -c:v libx264 -crf 0 -preset ultrafast capture.mp4
(Windows) to be sufficient for capturing smooth, nearly-lossless masters that I can further edit via other programs. Just make sure it's writing to a drive with good throughput rates since the file will be somewhat large.
I've found
ffmpeg -f gdigrab -framerate 60 -i title="Window Title (DEBUG)" -c:v libx264 -crf 0 -preset ultrafast capture.mp4
(Windows) to be sufficient for capturing smooth, nearly-lossless masters that I can further edit via other programs. Just make sure it's writing to a drive with good throughput rates since the file will be somewhat large.
Thank you for this! Great temporary solution. Not the same as having a native game engine method to use in game for replays, but great nonetheless. I wonder if we can make it into a plugin, since ffmpeg is OS. This works on Linux as well, but of course instead of
-f gdigrab
You would need
-f x11grab
And windowid (discoverable with xwininfo)
Is anyone working on this? My GSoC proposal was rejected for some reason but I'd still like to try working on it.
@keptsecret Yea, it would be cool if new updates come on this recording proposal.
I'm opposed to this being in core. I'm one person, but that's my opinion. There's no reason it couldn't be a plugin as far as I can tell, there are lots of workarounds, and it would be very rarely used.
Is anyone working on this? My GSoC proposal was rejected for some reason but I'd still like to try working on it.
You had a good proposal, and so did many others. Unfortunately, we have a limited number of spaces for proposals in each GSoC session, and had to be very selective as a result. I hope this doesn't discourage you from participating in the project and/or future GSoCs :slightly_smiling_face:
This feature's implementation can still be tackled as an independent effort.
I should have mentioned that I'm not commenting on the proposal's quality or anything like that. Just that I think it should be hooked up via a plugin instead. I'm sorry, I re-read what I wrote and it sounds super rude :(
I should have mentioned that I'm not commenting on the proposal's quality or anything like that. Just that I think it should be hooked up via a plugin instead. I'm sorry, I re-read what I wrote and it sounds super rude :(
No worries :slightly_smiling_face: I didn't interpret it as rude.
Is anyone working on this? My GSoC proposal was rejected for some reason but I'd still like to try working on it.
You can always try next year! Even my proposal didn't make it in first attempt. But that didn't stop me. I started making more contributions, fixing bugs, adding features! Meanwhile, made my proposal better for the next year! And guess what, it got selected! All the best! Let's make Godot da best! ❤️
Note: I discussed this with reduz a few months ago, so the feature is probably good to implement. We may want to discuss the specifics further, still.
Describe the project you are working on
The Godot editor :slightly_smiling_face:
Describe the problem or limitation you are having in your project
If you've ever watched indie game trailers, you've probably noticed at some point that stuttering and dropped frames are quite common, especially in lower-budget productions. Also, when troubleshooting specific issues such as physics, audio or rendering bugs, having pixel-perfect video and audio recording can be valuable.
There are several ways to record gameplay videos. None of them are perfect, but some of them are more suited to trailer production and troubleshooting than others:
Using a screen/window/game recording tool (OBS, AMD ReLive, NVIDIA ShadowPlay, …)
Upsides: Easy to acquire, well-known solution.
Downsides: It can take some time to configure the software to avoid frame drops. Also, while hardware-based encoding solutions such as NVENC free up CPU resources, they often result in lower visual quality. Hardware-based encoding is also dependent on your GPU model. CPU-based encoding can be very expensive at high resolutions and framerates. Also, however small it may be, the quality loss may be undesirable for troubleshooting purposes (e.g. pixel-perfect comparisons).
Using non-realtime simulation and a PNG frame sequence output
Upsides: Can achieve very high visual quality, since frame time budget essentially becomes a non-issue. Your trailer can take as long to render as you want to :slightly_smiling_face:
Downsides: Does not allow for player input, and audio cannot be recorded. If you want to show gameplay sequences this way, you need to implement a demo/replay system in your game, which is a lot of work if you only plan to use it for a trailer. When a game trailer relies on non-realtime rendering, audio is typically added in post-production. (There are ways around this, but it's not easy to implement.)
Using a hardware capture card
Upsides: Works with any application or game. A good capture card will pretty much never drop frames, even if your internal storage is too slow (since capture cards generally have their own storage medium).
Downsides: Not only capture cards are expensive for an indie developer, but they also have many limitations, especially if you want to use a high refresh rate monitor for your own purposes. Linux support for hardware capture cards can also be spotty, especially for high-end options that support 4K @ 60 FPS capture.
Using built-in uncompressed video recording (what's proposed here)
Upsides: Uses very little CPU resources and no GPU resources.
Downsides: Requires lots storage space with fast sustained write speeds. A SATA SSD is required to save 1080p60 or 1440p30 video, and a M.2 SSD is required to save anything higher than that. Nonetheless, it's possible to compress videos after the fact to perceptually lossless formats to save a lot of space with very little quality loss (such as H.264 with CRF set to 15).
Describe the feature / enhancement and how it helps to overcome the problem or limitation
The de facto SSD requirement for uncompressed video recording used to be a problem, but now that relatively-large SSD availability is widespread, it's not as problematic.
Recording long videos will still require you to split the video into smaller segments (which you can then compress using an external tool), but most game trailers don't feature long, uncut gameplay segments anyway.
Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
From an user point of view, video/audio recording could be configured in the Project Settings to toggle various features such as audio recording (to record video only), FPS, etc. Ideally, it should be possible to render the video at a lower size compared to the actual window size to allow for supersampling while still having low storage requirements.
Recording would be activated at run-time by pressing a built-in input action which could have a default value such as Ctrl + Shift + R. Pressing the same key combination again would stop recording. When recording is started or stopped, a message should be printed to standard output. The "end" recording message should specify the duration of the recording that was saved, as well as its save location.
To decrease binary size, video recording functionality would likely be included in debug builds (editor and export templates), but not in release export templates.
As for the technical implementation, I don't know much about getting the audio and window's contents quickly and writing an AVI file with the resulting audio/video. Nonetheless, the (zlib-licensed) implementation of an AVI recorder in Red Eclipse could be used as an inspiration: https://github.com/redeclipse/base/blob/master@%7B2021-08-03T16:33:29Z%7D/src/engine/movie.cpp
Question: Why not implement compression support as well (even lossless)?
CPU-efficient compression algorithms are often patent-encumbered, unless you use something really old. By "CPU-efficient", I refer to algorithms that use little enough CPU to be usable in real-time manner, with a very low risk of frame dropping — VP9 and AV1 are too demanding for this. Having a high-end CPU should not be a requirement to make a good game trailer. Also, upgrading a SSD is often easier and cheaper than upgrading a CPU (which may require replacing the entire machine in the case of laptops).
Question: Can we further optimize the video recording for low-end machines?
By running the project in slow motion (while also slowing down the audio playback), the storage write speed requirement could be lowered significantly. You could still input actions this way, and audio would still sound correct in the final AVI file. Also, if you have fast storage, this could be used to allow you to record videos with a even higher framerate (e.g. for motion blur) or use higher visual quality options that your machine could otherwise not sustain :slightly_smiling_face:
If this enhancement will not be used often, can it be worked around with a few lines of script?
No. Given how this interacts with low-level functionality and performance is paramount, I don't think this can be implemented with GDNative either.
Is there a reason why this should be core and not an add-on in the asset library?
See above.