Xrayez
commented
3 years ago CPU-efficient compression algorithms are often patent-encumbered, unless you use something really old. By "CPU-efficient", I refer to algorithms that use little enough CPU to be usable in real-time manner, with a very low risk of frame dropping — VP9 and AV1 are too demanding for this.
Still, Godot could have ability to encode webm: https://github.com/godotengine/godot/blob/b3c2281706778cf5311a68fe0071aa2740d01131/thirdparty/libvpx/vpx/vpx_encoder.h.
But when we talk about use cases like troubleshooting, for instance, GitHub does not support drag-n-dropping webm into issues (only mp4 and mov are supported as of now).
Some Goost users have asked me about ability to record videos as well for similar purposes. Due to complexity and patent-encumbered algorithms, all I could do is to implemented gif saver: goostengine/goost#104. Users look satisfied now, the problem is that gif is such a legacy format, and if we were to choose between apng and webm/mp4 nowadays, that would be the latter, obviously.
aleksfadini
Calinou
seocwen
Gastronok
TechnoPorg
ellenhp
keptsecret
ghost
theoway
Note: I discussed this with reduz a few months ago, so the feature is probably good to implement. We may want to discuss the specifics further, still.
Describe the project you are working on
The Godot editor :slightly_smiling_face:
Describe the problem or limitation you are having in your project
If you've ever watched indie game trailers, you've probably noticed at some point that stuttering and dropped frames are quite common, especially in lower-budget productions. Also, when troubleshooting specific issues such as physics, audio or rendering bugs, having pixel-perfect video and audio recording can be valuable.
There are several ways to record gameplay videos. None of them are perfect, but some of them are more suited to trailer production and troubleshooting than others:
Using a screen/window/game recording tool (OBS, AMD ReLive, NVIDIA ShadowPlay, …)
Upsides: Easy to acquire, well-known solution.
Downsides: It can take some time to configure the software to avoid frame drops. Also, while hardware-based encoding solutions such as NVENC free up CPU resources, they often result in lower visual quality. Hardware-based encoding is also dependent on your GPU model. CPU-based encoding can be very expensive at high resolutions and framerates. Also, however small it may be, the quality loss may be undesirable for troubleshooting purposes (e.g. pixel-perfect comparisons).
Using non-realtime simulation and a PNG frame sequence output
Upsides: Can achieve very high visual quality, since frame time budget essentially becomes a non-issue. Your trailer can take as long to render as you want to :slightly_smiling_face:
Downsides: Does not allow for player input, and audio cannot be recorded. If you want to show gameplay sequences this way, you need to implement a demo/replay system in your game, which is a lot of work if you only plan to use it for a trailer. When a game trailer relies on non-realtime rendering, audio is typically added in post-production. (There are ways around this, but it's not easy to implement.)
Using a hardware capture card
Upsides: Works with any application or game. A good capture card will pretty much never drop frames, even if your internal storage is too slow (since capture cards generally have their own storage medium).
Downsides: Not only capture cards are expensive for an indie developer, but they also have many limitations, especially if you want to use a high refresh rate monitor for your own purposes. Linux support for hardware capture cards can also be spotty, especially for high-end options that support 4K @ 60 FPS capture.
Using built-in uncompressed video recording (what's proposed here)
Upsides: Uses very little CPU resources and no GPU resources.
Downsides: Requires lots storage space with fast sustained write speeds. A SATA SSD is required to save 1080p60 or 1440p30 video, and a M.2 SSD is required to save anything higher than that. Nonetheless, it's possible to compress videos after the fact to perceptually lossless formats to save a lot of space with very little quality loss (such as H.264 with CRF set to 15).
Describe the feature / enhancement and how it helps to overcome the problem or limitation
The de facto SSD requirement for uncompressed video recording used to be a problem, but now that relatively-large SSD availability is widespread, it's not as problematic.
Recording long videos will still require you to split the video into smaller segments (which you can then compress using an external tool), but most game trailers don't feature long, uncut gameplay segments anyway.
Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
From an user point of view, video/audio recording could be configured in the Project Settings to toggle various features such as audio recording (to record video only), FPS, etc. Ideally, it should be possible to render the video at a lower size compared to the actual window size to allow for supersampling while still having low storage requirements.
Recording would be activated at run-time by pressing a built-in input action which could have a default value such as Ctrl + Shift + R. Pressing the same key combination again would stop recording. When recording is started or stopped, a message should be printed to standard output. The "end" recording message should specify the duration of the recording that was saved, as well as its save location.
To decrease binary size, video recording functionality would likely be included in debug builds (editor and export templates), but not in release export templates.
As for the technical implementation, I don't know much about getting the audio and window's contents quickly and writing an AVI file with the resulting audio/video. Nonetheless, the (zlib-licensed) implementation of an AVI recorder in Red Eclipse could be used as an inspiration: https://github.com/redeclipse/base/blob/master@%7B2021-08-03T16:33:29Z%7D/src/engine/movie.cpp
Question: Why not implement compression support as well (even lossless)?
CPU-efficient compression algorithms are often patent-encumbered, unless you use something really old. By "CPU-efficient", I refer to algorithms that use little enough CPU to be usable in real-time manner, with a very low risk of frame dropping — VP9 and AV1 are too demanding for this. Having a high-end CPU should not be a requirement to make a good game trailer. Also, upgrading a SSD is often easier and cheaper than upgrading a CPU (which may require replacing the entire machine in the case of laptops).
Question: Can we further optimize the video recording for low-end machines?
By running the project in slow motion (while also slowing down the audio playback), the storage write speed requirement could be lowered significantly. You could still input actions this way, and audio would still sound correct in the final AVI file. Also, if you have fast storage, this could be used to allow you to record videos with a even higher framerate (e.g. for motion blur) or use higher visual quality options that your machine could otherwise not sustain :slightly_smiling_face:
If this enhancement will not be used often, can it be worked around with a few lines of script?
No. Given how this interacts with low-level functionality and performance is paramount, I don't think this can be implemented with GDNative either.
Is there a reason why this should be core and not an add-on in the asset library?
See above.