Add support for QOA (Quite OK Audio) format

DeeJayLSP commented 5 months ago

Related to #1144, #4264, #7599 and godotengine/godot#80160.

Describe the project you are working on

Game with sound effects.

Describe the problem or limitation you are having in your project

The options we currently have on the engine for sound effects are:

WAV 16-bit: loses on size;
Vorbis and MP3: loses on decoding speed, making it not viable for lots of sound effects playing at the same time;
WAV IMA-ADPCM: loses drastically on quality;

ADPCM-XQ somewhat makes IMA-ADPCM better, but the resulting audio will still have noticeable artifacts;

About the same "problem or limitation" as #7599 (ADPCM-XQ proposal).

Describe the feature / enhancement and how it helps to overcome the problem or limitation

Add support for the Quite OK Audio Format (QOA).

When it comes to characteristics above:

Size: smaller than IMA-ADPCM (3.2 bits vs 4 bits);
Decoding speed: about the same as IMA-ADPCM;
Quality: while it somewhat loses to MP3 and Vorbis (mostly due to a noise that's also somewhat present in ADPCM-XQ, it's still much better than IMA-ADPCM;

This makes it an ideal lossy compressed format for sound effects.

Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams

Approach 1: QOA file importing

Users would be able to import QOA files like they would do with WAV, MP3 and Ogg. One problem with this approach however is that no common software nowadays have support for exporting files as QOA, and there's little interest in implementing it. At the time of this writing, FFmpeg has only implemented a decoder.

Approach 2: Import WAVs as QOA

Enable a Quite OK Audio importer on the Import As dropdown for WAVs:

It's an implementation of a built-in converter. This way, an user could work with WAVs within the game files, but internally it will be imported as QOA.

Approach 3: QOA as a WAV compression mode

This one was added months after the proposal was created, as I found it to be possible.

Add QOA as a WAV compression mode:

There is a bit of resistance when it comes to implementing encoders inside Godot, but for QOA I believe it makes sense since QOA conversion isn't supported by major file converters and there's little interest on it

Approaches 1 and 2 have been used in a custom working module I made (the screenshot above is from it). It adds around 8kB into release template file size, but this gets mitigated as long as at least 0.06-0.13s of WAV audio gets QOA'd.

Approach 3, on the other hand, only had a binary size increase of 32 bytes.

If this enhancement will not be used often, can it be worked around with a few lines of script?

Unlikely.

Is there a reason why this should be core and not an add-on in the asset library?

In the case of Approach 3 and considering there's some support for it, I don't think an importer with negligible size penalty meant for compression should be an addon.

DeeJayLSP commented 5 months ago

The implementation is ready, but I would like to see if there's support for it before opening a PR (which would likely still undergo a lot of changes).

Calinou commented 5 months ago

I prefer approach 2 personally, as it lets you use a lossless file as your source data (so you can keep doing edits and not suffer from generation loss).

However, would it be possible to provide QOA encoding as a checkbox in the WAV import options, rather than a separate import type? This would avoid the need for duplicating code for loop point handling and stuff like that. It would also ensure you can use QOA audio anywhere you can use WAV audio.

It adds around 16kB into release template file size

Does this mean it's possible to save .qoa files at run-time in an exported project?

It adds around 16kB into release template file size, but this gets mitigated as long as at least 0.13-0.25s of WAV audio gets QOA'd.

This sounds like a definitive win for any project that has sound :slightly_smiling_face:

DeeJayLSP commented 5 months ago

Does this mean it's possible to save .qoa files at run-time in an exported project?

We could add a function to do that, but I don't really see an use for it other than extracting audio data from a project.

Edit: that's already possible by writing data into a .qoa file.

However, would it be possible to provide QOA encoding as a checkbox in the WAV import options, rather than a separate import type?

I think it would be better as a third option in the Compress dropdown (where IMA-ADPCM is).

Adapting it to work with ResourceImporterWAV and AudioStreamWAV is a lot more complicated. We could still keep AudioStreamQOA as most of it is not duplicated.

Calinou commented 5 months ago

I think it would be better as a third option in the Compress dropdown (where IMA-ADPCM is).

Yes, that's what I wanted actually :slightly_smiling_face:

DeeJayLSP commented 5 months ago

I think implementing the converter inside ResourceImporterWAV is complicated. I don't even know if there's a way to make a single ResourceImporter import for two different Resource formats (or else QOA would need to be decoded by AudioStreamWAV).

Leaving it as a full module on modules/qoa makes the codebase simpler. And there's no duplicated conversion code in template exports.

I should also mention, the conversion code in question is actually an attempt to follow QOA's own reference converter, slightly adjusted to fit in Godot's style (and this is where the duplication is).

DeeJayLSP commented 5 months ago

Managed to decrease release size penalty from 16.416kB to 12.32kB (update: further to 8KiB, but after updating for 4.3, documenting it on the other ResourceImporters and adding the copyright it went to 16KiB).

~~Removing bpm, beat count and bar beats decreases size by 32 bytes, but it leaves some unusable buttons on import window, so it's better to not touch it.~~ Edit: that was a partial removal. A complete one managed to reduce by 4KiB.

The editor build on the other hand has a size penalty of 41KB. I don't think there's a way to not duplicate WAV conversion code without making a header just for it and including on both ResourceImporters.

A commit implementing this in core is available here.

DeeJayLSP commented 3 months ago

I think it would be better as a third option in the Compress dropdown (where IMA-ADPCM is).

Yes, that's what I wanted actually :slightly_smiling_face:

An alternative I've been thinking of.

qoa.h has qoa_encode() (already used in my current PR for importing) and qoa_decode(), which I believe could be used directly in WAV functions (ResourceImporterWAV::import() and AudioStreamWAV::set_data()) and make it play like it's a 16 bit PCM. This way, it can be imported as a compress mode instead of another audio stream type.

My only concern is that, unlike that PR, decoding is done at load-time instead of playback, trading CPU usage for memory usage.

It is a simple working alternative, but it could be better.

Here it is. Only 32B of penalty.

Edit: A 4th alternative, actually decoding QOA within AudioStreamWAV, has been submitted for review.

fire commented 3 months ago

Superseded by: https://github.com/godotengine/godot/pull/91014

godotengine / godot-proposals