Calinou
commented
2 years ago Related to https://github.com/godotengine/godot-proposals/issues/1836 and https://github.com/godotengine/godot-proposals/issues/2765.
I'm creating a racing game where I have pre-recorded engine sounds which I added to the car AudioStreamPlayer3D, I would like to some audio post-processing to the indivual players of each car, for example rpm-based pitch scaling, high-frequency rev-limiting and reverb based on current ambient the car is located.
I'm surprised you need real-time audio effects to do that. Typically, engine sounds are done by pitch-shifting and blending a few prerecorded sounds together depending on RPM. No advanced real-time effects are needed. You usually only need 6 sound samples, maybe even just 5:
- Idle
- Forward low RPM
- Forward medium RPM
- Forward high RPM
- Reverse low RPM
- Reverse high RPM
This is how most AAA titles still do engine sounds nowadays :slightly_smiling_face:
ghsoares
marcinn
Describe the project you are working on
A racing game with car engine SFX.
Describe the problem or limitation you are having in your project
I'm creating a racing game where I have pre-recorded engine sounds which I added to the car AudioStreamPlayer3D, I would like to some audio post-processing to the indivual players of each car, for example rpm-based pitch scaling, high-frequency rev-limiting and reverb based on current ambient the car is located. The problem is that I need to add some of these features by modifying the sound wave itself, like a custom-filter, because some of these features would change the sound wave with high-frequency, which running in
_processor_physics_processand simply changing the pitch scale and the volume wouldn't be sufficient.Another problem is that even if there was a way to create custom audio effects, it would require to insert then in a audio bus, which would apply the processing for all the cars in the game, which isn't what I want.
I'm playing around with GDNative and AudioStreamGeneratorPlayback to generate the audio, but there is no way to modify a source audio frame, I can only generate plain new audio frames, unless I pass a AudioStreamSample to the generator script, read the frames from the data and use then to generate the new audio, but this requires a lot of effort to convert the pcm byte data to audio frames.
Describe the feature / enhancement and how it helps to overcome the problem or limitation
So I'm proposing two features:
Creation of custom audio effects
The idea is to have a way to inherit from
AudioEffectandAudioEffectInstanceand expose virtual functions to modify a chunk of audio frames, it receives the source audio frames chunk and expect the processed frames, each frame is a Vector2, being x the left channel and y the right channel. It would have some problems with GDScript as not being fast enough to handle a high sample rate, but I think it would work better with GDNative or C#.Per AudioStreamPlayer audio effect stack
The idea is the ability to have a audio effect stack for each AudioStreamPlayer instance, which is used to apply the effects in the audio frames before sending to the bus. This would be useful to apply specific post-processing effects to individual streams players, without interfering in the processing of other streams. One example is having two cars where one is inside the tunnel which have a reverb effect in the engine sound while the other car outside the tunnel sounds normal.
Describe how your proposal will work, with code, pseudo-code, mock-ups, and/or diagrams
For the custom audio effect, it would need to allow AudioEffect and AudioEffectInstance script inheritance, in the AudioEffect it would have
instancevirtual function to return a custom AudioEffectInstance and have two virtual functions for the AudioEffectInstance:void process(PoolVector2Array source_frames)to process a chunk of audio frames;bool process_silence()tells if this audio effect should process plain silence audio frames.Note: It seens that in 4.x the API it's already exposed, at least by looking at the AudioEffectInstance documentation there's the functions exposed, but I hope there is a way to backport into 3.x, maybe I can contribute with this.
Now, for the per AudioStreamPlayer audio effect stack, it could have a interface simillar to adding effects into a bus, which adds the effect into a internal effect vector, create the effect instances for each effect everytime the stack updates and use then when mixing the audio frames buffer by calling
instance->process(buffer, temp_buffer, buffer_size).If this enhancement will not be used often, can it be worked around with a few lines of script?
For custom audio effects one would require to manually sample from a audio stream and insert modified frames into a AudioStreamGeneratorPlayback, and wouldn't be much scalable. For per AudioStreamPlayer audio effect stack, it requires core modification to change how AudioStreamPlayers send audio buffer to the audio buses.
Is there a reason why this should be core and not an add-on in the asset library?
Simillar to above, the addon would require to sample from a AudioStreamSample and feed into a overridable function to modify the audio frames, but wouldn't be able to implement per AudioStreamPlayer audio effect stack.