Lecrapouille / gdcef

[Plugin][Version 0.10.0][Functional] Chromium Embedded Framework Webview for Godot 3 and 4
https://discord.gg/EckEwy7S5U
MIT License
222 stars 26 forks source link

Add CEF audio rerouting to audio_data signal #31

Closed pixaline closed 7 months ago

pixaline commented 1 year ago

Hello there, I found myself wanting to route the audio produced by CEF in to Godot. I implemented some prototype code from skimming other open source projects. I intend this to be a starting point for something better, perhaps added comments, audio buffer, or a toggle to toggle it on or off. The code only plays mono audio for now.

You can use this in Godot with the following example code:

audioPlayer = AudioStreamPlayer3D.new()
audioPlayer.stream = AudioStreamGenerator.new()
audioPlayer.playing = true

var browser = cef.create_browser(...)
browser.connect("audio_data", self, "on_audio_data")

func on_audio_data(data : PoolVector2Array):
    var playback = audioPlayer.get_stream_playback()
    if playback.can_push_buffer(data.size()):
        playback.push_buffer(data)
Lecrapouille commented 1 year ago

@parulina Thank you for your pull request! I'm not sure to understand what is the utility of rerouting the sound (for doing signal processing ?), but why not ! I made reviews. Feel free to modify the 2D demo https://github.com/Lecrapouille/gdcef/tree/master/addons/gdcef/demos/2D by adding your example. The 2D demo is made for showing the most features of this module, like a sandbox.

PS: I give a try: I hear no sound :( and I have an error inside your callback when halting the demo2D. In addition, the rerouting pipeline should be set only if desired by the user when creating a browser var browser = $CEF.create_browser(url, name, S.x, S.y, {"sound_rerouting":true}), else let the default sound pipeline like currently made, this will avoid forcing people to add a Stream node a callback for something they have freely without any additional line of code :)

pixaline commented 1 year ago

Thank you for the detailed feedback, useful to know what parts exactly to improve. I agree about the optional parameter. I can get back with the added improvements in a bit and double check if the demo would work.

I'm not sure to understand what is the utility of rerouting the sound (for doing signal processing ?)

Similar to showing the CEF texture on a mesh in-game, this allows you to route the audio CEF produces to AudioStreamPlayer or AudioStreamPlayer3D, making the audio play in the game instead of from the subprocess, I give an example: https://user-images.githubusercontent.com/6361957/236702464-669c51d6-ce41-45f0-a8da-660e548ad939.mp4

This is a game recording, normally this recording would exclude the played audio since CEF is a separate process. I route the CEF audio to my game's audio bus that also has spectrum analyzer, so that's how I can add visualizer at the bottom left as well as do subtle camera effects like screen shake or bloom depending on the lower frequencies like drums etc.

Routing the audio ingame also allows the user to change the volume in-game rather than having to adjust the volume of the subprocess (at least in Windows you have to right click audio device > volume mixer, find the cryptic "gdcefSubprocess.exe" with no icon, adjust its volume that it will also forget) making it more user friendly.

Lecrapouille commented 1 year ago

@pixaline Any news ?

Lecrapouille commented 7 months ago

@pixaline sorry I crapped the merged conflict, I do not how to use this poor Github UX. Commited manually