Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
134 stars 2 forks source link

Virtual audio output devices #778

Open 5H4D0W-X opened 9 months ago

5H4D0W-X commented 9 months ago

Is your feature request related to a problem? Please describe.

It's not possible to selectively stream audio sources out of Resonite

Describe the solution you'd like

The ability to spawn an interface with a field that accepts an IAudioSource, and have Resonite generate a virtual mic that outputs whatever audio source is in the field. This could be especially useful together with the planned audio DSP system. As an example, if a user had a voice changer that uses the audio DSP system to modify their voice, they could use Reverse Audio Streams to be able to get the modified voice from Resonite and use it in voice calls, live streams or video recordings

Describe alternatives you've considered

There are none that don't involve muting audio for the user

Additional Context

This could also open possibilities such as modifying audio outside of Resonite before streaming it back in, allowing fast fourier transforms or audio effects to be used with in-game sources

Frooxius commented 9 months ago

I'm not sure I quite understand what exactly are you looking for with this?

For the modifying voice and such, ProtoFlux DSP will support doing that, you can just route the audio through a bunch of filters. There is no need for any sort of "reverse audio stream" to achieve that functionality.

Are you looking to essentially have a separate audio output? Like you'll have another audio device in your system that you can feed your audio data into for any external use?

It's potentially possible, but I'm not sure how involved it would be to create virtual audio devices like that, we'd have to do some research.

TisFoolish commented 9 months ago

From what I've gathered, what they want is separate audio sliders (master, voice, ui, etc) for the stream camera

ghost commented 9 months ago

this is related to #777

5H4D0W-X commented 9 months ago

separate audio sliders (master, voice, ui, etc) for the stream camera

That's closer to issue #777 , I'm thinking of what froox said where creating a stream makes a virtual audio input device on your PC along with a panel like the audio stream panel. The virtual audio device then exclusively outputs the audio from whatever source is in a field in the panel.

The idea of applying effects externally is not really relevant, Im just saying it could be a temporary replacement for the current non-existence of built in audio DSP.

lxw404 commented 9 months ago

An additional use case for this in addition to having https://github.com/Yellow-Dog-Man/Resonite-Issues/issues/777 is for natural language processing / audio recognition / translation where an output source that contains purely voice or whatever audio you're focused on streaming out can be fed directly into an inference model.

Also, using Resonite as a live instrument is quite appealing I would say, and having the ability to take any in game audio source, preprocessed or otherwise, out of the game into another sound system would be great.