search

werman / noise-suppression-for-voice

Noise suppression plugin based on Xiph's RNNoise
GNU General Public License v3.0
4.59k stars 223 forks source link

Idea/Feature request: Output VAD as MIDI control message #7

Open DOCaCola opened 5 years ago

DOCaCola commented 5 years ago

The rnnoise_process_frame function returns a float value which is the "voice activity detection probability" rnnoise outputs. This value could be easily used for various automation tasks to control other plugins depending on the detected voice probability in the audio source.

werman commented 5 years ago

The rnnoise_process_frame function returns a float value which is the "voice activity detection probability" rnnoise outputs.

Looks like it.

This value could be easily used for various automation tasks to control other plugins depending on the detected voice probability in the audio source.

Could you elaborate please? And also what is " MIDI control message"?

I'm don't have knowledge in audio processing - I just packages this lib.

DOCaCola commented 5 years ago

I am no expert in LV2 or LADSPA but i know VSTs are able to output MIDI data.

The VAD float could be packaged in a MIDI. This would make this value usable in a DAW (digital audio workstation) that hosts this and other audio plugins. That means you could use the VAD value packaged in a midi message to control another plugin's settings in real time. A very basic example would be to turn down the volume of another audio track when voice is detected.

MIDI has two type of data, notes and control messages. Since the VAD has no relation to an audible note, a control message would probably make most sense. Control messages are/were mostly used to control certain settings of (hardware) MIDI devices, but i think they don't have too much meaning in modern DAW environments. They are usually used today for these mentioned automation tasks as far as i am aware. Control messages can hold an integer value of 0 to 127.

Here is a list with available control messages. https://www.midi.org/specifications-old/item/table-3-control-change-messages-data-bytes-2

For example it would probably make sense to output the VAD value (mapped to an 0-127 integer) as a "Effect Control 1" control message.

MIDI message are sent by using the sendVstEventsToHost function in VSTs. Here is an example i found where midi messages are generated. https://github.com/ataweg/meeblip-VST/blob/master/source/MeeblipVST.cpp (check the sendMidiCC and sendVstEventsToHost functions).

werman commented 5 years ago

Thanks for the explanation!

That could be useful. I looked at LADSPA and probably something similar will be output control port but I don't know if any software uses it or if this even allowed... LV2 probably has something similar.

I'd be wary to implement something I'm not able to test so it's either you can implement and test it or I maybe will look into it later. But, yes, it looks like something useful.