mochi-neko / voice-activity-detection-unity

A voice activity detection (VAD) library for Unity.
MIT License
44 stars 6 forks source link

Hi, how to animate audio ripples based on current recording? #4

Open WilliamCheen opened 3 months ago

WilliamCheen commented 3 months ago

Thank you very much for this library, it helped me a lot. I need to animate the sound waves during the recording process, and I have found a way to do it, it seems to get the current VoiceSegment, and then use voiceSegment.Volumn to do some calculations to correspond to the Image's sizeDelta.y. But unfortunately, I never found the mapping relationship. How to realize the sound wave animation during recording?

mochi-neko commented 3 months ago

VoiceSegment.Volume is calulated by root mean square (in range [0, 1]) of wave data in a time segment. This value is usually small, e.g. 0.01~.

An idea of mapping volume to animation is as follows:

  1. Specify an upper threshold of volume, e.g. 0.2.
  2. A lower threshhold of volume equals to a specified value at CumulativeVoiceActivityDetector.activeVolumeThreshold (or QueueingVoiceActivityDetector.activeVolumeThreshold).
  3. Normalize a volume value from lower threshold(=2) to upper threshold(=1), e.g. ( Mathf.Clamp(volume, lower, upper) - lower ) / (upper - lower)
  4. Use normalized value (in range: [0, 1]) as an animation value.

Please adjust upper threshold value or calculation logic for your use case.

WilliamCheen commented 3 months ago

Thank you very much for the pointers, following the steps above I achieved a similar effect, thanks!