search

keijiro / Lasp

Low-latency Audio Signal Processing plugin for Unity
The Unlicense
1.57k stars 121 forks source link

How to change sampling rate? #49

Closed Darth-Carrotpie closed 2 years ago

Darth-Carrotpie commented 2 years ago

Hello, awesome package! I've tried using Lasp.AudioLevelTracker.audioDataSlice to read raw audio data and from it construct a precise size buffer. My use case is a neural network, it has to have a precise length max input, therefore this package seems like a perfect candidate for fast data acquisition. All I'm doing is looping through audioDataSlice[i] and writing values to my buffer until it's full. Then run inference with it.

Problem: NN Input also requires sample rate of exactly 16000, not more not less. Seems like in this package default is 48 000.

Question: How to change sampling rate from 48000 to 16000? Cannot a way how I'm meant to do it. AudioLevelTracker does not give access to it's device. I cannot just Lasp.AudioSystem.GetInputDevice(_input.deviceID).SampleRate = 16000;` because there is no setter in SampleRate constructor.

keijiro commented 2 years ago

SoundIO (the audio backend library) uses the default sampling rate of the audio device, so you should change the device settings. However, I don't think modern audio devices support 16kHz sampling rate. You may have to implement sampling rate converter by yourself.

keijiro commented 2 years ago

I'm closing this issue now. Please reopen it for further problems.

Darth-Carrotpie commented 2 years ago

Ok, thanks! I will look for a way how to down sample it. Is the approach I described the correct one to get raw audio data? I mean, by using audioDataSlice ?

keijiro commented 2 years ago

yes

Darth-Carrotpie commented 2 years ago

So I've researched various ways for downsampling and come to a conclusion that it's very not feasible. Downsampling requries a complicated algo implementation with filters and whatnot, while existing solutions (like NAudio) don't wanna run in Unity. Another huge downside, it has an additional CPU costs, as opposed if sampled correctly right away.

I've tested Unity's default Microphone.Record(a, b, 16000) and it seems to work properly, thus devices seem to support the sampling rate. However the default approach appears to be slow if used on "per frame" basis.

Within the soundlibio source I've found that there's a function called soundio_device_supports_sample_rate in script libsoundioa/src/soundio.c. Moreover, in the example soundlibio/example/sio_record.c they manually select the sampling rate using the mentioned function via instream->sample_rate = sample_rate;. Is it baked into soundlibio used in this package, if yes, could I make an addition to wrapper and add the functions to check for support and select sample rate? Or is it perhaps not doable for some reason which I don't understand? If no pull requests are allowed for the repo, could you make it?

There are also functions in libsoundioa/src/wasapi.c called detect_valid_sample_rates and even add_sample_rate which suggest available functionality.

keijiro commented 2 years ago

I think you can modify the sample rate here: https://github.com/keijiro/Lasp/blob/v2/Packages/jp.keijiro.lasp/Runtime/Internal/InputDeviceHandle.cs#L196

Darth-Carrotpie commented 2 years ago

For future reference, to anyone who might be interested in this problem, I went on not to change the sampling rate of the device. It proved simpler and a more universal approach (to adapt to various rates of many devices out there) to just resample everyting every few frames. I used NWaves.Operations for that purpose, because most other solutions did not work or did not even load.