superpoweredSDK / Low-Latency-Android-iOS-Linux-Windows-tvOS-macOS-Interactive-Audio-Platform

🇸Superpowered Audio, Networking and Cryptographics SDKs. High performance and cross platform on Android, iOS, macOS, tvOS, Linux, Windows and modern web browsers.
https://superpowered.com
1.32k stars 284 forks source link

TimeStretching added extra latency? #745

Closed drusak closed 1 year ago

drusak commented 1 year ago

I have an issue with TimeStretching. I'm using it just for pitch shifting. So I use rate == 1, and changing pitch cents only. But I hear that latency is increased by small value, even if I set cents to 0 (no pitch shifting). Is there internal extra latency with using TimeStretching? or it should be used with tempo changing?

Thanks!

balazsbanto commented 1 year ago

Hey @drusak , thanks for reaching out!

It shouldn't add audible delay. On what platform are you using Superpowered, iOS, Android, web? Did you check out the docsite? https://docs.superpowered.com/reference/latest/time-stretching?lang=cpp#offline-timestretching

drusak commented 1 year ago

Hi @balazsbanto Yes, I checked that. I'm using pitch-shifting in realtime and didn't find anything about latency, but I hear it for sure( event if pitch cents is 0...

Any help appreciated

balazsbanto commented 1 year ago

On what platform are you using Superpowered, iOS, Android, web, etc? Could you share with use some sample code?

drusak commented 1 year ago

It is Android. Sure, I can share the code, but it is not much to share

timeStretching = new Superpowered::TimeStretching(sampleRate, 1);

bool process(float *input, float *output, unsigned int numberOfSamples) {
    timeStretching->samplerate = samplerate;

    timeStretching->addInput(input, (int) numberOfSamples);
    bool processed = timeStretching->getOutput(output, (int) numberOfSamples);

    return processed;
}

Hope it may help

gaborszanto commented 1 year ago

Time domain to frequency domain transformation adds a little bit of latency, this is coming from maths and is unavoidable. Based on the FFT size and the sample rate, the typical value is a couple of milliseconds.

drusak commented 1 year ago

Thanks, is it possible to calculate the value of latency knowing sample rate?

gaborszanto commented 1 year ago

If windowing is set to 4:1, FFT size is 2048 and sample rate is 48000, then the latency will be: 2048 / 48000 / 4 = around 11 ms.

drusak commented 1 year ago

@gaborszanto thanks, that's exactly I was looking for. How to know what value of window/fft size is set to (sample rate is known)? There are no such params in constructing of TimeStretching.

ivannador commented 1 year ago

You can calculate using these window sizes:

You can use FFT size 2048.

drusak commented 1 year ago

Thanks, @ivannador. I have minimum rate 1, so will user Window size 4 and FFT 2048. @gaborszanto if you do have additional thoughts it would be useful as well.