Can you provide examples for vocoder, and pitchshifting?

[gptshubham595]

Hi I wanted to use vocoder and pitchshifter function but was not getting proper output as I was confused about parameter limits to be used for factor.

Is it possible to provide example for a using vocoder function and pitchshifter parameter?

Thank you

[jurihock]

I was confused about parameter limits to be used for factor.

Which part of the README.md file confuses you exactly?

Is it possible to provide example for a using vocoder function and pitchshifter parameter?

Have you already noticed the demo page, exemplary showing different parameter combinations?

[gptshubham595]

Yeah, i checked the readme, but when i tried with pitch factor 2 then it was actually sounds like breaking, I was trying to write function to implement vocoder but failed

Here is what I did, I'm working on android ndk. I copied all files .h and .cpp and created this cpp_code.cpp file that i used to run,

I tried getting the audio read java float array that comes every 100ms and sending to the below function with parameter sampleRate and inputArray, I tried applying the pitchShifter and it worked but voice was cracking.

can you provide an example code for apply vocoder not necessary like below code but a C++ code like

// I Checked in vocoder there is encode and decode function, i have a input array and sample rate how should i do this?
Vocoder(const size_t framesize, const size_t hopsize, const double samplerate) :
Vocoder(1024, 256, 441000)

This is what i did

#include <jni.h>
#include <string>
#include "StftPitchShift.h"
#include <vector>
#include <iostream>

using namespace stftpitchshift;

extern "C"
JNIEXPORT jfloatArray JNICALL
Java_com_my_exampleKt_method(JNIEnv *env,
                                                                                jclass clazz,
                                                                                jfloatArray input_array,
                                                                                jint sample_rate) {
    StftPitchShift pitchshifter(1024, 256, sample_rate);

    std::vector<float> x(sample_rate, 0);
    std::vector<float> y(x.size(), 0);

    // get and put java Float Array elements inside a vector<float>
    float *ptr = env->GetFloatArrayElements(input_array, NULL);

    for (int i = 0; i < sample_rate; i++) {
        x[i] = ptr[i];
    }

    // Apply pitchShifter with factor 2
    pitchshifter.shiftpitch(x, y, 2);

    // Create a new java Float Array as return type is jFloatArray
    jfloatArray result = env->NewFloatArray(sample_rate);

   // feed vector<float> array of y inside result jFloatArray
    float *data;
    data = static_cast<float *>(malloc(sizeof(float) * sample_rate));
    for (int i = 0; i < sample_rate; i++) {
        data[i] = y[i];
    }

    // set Float Array
    (env)->SetFloatArrayRegion(result, 0, sample_rate, data);

    // free memory
    free(data);
    x.clear();
    y.clear();

    //return
    return result;
}

[gptshubham595]

Vocoder vocoder(1024,256,441000); vocoder.encode(// complex number how to get this parameter)

Do i need to use FFT over input_array and then apply that as parameter and the same input_array will be transformed.

[jurihock]

The StftPitchShift class is designed to process a single (e.g. large) audio file at once. If you plan to process a continuous audio stream (e.g. in 100ms steps), you have to use the StftPitchShiftCore class inside of your own STFT pipeline, implemented by yourself. The STFT pipeline provided in this project cannot handle a continuous audio stream.

[gptshubham595]

oh! got it for proper audio output for real time streaming I should use StftPitchShiftCore.

One more thing i tried vocoder like this but i got no sound for the respective can you help me if possible, definitely i will try working on StftPitchShiftCore.

coz in pitch shift i got the voice back but cracking a bit, in below approach for vocoder I'm unable to listen anything

extern "C"
JNIEXPORT jfloatArray JNICALL
Java_com_my_exaxmpleKt_vocoderNDK(JNIEnv *env,
                                                                                    jclass clazz,
                                                                                    jfloatArray input_array, 
                                                                                    jint buffer_size, //3528 used nearest and low power of 2 as 2048
                                                                                    jint sample_rate //44100 ) { 
    /* Convert input array via FFT
     * */
    stftpitchshift::RFFT rfft;
    vector<double> audioFloatData(2048); // or std::vector<double> audioDoubleData;

    // Populate audioFloatData or audioDoubleData with your audio samples
    vector<std::complex<double>> dftSpectrumFloat(audioFloatData.size() / 2 + 1);

    /* Fetch input array
     * */
    float *ptr = env->GetFloatArrayElements(input_array, NULL);

    for (int i = 0; i < 2048; i++) {
        audioFloatData[i] = ptr[i];
    }

    /* Transforming via FFT
     * */
    rfft.fft(audioFloatData, dftSpectrumFloat);

    //Apply vocoder
    Vocoder<double> vocoder(2048, 256, sample_rate);
    vocoder.encode(dftSpectrumFloat);

    /* Transforming back via iFFT
     * */
    rfft.ifft(dftSpectrumFloat, audioFloatData);

    //Feeding the float into the output array
    jfloatArray result = env->NewFloatArray(2048);

    float *data;
    data = static_cast<float *>(malloc(sizeof(float) * 2048));
    for (int i = 0; i < 2048; i++) {
        data[i] = (float)audioFloatData[i];
    }

    (env)->SetFloatArrayRegion(result, 0, 2048, data);

    free(data);
    return result;
}

[gptshubham595]

These two show pitch and vocoder applied data can you help me debug which step i missed

[jurihock]

As I wrote, you have to implement the proper STFT pipeline on your own and call the StftPitchShiftCore inside of it, e.g. between FFT and IFFT.

[jurihock]

Further reading: DAFX (chapter 7 Time-frequency processing)