Experiencing pops and clicks when switching model

[stephanepericat]

hi,

I implemented a simple LSTM network using the compile time approach, as such:

RTNeural::ModelT<float, 1, 1, RTNeural::LSTMLayerT<float, 1, 32>, RTNeural::DenseT<float, 32, 1>> model;

I also created two different profiles that can be toggled via a combobox in my juce plugin. it works great, but when i toggle between the profile in Logic, it experienced a loud pop / click that occurs when parsing the json and assigning it to the model.

I tried to call model.reset() after parsing the json, to no avail.

Any idea how I could solve this ?

Thanks in advance !!

Stephane

[jatinchowdhury18]

Hello!

Sounds like a cool plugin you've got going! It's a little bit tricky to guess what might be going on with your case without more specific code examples, but I can share a couple tips relative to how I've done this sort of thing in the past.

First, rather than creating one model and loading in a new set of weights when you switch the combobox to a new value, I usually prefer to create two (or more) models as member variables in my class, and then swap between them when the parameter changes. So that might look something like this:

class MyPlugin
{
...
private:
    static constexpr int numModels = 2;
    using ModelType = RTNeural::ModelT<float, 1, 1, RTNeural::LSTMLayerT<float, 1, 32>, RTNeural::DenseT<float, 32, 1>>;
    ModelType models[numModels];
};

MyPlugin::MyPlugin()
{
    for (auto& model : models)
    {
        // load model weights here...
    }
}

MyPlugin::prepareToPlay(...)
{
    for (auto& model : models)
    {
        // reset model state here...
        model.reset();
    }
}

void MyPlugin::processBlock(...)
{
    auto& modelToUse = models[comboBoxValue];
    // do processing with modelToUse...
}

The problem with loading the model weights in real-time is that is takes a little bit of time, which can block the audio thread, leading to a click.

That said, it's definitely still possible to get a click anytime you switch between two distinct processing paths, especially when those paths are stateful, like with the LSTM. If you're still getting a click, there's a couple things to try:

• try calling model.reset() on the model that you're about to switch to, whenever switching between two models.
• try fading between the two models whenever you need to switch. I can go into more details on this if you need.

Hope this helps! Good luck :)

• Jatin

[stephanepericat]

hey Jatin,

thanks for the suggestion ! I will give it a try , as it might very well fix my issue since the pops do not occur as long as i don't reload the model...

I will keep you posted :)

[stephanepericat]

hey Jatin,

I tried your suggestion:

1. created an array of network instances: https://github.com/shortwavaudio/NocturneDSP/blob/rtneural/Source/PluginProcessor.h#L66
2. preloaded all the models in the constructor of PluginProcessor: https://github.com/shortwavaudio/NocturneDSP/blob/rtneural/Source/PluginProcessor.cpp#L27-L29
3. in the process block, before calling the forward method on the audio buffer, I check if a different model was selected, and if so, i call reset() on the model I am about to go into: https://github.com/shortwavaudio/NocturneDSP/blob/rtneural/Source/PluginProcessor.cpp#L284-L289
4. finally, I call process on the model stored at the active model index: https://github.com/shortwavaudio/NocturneDSP/blob/rtneural/Source/PluginProcessor.cpp#L177

While it did seem to help mitigate the problem, it is still noticeable, especially if i stop the audio playback, select a different model and then resume the audio playback... :(

You also mentioned something about fading between models; could you please elaborate on this point ? :)

Regards, Stephane

[stephanepericat]

This PR solved my issue. Thank you so much, @jatinchowdhury18 !!