Unexpected Behavior when Rendering Audio from Multiple Presets with PluginProcessor

[malekinho8]

Hello again @DBraun ,

I wanted to describe an issue that I have been experiencing lately with the PluginProcessor and if you would possibly be able to point me in the right direction to help debug this. I am using a very specific VST synthesizer (TAL-U-NO-LX Juno Emulator), and it works well 90% of the time, but there are a few cases where when I render audio from multiple presets in a certain sequence, the render engine will not output anything. This concerns me because I have been trying to create an audio dataset for presets in my VST synthesizer, but I'm now not sure if I can trust the audio outputs since it appears that sequential audio renders are not independent of one another...

Problem Description

To make things a bit more clear, here is the test code I am trying to run:

import torch
import dawdreamer as daw
import sys; sys.path.append('./')
from scipy.io import wavfile
from src.utils import *
from src.config import *

# load the df containing the preset to parameter mapping
df = torch.load('./dataset/tal-uno-preset-to-parameter-mapping.pt')

# load the engine
engine = daw.RenderEngine(sample_rate=daw_settings['SAMPLE_RATE'], block_size=daw_settings['BLOCK_SIZE'])

# load the plugin
plugin = load_plugin_with_dawdreamer(daw_settings['SYNTH_PLUGIN_PATH'], daw_settings['SYNTH_NAME'], engine)

# preset name
preset_names = ['LED Spartacus IV FMR','MT Tailwhip Lead','Super Funky Lead']

for preset_name in preset_names:

    # get the parameters from the df
    parameters = df[df['preset_names'] == preset_name]['parameters'].iloc[0]

    # assert that the length of parameters is NUM_TAL_UNO_PARAMETERS
    assert len(parameters) == NUM_TAL_UNO_PARAMETERS, f"Length of parameters is {len(parameters)}"

    # set the parameters of the plugin
    loaded_synth = set_parameters(plugin, parameters)

    # render the audio
    audio = render_audio(60,100,0.25,loaded_synth,engine,1,daw_settings['SAMPLE_RATE'])

    # specify where to save the audio
    audio_path = os.path.join('./sandbox', f'test-save-{preset_name}-params.wav')

    # save the audio
    wavfile.write(audio_path, daw_settings['SAMPLE_RATE'], audio)

    # save the parameters to a tal uno file
    save_param_vector_to_pjunoxl(parameters,f'test-save-{preset_name}-params', os.path.join(daw_settings['PRESET_FOLDER'],'test-2-may-28'))

Basically in this code, I am specifying the presets in a list that I want to render 1 seconds audio clips from, and looping over them and outputting the audio files to a specific directory. I am also outputting the parameter vectors to a .pjunoxl file (which is basically just an xml file with the parameter mappings that the TAL-U-NO-XL VST can read).

Expected Behavior

I expect that the code above should output 3 1-second wav files that contain audio rendered according to the parameters of the synthesizer preset. I can confirm that the preset audio from Python matches the expected preset audio, as I am saving the parameter settings into .pjunoxl files I can load into the VST plugin in Ableton.

Interestingly, when I run the code above for the list I have put above in the specific sequence I have put it in, i.e.

preset_names = ['LED Spartacus IV FMR','MT Tailwhip Lead','Super Funky Lead']

the Super Funky Lead preset applied to the PluginProcessor outputs no sound, but the other two presets are able to render the correct sound (confirmed by rendering MIDI clip in Ableton with the same settings as the code above). However when I run the code above with the list in this order:

preset_names = ['Super Funky Lead','MT Tailwhip Lead','LED Spartacus IV FMR']

the Super Funky Lead and MT Tailwhip Lead presets output sound, but the LED Spartacus IV FMR gives an empty wav file.

I've also tried isolating presets to see if that would help, for example, settings presets_names as follows:

preset_names = ['MT Tailwhip Lead']

and this gave an audio output that is quiet and completely unrelated to the synth preset that I have loaded to the PluginProcessor object.

What I would expect is that the order in which I load plugins and render audio should have no effect on the next audio render output, however, the results as I have explained above seem to prove otherwise. I don't think this is a problem with the synthesizer I am using because I am checking the synthesizer presets in Python by loading them to the VST in Ableton, and I am able to render the expected audio just fine.

I have also tried running the same code on a separate Windows system that also had the TAL-U-NO-LX VST synthesizer, and it behaved the exact same way. This leads me to believe there could be some strange behavior happening with my specific synthesizer interfacing with DawDreamer...

Regardless, if you have any idea how I might be able to troubleshoot or fix this issue potentially that would be great. I have been trying to understand what was going on for the last week or so and just decided to make a post because I don't know what else to try at this point. If there's anything else I can provide to help clarify things more, just let me know. Thanks so much for your hard work on this repository,

Best, Malek

[DBraun]

Sorry for the issues and thanks for reporting. Now that I've updated libfaust I really want to fix these kinds of issues.

• Use a normal block size of 256 or 512 in case you're using something extreme like 1
• Have you tried time.sleep(5) in between calls to render?
• Have you tried starting the notes with a minor time offset, like starting the note at 1 second instead of 0 seconds?

Have you tried preset_names = ['MT Tailwhip Lead']*100 but saving each one to a different file?

[malekinho8]

No problem, thanks so much for the prompt response. Here's what I've found from experiments based on your guidance:

• I am using a normal block size, 512
• I tried time.sleep(5) between calls and the behavior did not change
• Starting the MIDI notes with a minor time offset fixed the issue, or at least I can hear the expected audio from all 3 instruments at the right time. Interestingly with the MT Tailwhip Lead instrument, I hear a quiet sound at the very beginning of the rendered audio, not sure what to make of that. You can listen to it here if you are curious.
• I just tried the repeated list idea, and it also worked in that the first wav file was basically silent (though I could hear a small unrelated sound at the beginning), and all the other wav files were contained rendered audio. However, upon playing them back, I did notice slight variations in the sound timbre even though the preset has no random parameters or settings. You can hear all of the files for yourself here if needed.
• I also performed the repeated list idea + starting midi note 1 second later and rendering 2 seconds of audio. I expected there to be no differences between the audio files, but basically the same behavior as the repeated list idea performed with the original settings is occurring (i.e. the first audio sample has a weird silent sound at the beginning and the other sounds work fine but have slight timbral variations).

Still not sure if I trust the audio rendering process enough right now to close this issue. If you have any other suggestions or guidance on things I could try, I would greatly appreciate it, but I also understand you have probably put a tremendous effort to getting this code base working so if I have to be patient it's no big deal. Good luck and if there's anything else I can do to help fix this, please let me know.