LSTM (3 layers, 30 hidden size) does not load first time opened. Also DC issue? [VIDEO]

[38github]

Plugin compiled about thirty minutes ago. LSTM model does not load first time opened. Also what I believe is a DC issue.

Video demonstration: https://youtu.be/uZFtO_z--os

NAM model: nam_marshall_jtm45_t10_ash_tubb-lstm_350_epochs.zip

@mikeoliphant @sdatkinson

[mikeoliphant]

I think this might be a Reaper LV2 bug. I get random load failures for that file in Reaper, but have no issues in Ardour.

Maybe Reaper doesn't like it if an LV2 work action takes too long?

[38github]

The first time it worked well in Ardour but when I save the project and re-opened it I only got the DC offset sound no matter if I choose a non-LSTM model and back to the LSTM. The non-LSTM gave sound though.

[rerdavies]

@mikeoliphant: There were load/save issues in Ardour that were addressed by the outstanding pull request.

@38github: You may want to take a look at the latest release (went out this morning) of PiPedal/ToobAmp, which contains a new LV2 port of Neural Amp Modeler, using the same compute backend as this project (NeuralAmpModelerCore). TooB Neural Amp Modeler seems to load and save the model in Ardour without problems.

[38github]

@rerdavies Nice! I will check it out tomorrow evening.

[mikeoliphant]

@rerdavies The issue in Reaper (at least in my testing) isn't specific to activation. For me, it happens randomly when loading this specific model. I tested your PR and it has the same issue.

@38github The weird thing is that I have only seen this problem happen with the specific model you shared. I haven't been able to reproduce the issue in the VST plugin, but LV2 code doesn't know anything about LSTM vs WaveNet models - that is all handled by the underlying core code.

[mikeoliphant]

I've now seen this happen with other LSTM models (although it is maybe less frequent that the big model you shared?). I have yet to see it happen with a WaveNet model.

[rerdavies]

@38github: With respect to the DC issue, this is actually kind of a known issue with LSTM models in particular, according to early published papers on LSTM modelling. One of the very earliest LSTM guitar modelling papers used DC blockers on training data, and model outputs during training to avoid this problem.

It's a very good catch. But it's hard to say whether a DC blocker is appropriate for all LSTM models. You might want to report the issue at https://github.com/sdatkinson/NeuralAmpModelerCore which is where the fix would have to be made anyway.

Or possible ping @jatinchowdhury18 who was (I think) the lead author on the paper I have in mind. (Also author of RTNeural which TooB ML uses. If anyone on planet earth knows whether LSTM models do need a following DC blocker, Jatin would.

[mikeoliphant]

There is an issue tracking the DC offset here: https://github.com/sdatkinson/NeuralAmpModelerPlugin/issues/271

[falkTX]

Maybe also worth asking the aidadsp folks, as the aida-x code always had a dc blocker from the start https://github.com/AidaDSP/aidadsp-lv2/blob/main/rt-neural-generic/src/rt-neural-generic.cpp#L581

it runs right after the model processing.

I recall some conversations about this, related to possible aliasing and other issues within LSTM.

[falkTX]

Regarding DC Offset, here is a file that generates DC

AMP Orange Nasty.nam.txt

Clearly visible in the waveform from audacity

[38github]

I had difficulties with a model not loading and only making the dreaded DC offset sound. I then remembered someone saying they couldn't load my WaveNet models that had "nam" in the beginning of the files. Therefore I thought why not try and rename this LSTM NAM model (even though other files with similar names work) and without nam it loaded where it before just didn't want to. The same problem in NeuralAmpModeler 0.7.4 using WINE.

I still doesn't load every now and then though.

[38github]

I wish I could program this myself but is anyone willing to try to add a quite steep highpass (so that it doesn't affect audible frequencies) at maybe 5 Hz and see if it solves any of the issues? Maybe it magically solves the denormalization (if that is what is happening) issue also.

EDIT: Accidentally typed lowpass instead of highpass. It has been corrected.

[38github]

I found a reason why it didn't load my LSTM models. At the end of the nam files is a sample_rate parameter and it was set to null instead of 48000. When I corrected them I was able to load them.

[mikeoliphant]

DC offset has been fixed for a while. Feel free to open a specific issue for the LSTM model loading if that is still happening for you.