search

GuitarML / PedalNetRT

Deep Learning Networks for Real Time Guitar Effect Emulation using WaveNet with PyTorch
https://www.facebook.com/smartguitarml
GNU General Public License v3.0
343 stars 39 forks source link

does it work with reverb pedals? #32

Closed creativeguitar closed 2 years ago

creativeguitar commented 2 years ago

I used this repo to model 1 of the BigSky reverb pedal effects and the result was a clean sound similar/identical to the clean signal (no reverb). any ideas if this repo only works for overdrive or I am doing something wrong?

diff_spectrogram signal_comparison_e2s_0 247 detail_signal_comparison_e2s_0 247

mishushakov commented 2 years ago

time-based effects, such as reverb, delay aren't supported

try this instead: Steerable discovery of neural audio effects

creativeguitar commented 2 years ago

thank you! By the way, thanks to you I am getting into all this now (model training, elk os, plugins, etc). Very inspiring. I started using Steerable and as result I got a .pt file. Any ideas how to convert this pt file to a vst3 plugin ? I looked at the JUCE forum but no success.

mishushakov commented 2 years ago

thank you! the research is coming out quicker than you could build plugins

.pt files are for PyTorch models

these models could be loaded into JUCE using RTNeural

the hardest part is manipulating the audio data to match the model inputs this is very math-heavy and requires need deep understanding of the model to be able reconstruct the inputs

the level of engineering complexity and the sound is what attracts people to work on the project

GuitarML commented 2 years ago

Thanks @mishushakov!

@creativeguitar I'd be interested in being able to run the Steerable models in real-time. After a quick look at their paper I think the main challenge would be having the processing power to keep up with real-time. Their model uses dilated convolutional layers, which is similar to wavenet (used here in PedalNetRT) but in order to have a longer receptive field (up to 2000ms for reverb, as opposed to 50ms for distortion) they made the network much larger than what we typically use in PedalNetRT for the real-time models. That being said, you can bump up the params in PedalNetRT to make the receptive field larger and you might be able to capture reverb, but it would be virtually unusable for real-time with the current plugins on current PCs.

Converting the .pt to the .json used in the plugins is a matter of re-formatting the data properly. The Automated-GuitarAmpModelling code is written with Pytorch and could write .pt models, but I think for readability the original author opted to export as .json. The data for the model is the same between .pt and .json, just formatted differently. Hope that makes sense.

If you want more clarification feel free to ask, otherwise I'll close this issue shortly as the original question was answered.