search

sweetcocoa / ddsp-pytorch

Implementation of DDSP (PyTorch), Differentiable Digital Signal Processing (ICLR 2020)
https://sweetcocoa.github.io/ddsp-pytorch-samples/
MIT License
149 stars 27 forks source link

what is the preprocessing for the timbre transfer? #3

Open JLC7 opened 2 years ago

JLC7 commented 2 years ago

On your demo page you convert the singing to violin. What is the preprocessing necessary to get the same violin audio? Use the voice as is does not produce the same result.

sweetcocoa commented 2 years ago

No preprocessing is needed for voice. only f0 and amplitude may affect the result of the violin model.

JLC7 commented 2 years ago

Thank you for the quick response. Here is the audio I get with the singing. It sounds different from the one on the webpage so I thought there was some preprocessing, did you shift the pitch of the voice like they did in the paper? If not, perhaps I am running your example incorrectly. Any guidance would be appreciated.

In order to transfer the voice to violin, I ran the following:

python test.py --input marg_singing_voice.wav --output out.wav --ckpt ../../ddsp-violin/200220.pth --config ../configs/violin.yaml --wave_length 100000