max/msp port

[pietrobolcato]

Hello Antoine! :)

First of all thank you for your incredible work! 🥇 I was wondering if you tried to port the PD external to max/msp and if not, if you have any clue whether this would be possible or not :) I would really appreciate some guiding and direction here!

Thank you so much in advance! keep up the great work! :)

[caillonantoine]

Hi ! Well, there should be no problem porting it, but I'm not planning to work on a max external for now. I'm open for PR :)

[pietrobolcato]

Sure thing! I started working on it, and so far so good but when I run inference on the model within max I get the following error:

RuntimeError: fft: ATen not compiled with MKL support

After googling, I tried installing MKL on my machine, linking the libraries to the XCode project of the Max external but this still happens. Do you have by any chance any clue why this is happening?

Thank you so much again in advance! :)

Ps: I'm on Mac 10.14.6

[esling]

I think this actually comes from PyTorch (as the problem is with the FFT function), the problem is documented here: https://github.com/pytorch/audio/issues/299 https://discuss.pytorch.org/t/error-using-fft-runtimeerror-fft-aten-not-compiled-with-mkl-support/21671

Hope this helps ;)

[pietrobolcato]

Yep! Those are the links I was scavenging for a while! But I still couldn't manage to make it work :/

The thing is that when I run inference directly from python it works, but with the c++ version not. I am a bit stuck here. Any clues / suggestions? :) the thing is, I have MKL installed now but still the c++ version doesn't seem to find it and I can't understand why

Thank you so much I really appreciate your input! :)

[esling]

Hmmm not sure to see why this would happen, but maybe you need to recompile libfftw3 and pytorch as well ?

My (completely uneducated) guess is that libtorch (c++) might need to be compiled with MKL after libfft is as well. A similar issue is this : https://github.com/pytorch/pytorch/issues/20156

And also this full resource for compilation options 👍 https://pytorch.org/docs/stable/notes/cpu_threading_torchscript_inference.html#build-options

[pietrobolcato]

Alrght I see what you mean! Thanks a lot! :) I will try to re compile libtorch and let you know how it goes!

I just have one question: why should I also recompile libfft?

Thank you so much once again! :)

[pietrobolcato]

Update: I re compiled pytorch from scratch with MKL support, now inference works! The max porting projects goes on :)

Thank you so much @esling! Your comment saved the day :)

[esling]

Excellent ! We are very excited about this max/msp port @seicaratteri, please keep us in touch with your advances or if you run into any other difficulties ;)

NB: I will keep the issue open for laters just in case

[pietrobolcato]

Will surely do! thank you so much for your help and support! :)

[pietrobolcato]

I have news! :) I might have succeded, but I am not super sure because the resulting ddsp processed audio sounds quite bitcrushed. The following are my test settings:

• Model: ddsp violin pretrained by you guys
• Reverb: in the wet version, applied with freeverb, while in the dry version none is applied

Here comes the tests (it works in realtime but I recorded the results so that you guys can have a listen):

Original sound: https://drive.google.com/file/d/1j3N9EIv6fJriBIAF7n3IFEEKnYgvAXtw/view?usp=sharing DDSP dry: https://drive.google.com/file/d/1S92oeUkuBAIWpfikdlMmPELmIJHhrteO/view?usp=sharing DDSP wet: https://drive.google.com/file/d/1OBTGK95-PjYah60qfCo05f6oJLSk_tjZ/view?usp=sharing

Original sound: https://drive.google.com/file/d/1rwG7MzpFxEPMqtjCTIzzL-q4Nu1JUX_K/view?usp=sharing DDSP dry: https://drive.google.com/file/d/1BWZdvTOJpt0Z_-yl2-lFyHKs-E2glUQf/view?usp=sharing DDSP wet: https://drive.google.com/file/d/1U6WildKlPSl7Mh9RYdAL0Esp2gGszkmQ/view?usp=sharing

What do you think? Is this an expected outcome?

Thanks a lot again! :)

[caillonantoine]

Congrats!! Ddsp is very touchy with the way data is preprocessed, so if you use another pitch extraction / loudness extraction pipeline than the one used for training it might result in that kind of glitch-sounding sounds... Can you create a PR of your external so we can try it ourself and see if everything works as expected?

[pietrobolcato]

Dear Antoine :) thank you for your input, very interesting! I am using a max/msp port of sigmund as pitch loudness extractor but perhaps it differs a bit from the PD one you are using. I am very happy to share a PR, I will clean up the code and then do it so that we can check together if everything is set up and working as expected :)

Thank you again for everything! I very much appreciate it! :)

[Klong-Ping]

Hello again! it's me seicaratteri just with another account :) I opened the PR! let me know if I can do anything to help / something doesn't work / something is unclear :)

[caillonantoine]

I'm closing the issue since it's addressed in #16

[pietrobolcato]

Hello there! :) I hope you had a great easter and everything is going well!

I was wondering whether you tried the port and if everything is working as expected. I was asking myself these days if the glitchy sounds originate from the conversion from float to signal of the output of sigmund. What do you think about it? :)

Thank you as always! :)

[pietrobolcato]

Update: I modified the source code of the max port of sigmund in order to output signals rather than floats, and the output from DDSP sounds the same. Therefore I can conclude that the glitchy sounds aren't caused by this :) any other idea?