search

tw40210 / Hierarchica_remake

2 stars 2 forks source link

Reproducing results for "Shape of You" output on demo site #1

Open smcio opened 2 years ago

smcio commented 2 years ago

Hi there,

Very interesting work here - thanks very much!

I was wondering if you wouldn't mind me asking a few questions that popped up whilst trying to reproduce your output from the model for "Shape of You" (Ed Sheeran), as per your demo site here: https://tw40210.github.io/Real-time-pop-music-accompaniment-generation-according-to-vocal-melody-by-deep-learning-models_DEMO/

I downloaded the MP4 for "shape of you" for "pure vocals" from your demo site, converted it to WAV, and then ran the resulting through worldvocoder.py to get a WAV file written to 'offline_output/finaltest.wav'. The sample produced for me sounds a bit different to the sample on your demo site, and I was hoping to figure out why, because your sample sounds really good IMO!

Is the model shared on github (standard_checkpoint/3150_on079l3dnesec8.pth) the exact same model used to output the accompaniment on your demo site? If so, would you mind sharing with me what values for 'song_bpm' and 'song_tone' you used in hparam.py, please, and would you mind also sharing what soundfont (.SF2 file) you're using as well, please? Anything else I would need to tweak to get the same results?

Thanks very much indeed for your time.

tw40210 commented 2 years ago

Hi, I've checked my code it's still reproducible using "3150_on079l3dnesec8.pth". I think I can't provide the soundfont file but I believe the fact that if you use 'song_bpm'=96 and 'song_tone'=4 which are also mentioned in my paper, the result should be very similar no matter which soundfont file you choose. Good luck!

smcio commented 2 years ago

Thanks very much for checking that - much appreciated.

I've just noticed I'm getting a RuntimeWarning being raised on line 102 of preprocess.py for a divide-by-zero error, i.e.

/Users/XXXX/Hierarchica_remake/preprocess.py:102: RuntimeWarning: divide by zero encountered in true_divide

Perhaps this is why I'm getting different results. Have you ever observed this error before, by any chance?

Perhaps you also wouldn't mind pointing me in the direction of where you got the "shape of you" WAV file you used in your sample outputs, please?

Thanks again for your time - it's much appreciated :)