search

deezer / spleeter

Deezer source separation library including pretrained models.
https://research.deezer.com/projects/spleeter.html
MIT License
25.63k stars 2.81k forks source link

[Discussion] Difference between normal finetune version... #242

Open farsdewibs0n opened 4 years ago

farsdewibs0n commented 4 years ago

As I was looking at the releases, there are 2 versions of each stems (normal one and labeled "finetune"). What does finetune means and is it useable, or superior, or something different?

Waffled-II commented 4 years ago

Where would they be located? I've looked through it and I don't see anything about finetune models, unless you're talking about 16KHz variants of the regular models.

farsdewibs0n commented 4 years ago

Ehere would they be located? I've looked through it and I don't see anything about finetune models, unless you're talking about 16KHz variants of the regular models.

Its on Releases https://github.com/deezer/spleeter/releases

Mixerrog commented 4 years ago

I found for the 1.43 older version of Spleeter, you could edit & add the finetune stems .json files & the program would download them & use them on the first run but the newer 1.49 program will not. I asked about this in issue #232 (said they were downloaded with 1.49 but I found they were not) & then again in issue #233 which has not been answered.

I assume the finetune models would be preferred for training as they are much larger files so in theory should give better separations. You can download them at the website https://github.com/deezer/spleeter/releases I was able to get them to work I think in version 1.49 by substituting these new finetune files for the training files inside each of the stem's training folders but I think there might be an error in how the program is running as per my issue #232, they answered that version 1.49 should download these finetune stems the first time Spleeter version 1.49 runs but as I said it did not for me.

I then did stem comparisons using the older training files compared with the new finetune training files and found no difference at all so still not sure if the program is using them or using them correctly? I did phase inversion addition of each of the 2 stems obtained (2 for each of vocals, drums, bass & other) and they all cancelled which proves no difference between using the 2 different training bases which I would have thought there should be? I have also tested this on 4 stems separations & got NO difference so I feel the 1.49 program is NOT using these finetune stems correctly & even wonder if additional training files would be used correctly?

Roger

mmoussallam commented 4 years ago

Hi @farsdewibs0n , @Mixerrog

The standards and finetune version are, in fact, the same model. The difference is that if you want to fineture (that is to say, keep training over the existing model parameter on your own dataset) it turns out the standard models where lacking some info and tensorflow was not able to pick up on it and initialize training from it.

So for the inference part (separating your sources) there is absolutely no difference between the models, but if you want to finetune it on your data you will need the finetune version.

Hope this answer your question

Mixerrog commented 4 years ago

Moussallam, Thanks for the support first. Why is the the finetune training models much larger training file than the regular files if they are not being used? I would think with a larger training files, you could expect at least slightly better separations or at least a change? You are saying they are not ever needed? I thought maybe they were necessary for better high frequency separations but that is not the case also? Roger

On Monday, January 27, 2020, 10:00:24 AM CST, Moussallam notifications@github.com wrote:

Hi @farsdewibs0n , @Mixerrog

The standards and finetune version are, in fact, the same model. The difference is that if you want to fineture (that is to say, keep training over the existing model parameter on your own dataset) it turns out the standard models where lacking some info and tensorflow was not able to pick up on it and initialize training from it.

So for the inference part (separating your sources) there is absolutely no difference between the models, but if you want to finetune it on your data you will need the finetune version.

Hope this answer your question

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub, or unsubscribe.

mmoussallam commented 4 years ago

No the finetune files are bigger because they embark a lot more data, that are required by tensorflow for training, but not used for the separation.