search

deezer / spleeter

Deezer source separation library including pretrained models.
https://research.deezer.com/projects/spleeter.html
MIT License
25.67k stars 2.81k forks source link

Extract stems using same song as training data? #420

Open amine-aboufirass opened 4 years ago

amine-aboufirass commented 4 years ago

This is a long shot, but I wonder if there is any way of extracting stems from songs using spleeter without having access to hundreds/thousands of examples of training data.

I have tried to use phase cancellation from different parts of the same song to isolate individual elements but the results are not spectacular. Even for electronic music there seem to be slight phase offsets which always punch through. In some cases the phase inversion technique is ok but not enough that it an be a viable way of extracting stems.

I have had success using spleeter in isolating drums, vocal, bass, piano and other but sometimes I need more than 5 stems. My goal is to isolate as many elements as possible such that I can recreate tracks in my DAW and get insight on the underlying song structure.

Perhaps there is a way of combining phase inversion, onset detection and machine learning/NN to extract features from a song if a large number of training data is not available? Perhaps parts of the same song can be used as examples of training data?

Thanks.

junh1024 commented 4 years ago

Perhaps parts of the same song can be used as examples of training data?

You can, but it will be really slow. You can already do this much faster using traditional FFT tools such as Mashtactic.

amine-aboufirass commented 4 years ago

Perhaps parts of the same song can be used as examples of training data?

You can, but it will be really slow. You can already do this much faster using traditional FFT tools such as Mashtactic.

Can you use FFT to literally split up a song into clean stems for each instrument? Also are there any open source tools to do this? Mashtactic is a commercial product...

junh1024 commented 4 years ago

Can you use FFT to literally split up a song into

Yes

clean stems for each instrument?

No, it will sound a bit messy.

open source tools

You DID say "parts of the same song can be used as examples of training data", so FFT tools subtract, except that they don't have to be the same phase.