search

deezer / spleeter

Deezer source separation library including pretrained models.
https://research.deezer.com/projects/spleeter.html
MIT License
26k stars 2.85k forks source link

[Discussion] increasing extracted vocal quality #234

Closed davidhin closed 4 years ago

davidhin commented 4 years ago

Many high-quality acapellas can be found online. The vocals extracted by spleeter can sometimes be a bit noisy or artifact-y. Would it be worth looking into training a model to "denoise" the vocals, similar to how an image may be denoised, using Spleeter as the "noise" introducing mechanism? I know this would be a hacky downstream task, but it could be useful for pure convenience purposes (music production) at the moment.

My experience in machine learning is all in NLP, so I'm not too sure about the progress of AI in audio/music tasks; however, if this idea is reasonable I would be happy to look more into this, particularly if there are any starting points / relevant research.

HandsomeDevilv112 commented 4 years ago

I'm having this as well, I've been looking into using audacity to try to reduce the "echo" quality to the extracted vocals. However, I haven't found anything solid at this time.

mmoussallam commented 4 years ago

Hi @davidhin

Reducing separation artefacts is actually an active research topic.. and it's not easy at all :) if you're interested in current trends you will find very good pointers here.

redbar0n commented 3 years ago

any luck so far @HandsomeDevilv112 ?

I'm trying to do the same, in Audacity, but haven't had much luck. It should be possible using some kind of tool, since Audo.ai uses spleeter and is able to remove the echo / machine / chromatic artefacts from the spleeter vocals (as observed when I tried their demo on a small clip).

Update: It seems like the echo / machone / chromatic artefacts are not really from spleeter originally, but from the reverberation in the room of the audio recording. Spleeter simply doesn't filter it out, as the reverb also registers as the same stem (e.g. vocals). So a de-reverberation plugin to Audacity should do the job. You can confirm if this is the case for you by trying the Acon Deverberate 2 plugin for Audacity (it will be a demo version, where output clip will have silences inserted, and the full version is pricey).

Anyone know of a free de-reverb plugin for Audacity, or a free standalone de-reverb program?