Remove voice from songs with Spleeter

[manureini]

Issue type: Development discussion

Problem or question to be solved

There could be an option to remove all voice from an song to get "real" karaoke songs.

Suggested solution/s

-> https://github.com/deezer/spleeter

Your opinion on the topic

Would be a great feature ;)

[basisbit]

Thank you for this feature request! Spleeter definitely looks interesting and helpful for song creators. We will probably not add spleeter (plus all its dependencies) directly into the UltraStar Play game as that would complicate or break OS/platform-independence, but spleeter might be a good addition which could be shipped in a song creator tool package / bundle for windows and linux. For now, I added it to the tools section of our wiki: https://github.com/UltraStar-Deluxe/Play/wiki/Tools

[basisbit]

@bohning this tool might be interesting for UltraStar Creator, too.

[achimmihca]

Spleeter would also be useful in the song editor (pitch detection, timing) because it separates the voice from the rest of the music.

Pretty impressive AI stuff. Unfortunately it is not so easy to integrate because of different programming language and environment (Python)

Further, Spleeter is computationally intensive and requires some time. However, the website says that it is faster than real-time (e.g. decomposing 3 minutes of music should be doable in less than 3 minutes).

[achimmihca]

If you want to try Spleeter, there is a simple to use GUI app for Windows: https://makenweb.com/SpleeterGUI Further, there are some online services that use spleeter under the hood. I tried this and it produces the same output: https://vocali.se

[achimmihca]

I implemented the Spleeter integration successfully for the upcoming Steam release. It can be used in the singing game to mute the voice of the music, similar to classic karaoke.

Furthermore, I also added speech recognition via Vosk, which can be useful for creating new songs.

For now, I did the implementation in a private repository.