Feature requests: Remove vocal

[wongcheehong]

There might be some case where the music video is come with vocal. There are many projects that use AI to remove vocal. Integrate this kind of feature would be nice for this project.

[xuancong84]

My forked branch (https://github.com/xuancong84/pikaraoke) has implemented this function in both ways:

1. DNN-based vocal remover (from https://github.com/tsurumeso/vocal-remover): vocal removal quality is excellent on most voices; however, for certain synthetic voice (like Project DIVA), it does not always work very well
2. stereo track subtraction: does not work well whenever audio compression loss is high, might not work on non-standard stereo vocal channel encoding

My implemented solution is to run another dedicated process to extract and process audio and store into an .m4a file, and then launch vlc with --input-slave to play video from one file while playing audio from another file synchronously. I feel that is quite ad-hoc, but it can co-exist with pitch shifting. Is there a more elegant method?

[wongcheehong]

@xuancong84 I have tried your code but it doesn't work on my Windows Machine. I have workaround some of the issues arise before this but this one is much more trickier. I assume you only test your code on Linux and Mac machine.

[xuancong84]

@wongcheehong Window's support of Python is very limited. Even if it can get past this one, PyTorch is not going to work unless you use WSL2, so vocal splitter will not run. However, if you use WSL2, PyTorch will work, but pygame does not work, nothing will get displayed.

[xuancong84]

Hi @wongcheehong , it is truly amazing that Facebook has made their PyTorch working on Windows (with CUDA/GPU-acceleration working on both native Windows and WSL (Windows Subsystem Linux)). As a result, I have managed to make the vocal splitter (with CUDA/GPU acceleration) working on Windows. See what other problems do you encounter. Thanks for pointing out!

[wongcheehong]

Hi @xuancong84, please enable issues feature in your repo so I can open issue easily.

[xuancong84]

Hi @xuancong84, please enable issues feature in your repo so I can open issue easily.

Done, thanks!

[Kubik2000]

maybe this tool can be integrated https://vocalremover.org/

[NeonLightning]

tried to get this working on my 02 couldn't get either the release or git version to work.

My forked branch (https://github.com/xuancong84/pikaraoke) has implemented this function in both ways:

1. DNN-based vocal remover (from https://github.com/tsurumeso/vocal-remover): vocal removal quality is excellent on most voices; however, for certain synthetic voice (like Project DIVA), it does not always work very well
2. stereo track subtraction: does not work well whenever audio compression loss is high, might not work on non-standard stereo vocal channel encoding

My implemented solution is to run another dedicated process to extract and process audio and store into an .m4a file, and then launch vlc with --input-slave to play video from one file while playing audio from another file synchronously. I feel that is quite ad-hoc, but it can co-exist with pitch shifting. Is there a more elegant method?

[xuancong84]

tried to get this working on my 02 couldn't get either the release or git version to work.

What do you mean by 02? What is your OS version? It is a quite common knowledge that open-source packages do not run after a while because the dependent packages get updated and certain features get deprecated and removed. For example, recently Numpy has removed np.complex which causes a chain of dependencies to break. Even though we can use requirements.txt to lock package versions, however, when the root is too deep, even that fails to guarantee. Thus, sometimes it requires new users to have very high IQ to get things working.

[NeonLightning]

sorry i mean pi zero 2, and os is bullseye. and ya i figured that would likely be likely the problem