farihachaiti / DoKaraokeAI_Spleeter_Android_Implementation

AI Implementation of Karaoke Track Creation for Android Devices
https://github.com/farihachaiti/DoKaraokeAI_Spleeter_Android_Implementation
10 stars 3 forks source link

Does the Vocal Removal Work #1

Closed oluwabajio closed 2 years ago

oluwabajio commented 2 years ago

I tested it on my phone but it wasnt working as expected. Were you able to get spleeter to work effectively.

farihachaiti commented 2 years ago

Yes. You have to update the ffmpeg command as per your requirement. This is the base of android implementation of spleeter only. You have to do the minor tweaks yourself.

oluwabajio commented 2 years ago

Yes, you are correct, i made a lot of tweaks and was able to get it up, but it crash at the ffmpeg command. I would try again today and get back to you. Thanks.

iop

oluwabajio commented 2 years ago

Ok, i tried again. This is the error i am having.

java.lang.UnsatisfiedLinkError: dlopen failed: library "libmobileffmpeg_abidetect.so" not found

oluwabajio commented 2 years ago

Ok finally, i was able to get it to work on my device. Thanks for your help.

Also i noticed the result karaoke was of very low quality. Is there any way the quality can be increased

oluwabajio commented 2 years ago

i went through your code and later discovered that you are actually not using spleeter to extract the karaoke. You are basically using ffmpeg to perform phase cancellation on a stereo track.

That explains the reason for the low quality audio.

Nice work by the way.

farihachaiti commented 2 years ago

@oluwabajio Check correctly. I am only using ml tensorflow model here which enables the ffmpeg to split the vocal. This is the best possible result till now as an android solution of spleeter. You are free to extend it and make it better. You can also use the denoising processes to make the audio quality better which I commented out in this implementation. Good luck.

oluwabajio commented 2 years ago

Ok, thanks for your response. From your code, i noticed that after you used tflite model to split the audio, you were supposed to encode the audio data to an output format, but instead of executing the encoding command, you instead executed the command for phase cancellation to generate the output audio. phase cancellation = String command = "-i " + audioInputFile + " -af pan=\"stereo|c0=c0|c1=-1*c1\" -ac 2 " + audioOutputFile + " -y"; Encoding = String ffmpegCommand = "-f f32le -ac 2 -ar 44100 -i " + pipe1 + " -b:a 128k -ar 44100 -strict -2 " + outputPath_1 + " -y"; command2

Also checking through vvasanth project, This was the command he used. val ffmpegCommand = "-f f32le -ac 2 -ar 44100 -i " + pipe1 + " -b:a 128k -ar 44100 -strict -2 " + outputPath_1 + " -y"

![Uploading command3.png…]()

farihachaiti commented 2 years ago

@oluwabajio Oh yeah. The encoding command is the ffmpegCommand. You will see there I had tried multiple commands. I was just playing with the various commands to check the ffmpeg running or not. This project was in python basically. You are free to work with this android solution. Please make the ffmpegcommand work, cut my commentouts and do play as you need. 👍

oluwabajio commented 2 years ago

ok cool. Thanks for your feedback. I would try the encoding format and get back to you.

farihachaiti commented 2 years ago

Hey @oluwabajio This file is the working version of Spleeter, I forgot to update last time. Btw this throws out of memory error in my device with incomplete music output along with some noise, so please check on your device if works correctly. I guess if it could work completely without throwing any memory error the output would have been perfect. Moreover, I suggest you play with the ffmpeg commands and the denoising process to get the desired output if it still troubles. https://github.com/farihachaiti/DoKaraokeAI/blob/main/app/src/main/java/com/example/dokaraokeAI/SplitVocal.java https://github.com/farihachaiti/DoKaraokeAI/blob/main/app/src/main/java/com/example/dokaraokeAI/AudioFeatureExtraction.java

oluwabajio commented 2 years ago

ok cool. i would check it out

oluwabajio commented 2 years ago

ShowProgress.java Class is missing.

farihachaiti commented 2 years ago

@oluwabajio Yeah I intentionally didnt put it in. Its a progressdialog only. Please comment out these lines linked to showprogress.java. Run the application without the progressdialog.

oluwabajio commented 2 years ago

@farihachaiti i checked it out. it works fine, but the output result was really not good. I want to find out, is there ay other way we could increase the quality of the audio apart from the denoising method.

farihachaiti commented 2 years ago

@oluwabajio sure. Go ahead. 👍 If you find anything please open another issue. I am closing this one.