deezer / spleeter

Deezer source separation library including pretrained models.
https://research.deezer.com/projects/spleeter.html
MIT License
25.63k stars 2.81k forks source link

vocals file missing? wondering about language support. #7

Closed jameshe42 closed 4 years ago

jameshe42 commented 4 years ago

I tried using it on some japanese music, it did a pretty good job on the instrumental, but I didn't see any audio file for the vocals. I assume it's english only at the moment and I missed the language support or something in the readme.

alreadytaikeune commented 4 years ago

Hi @jameshe42, It should work seamlessly on any language and you should get vocal track. Can you provide us with the command you ran to better understand what's going on? Thanks!

jameshe42 commented 4 years ago

I just followed the instructions on the main page , but i replaced it with my file instead of audio example conda env create -f spleeter/conda/spleeter-cpu.yaml conda activate spleeter-cpu spleeter separate -i F:/test.mp3 -p spleeter:2stems -o output

test.zip I attached the mp3 i used

akx commented 4 years ago

Works fine for me (Python 3.7 on Mac, virtualenv), and I get both wavs and the Japanese sounds fine (as fine as I can tell, anyway ๐Ÿ˜…).

$ spleeter separate -i test.mp3 -o issue7
[...]
INFO:tensorflow:Loading audio b'test.mp3' from 0.0 to 600.0
INFO:tensorflow:File issue7/test/vocals.wav written
INFO:tensorflow:File issue7/test/accompaniment.wav written
$
awesomer commented 4 years ago

I am just a user like you, but I've noticed that different input files produce different subsets of output files, also dependent on which pretrained model you are using. In the worst case scenario, there is no output. I'm not sure if this is intended behavior, but an error message of some sort would go a long way. I've also had it create an output file of very short, empty output.

Faylixe commented 4 years ago

@jameshe42 Are you running Spleeter under Windows OS by any chance ? This could be a bug relative to path evaluation. Maybe you can try running the command again directly from the directory your input file is located in ?

somewhatlurker commented 4 years ago

I run Windows, and I've noticed some output files missing sometimes with input files in the current working directory (in the root of a python 3.7 virtualenv). Running spleeter again usually results in the files being created fine.
I've noticed Spleeter does come close to exhausting my virtual memory limit though, so perhaps that could be related.(?) I honestly have no clue, just thought I'd share that it seems non-deterministic.

Faylixe commented 4 years ago

@somewhatlurker Indeed the model is consuming memory heavily, and unfortunately we don't have any option now in order to prevent this (the process will simply get killed by the system OOM killer without any notice).

This could also be the reason why @jameshe42 you don't have any output, do you have any output saying the file has been written ? Or any killed message ?

dipaliso commented 4 years ago

Complete tensorflow/python noob here. It works great, but wondering how to train and use the newly trained set for Indian music. I checked out train.py but could not make any sense out of it. Any hints appreciated.

Faylixe commented 4 years ago

Complete tensorflow/python noob here. It works great, but wondering how to train and use the newly trained set for Indian music. I checked out train.py but could not make any sense out of it. Any hints appreciated.

Check the wiki : https://github.com/deezer/spleeter/wiki/2.-Getting-started#train-model

somewhatlurker commented 4 years ago

I should've been more specific actually. The process seems to never be killed (it continues and eventually uses less memory later as in normal operation), just doesn't create all output. (If there are memory spikes in these later stages that I never noticed due to low polling rate on monitoring then perhaps it is actually silently killed though)
I don't recall ever seeing an error message, but it'd be pretty easy to miss one among all the numpy deprecation warnings I'm getting. (these seem to be caused by tensorflow and tensorboard, not spleeter, but nonetheless they make the output a mess)

I don't understand the architecture of the program enough to really know, but it seems like different output files aren't fully processed in parallel. Is it possible for some allocations to fail without causing OOM and just render parts of the model unusable? (I use mwf if that makes any difference)

Faylixe commented 4 years ago

I should've been more specific actually. The process seems to never be killed (it continues and eventually uses less memory later as in normal operation), just doesn't create all output. (If there are memory spikes in these later stages that I never noticed due to low polling rate on monitoring then perhaps it is actually silently killed though)

Hard to say indeed, we already had reported case of process being killed by lacking of memory but not that way yet. This could be a Windows related issues, we would gladly like to have a example we can reproduce (all the steps performed, environment spec with version, etc) so we can manage to make a more reliable Windows version of it.

I don't recall ever seeing an error message, but it'd be pretty easy to miss one among all the numpy deprecation warnings I'm getting.

We need to fix this, verbose logging has been disabled but in a UNIX fashion.

I don't understand the architecture of the program enough to really know, but it seems like different output files aren't fully processed in parallel. Is it possible for some allocations to fail without causing OOM and just render parts of the model unusable? (I use mwf if that makes any difference)

Actually the separation is performed first in-memory using Tensorflow. Once the prediction is made, we then write separated output to file (this operation is multiprocessed in order to speed up). You are supposed to have a message indicating that a separated source has been written for each track. Could you give us your complete output log to see if there is any step of the separation process that does not work as expected ?

somewhatlurker commented 4 years ago

this is a log of stderr when the "other" track was not written, but all other tracks (from 4stems) were: https://gist.github.com/somewhatlurker/9d655df2dd2064a0985d5b91efaecb18 (I removed ~100 lines of what seemed to just be deprecation warnings from the end)
I don't see any messages indicating successful writes here, and stdout seems to be empty.

somewhatlurker commented 4 years ago

the three written files from the above all seem to be complete and normal

Faylixe commented 4 years ago

this is a log of stderr when the "other" track was not written, but all other tracks (from 4stems) were: https://gist.github.com/somewhatlurker/9d655df2dd2064a0985d5b91efaecb18 (I removed ~100 lines of what seemed to just be deprecation warnings from the end) I don't see any messages indicating successful writes here, and stdout seems to be empty.

Thanks, we will check that ASAP :)

jameshe42 commented 4 years ago

@jameshe42 Are you running Spleeter under Windows OS by any chance ? This could be a bug relative to path evaluation. Maybe you can try running the command again directly from the directory your input file is located in ?

I'm running it on an old windows PC, but I did get a lot of warnings.

c:\programdata\miniconda3\envs\spleeter-cpu\lib\site-packages\tensorflow\python\ framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synony m of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) c:\programdata\miniconda3\envs\spleeter-cpu\lib\site-packages\tensorflow\python\ framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synony m of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) c:\programdata\miniconda3\envs\spleeter-cpu\lib\site-packages\tensorflow\python\ framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synony m of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.

I can try running it on my mac to see if it produces something different

since you brought up path evaluation it was actually a bit difficult to find where the output file was supposed to go. it showed up somewhere I didn't expect. undert C:/users/user/output.

I also realized that the audio example only produced the vocals file and the other file is missing...

awesomer commented 4 years ago

I have tested about 15 different songs, and only 3 of them produced any output at all. From reading the above thread, this is not expected. My environment is a little unusual, I'm using spleeter-cpu on WSL (Ubuntu-for-Windows).

I get the "loading audio from 0.0 to 600.0" message and then it grinds for a few minutes consuming all CPU, and then just exits. The contents of "$?" are 0, indicating successful exit (or that the program doesn't set result codes). Sometimes only one of the models will produce any output for a given input file. Except for audio_example.mp3, which works as expected, I've yet to have the "[x]stems" model actually produce [x] output files. When there is output, I get 1 or 2 files at most, usually the vocal. I have tried FLACs, WAVs, and mp3s as input.

I have 24gb of RAM on this system, and there is no dmesg log of OOMkilling, so I doubt that it's memory concerns resulting in no output. Watching the process running in Resource Manager, it uses about 5gb of RAM.

Please let me know if there is any debug mode or log that might be helpful in understanding what is happening.

Faylixe commented 4 years ago

@jameshe42 Are you running Spleeter under Windows OS by any chance ? This could be a bug relative to path evaluation. Maybe you can try running the command again directly from the directory your input file is located in ?

I'm running it on an old windows PC, but I did get a lot of warnings.

Warning are produced by third party libraries like tensorflow or NumPy and does not reflect any potential issue here. An old windows PC though could be the reason. The model use a lot of memory and there is a high probability that you donโ€™t have enough while running separation.

since you brought up path evaluation it was actually a bit difficult to find where the output file was supposed to go. it showed up somewhere I didn't expect. undert C:/users/user/output.

I also realized that the audio example only produced the vocals file and the other file is missing...

Well the audio file will be written in the output directory provided in the command option -o.

jameshe42 commented 4 years ago

I'll test it on some other systems over the next few days and report back on findings. thanks, still a super cool software.

Faylixe commented 4 years ago

I will close this issue as it is mixing up lot of distinct scenario and / or environment. If you still have problem with execution under Windows I encourage you to recreate an issue with the new template filled properly and we will discuss the problem. We will also put an FAQ page we common issues and workaround to avoid duplication and issues overloading ๐Ÿ˜–