Freeze when loading mp4 muli-stem file

[f90]

I am using the musdb package and convert the mp4 files containing multiple audio sources to wave files, as shown here:

https://github.com/f90/Wave-U-Net/blob/master/Datasets.py#L132

But randomly during conversion (so with potentially any file), conversion just freezes forever. After interrupting the process I can read the following error:

Traceback (most recent call last): File "/opt/local/pycharm/helpers/pydev/pydevd.py", line 1668, in main() File "/opt/local/pycharm/helpers/pydev/pydevd.py", line 1662, in main globals = debugger.run(setup['file'], None, None, is_module) File "/opt/local/pycharm/helpers/pydev/pydevd.py", line 1072, in run pydev_imports.execfile(file, globals, locals) # execute the script File "/mnt/daten/PycharmProjects/Wave-U-Net/Training.py", line 326, in @ex.automain File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/sacred/experiment.py", line 137, in automain self.run_commandline() File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/sacred/experiment.py", line 260, in run_commandline return self.run(cmd_name, config_updates, named_configs, {}, args) File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/sacred/experiment.py", line 209, in run run() File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/sacred/run.py", line 221, in call self.result = self.main_function(args) File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/sacred/config/captured_function.py", line 46, in captured_function result = wrapped(args, *kwargs) File "/mnt/daten/PycharmProjects/Wave-U-Net/Training.py", line 348, in dsd_100_experiment dsd_train, dsd_test = Datasets.getMUSDB(model_config["musdb_path"]) # List of (mix, acc, bass, drums, other, vocal) tuples File "/mnt/daten/PycharmProjects/Wave-U-Net/Datasets.py", line 149, in getMUSDB vocal_audio = track.targets["vocals"].audio File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/musdb/audio_classes.py", line 113, in audio audio = source.audio File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/musdb/audio_classes.py", line 47, in audio filename=self.path, stem_id=self.stem_id File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/stempeg/read.py", line 91, in read_stems FFinfo = FFMPEGInfo(filename) File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/stempeg/read.py", line 19, in init self.json_info = read_info(self.filename) File "/home/daniel/tf-env-waveunet/local/lib/python2.7/site-packages/stempeg/read.py", line 55, in read_info out = sp.check_output(cmd) File "/usr/lib/python2.7/subprocess.py", line 567, in check_output process = Popen(stdout=PIPE, popenargs, *kwargs) File "/usr/lib/python2.7/subprocess.py", line 711, in init errread, errwrite) File "/usr/lib/python2.7/subprocess.py", line 1319, in _execute_child data = _eintr_retry_call(os.read, errpipe_read, 1048576) File "/usr/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call return func(args) KeyboardInterrupt

Process finished with exit code 1

It seems that the ffmpeg/ffprobe process that identifies the stems within the mp4 file never returns, or returns empty output, or sth of that sort, so that the stempeg library waits forever for a response at sp.check_output. It doesnt look like there is a timeout for waiting for the ffmpeg output either. Plus ffmpeg is called with -v error, maybe that is suppressing errors that we should react to?

Any idea of how to fix this?

[f90]

I just verified: It definitely hangs at

https://github.com/faroit/stempeg/blob/master/stempeg/read.py#L55

One might think this is because we wait for the ffmpeg process to terminate, and for some reason it doesn't. But I can't find any active ffmpeg process while it is hanging through "ps -C ffmpeg" either.

[f90]

ffprobe is on this version

ffprobe version 2.8.15-0ubuntu0.16.04.1 Copyright (c) 2007-2018 the FFmpeg developers built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609 configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv WARNING: library configuration mismatch avcodec configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv --enable-version3 --disable-doc --disable-programs --disable-avdevice --disable-avfilter --disable-avformat --disable-avresample --disable-postproc --disable-swscale --enable-libopencore_amrnb --enable-libopencore_amrwb --enable-libvo_aacenc --enable-libvo_amrwbenc libavutil 54. 31.100 / 54. 31.100 libavcodec 56. 60.100 / 56. 60.100 libavformat 56. 40.101 / 56. 40.101 libavdevice 56. 4.100 / 56. 4.100 libavfilter 5. 40.101 / 5. 40.101 libavresample 2. 1. 0 / 2. 1. 0 libswscale 3. 1.101 / 3. 1.101 libswresample 1. 2.101 / 1. 2.101 libpostproc 53. 3.100 / 53. 3.100

[faroit]

thanks for the report. can you check if the ffprobe command works from your bash with the files being affected?

[f90]

Yes the command issued by the stempeg library seems to work when I execute it in the console. Also, I am processing the same list of audio files every time, and every time it broke for a different file I think. It definitely did not consistently fail for any file.

I digged deeper and found that it hangs within the check_output function at the Popen command in the subprocess library. I found there was a bug with this years ago, but has been fixed since. At least supposedly. But it might be worth looking at issues deep within Python that could trigger this.

If you have the time, you might want to check out my Wave-u-net repo and run it together with MUSDB, it will start building the dataset immediately, and then you can see if you run into the same issue.

[faroit]

didn't had time to run your code but I had a another look at your error log from above. I noticed you run the code

1. from pycharm
2. using sacred

PyCharm is probably okay as long as you don't run it in debug mode, but I have no idea how pycharm works. Concerning sacred. This could be a potential problem. Also I found this issue in the sacred issue tracker. Have you tried without sacred?

[faroit]

@f90 long time no see... can this be closed?

[f90]

Yes this can be closed! Couldn't reproduce it anymore after i made a code revamp on my end...