Closed b3326023 closed 5 years ago
Hi, Could you tell which version of audioread, ffmpeg and dcase_util you're using please ? (the full conda list could be even better)
I have no problem downloading on my side, so I don't think it is a corrupted file (you should delete tmp folder between two attempts, sometimes it comes from that)
Here is my dcase environment module list:
astroid 2.2.5 pypi_0 pypi audioread 2.1.8 pypi_0 pypi blas 1.0 mkl certifi 2019.3.9 py36_0 conda-forge cffi 1.12.3 pypi_0 pypi chardet 3.0.4 pypi_0 pypi colorama 0.4.1 pypi_0 pypi cycler 0.10.0 pypi_0 pypi dcase-util 0.2.10 pypi_0 pypi decorator 4.4.0 pypi_0 pypi ffmpeg 1.4 pypi_0 pypi future 0.17.1 pypi_0 pypi h5py 2.9.0 py36h5e291fa_0 hdf5 1.10.4 h7ebc959_0 icc_rt 2019.0.0 h0cc432a_1 idna 2.8 pypi_0 pypi intel-openmp 2019.4 245 isort 4.3.20 pypi_0 pypi joblib 0.13.2 pypi_0 pypi kiwisolver 1.1.0 pypi_0 pypi lazy-object-proxy 1.4.1 pypi_0 pypi librosa 0.6.3 pypi_0 pypi llvmlite 0.29.0 pypi_0 pypi matplotlib 3.1.0 pypi_0 pypi mccabe 0.6.1 pypi_0 pypi mkl 2019.4 245 mkl_fft 1.0.12 py36h14836fe_0 mkl_random 1.0.2 py36h343c172_0 numba 0.44.0 pypi_0 pypi numpy 1.16.4 py36h19fb1c0_0 numpy-base 1.16.4 py36hc3f5095_0 pandas 0.24.2 py36ha925a31_0 pip 19.1.1 py36_0 pycparser 2.19 pypi_0 pypi pydot-ng 2.0.0 pypi_0 pypi pylint 2.3.1 pypi_0 pypi pyparsing 2.4.0 pypi_0 pypi pyreadline 2.1 py36_1 python 3.6.8 h9f7ef89_7 python-dateutil 2.8.0 py36_0 python-magic 0.4.15 pypi_0 pypi pytz 2019.1 py_0 pyyaml 5.1.1 pypi_0 pypi requests 2.22.0 pypi_0 pypi resampy 0.2.1 pypi_0 pypi scikit-learn 0.21.2 pypi_0 pypi scipy 1.2.1 py36h29ff71c_0 sed-eval 0.2.1 pypi_0 pypi setuptools 41.0.1 py36_0 six 1.12.0 py36_0 soundfile 0.10.2 pypi_0 pypi sqlite 3.28.0 he774522_0 tqdm 4.32.1 pypi_0 pypi typed-ast 1.4.0 pypi_0 pypi urllib3 1.25.3 pypi_0 pypi validators 0.13.0 pypi_0 pypi vc 14.1 h0510ff6_4 vs2015_runtime 14.15.26706 h3a45250_4 wheel 0.33.4 py36_0 wincertstore 0.2 py36h7fe50ca_0 wrapt 1.11.1 pypi_0 pypi youtube-dl 2019.6.8 pypi_0 pypi zlib 1.2.11 h62dcd97_3
I will try to delete the tmp folder! Thx!
turpaultn notifications@github.com 於 2019年6月13日 週四 下午10:57寫道:
Hi, Could you tell which version of audioread, ffmpeg and dcase_util you're using please ? (the full conda list could be even better)
I have no problem downloading on my side, so I don't think it is a corrupted file (you should delete tmp folder between two attempts, sometimes it comes from that)
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/turpaultn/DCASE2019_task4/issues/8?email_source=notifications&email_token=ADVKWCNC42YN7OFLBHF5WRTP2JN4BA5CNFSM4HXYO4S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXT67ZA#issuecomment-501739492, or mute the thread https://github.com/notifications/unsubscribe-auth/ADVKWCJOKVIZHBDXVES3ZUTP2JN4BANCNFSM4HXYO4SQ .
I just modified the download code slightly to print every downloading filename, so that I can see which file cause this error, and I got the following error:
(dcase2019) C:\Users\user\Google 雲端硬碟\NCKU_NETDB\SED\DCASE2019_task4\baseline>python download_data.py INFO - Download_data INFO -
Once database is downloaded, do not forget to check your missing_files
INFO - You can change N_JOBS and CHUNK_SIZE to increase the download with more processes. INFO - Validation data 0%|▏ | 1/517 [00:08<1:13:38, 8.56s/it] 2%|█▋ | 11/517 [00:08<50:35, 6.00s/it] INFO - YU5udL6UCigk_100.000_110.000.wav INFO - list index out of range WARNING: Unknown codec unknown WARNING: Unknown codec unknown WARNING: Unknown codec unknown WARNING: Unknown codec unknown 6%|████▊ | 31/517 [00:13<34:36, 4.27s/it]tmp/W7gn2todzbg.webm 8%|██████▍ | 41/517 [00:17<24:41, 3.11s/it]tmp/BiNbPvdPthY.webm tmp/LHY9L1MjQvQ.webm tmp/NyIzowSEuqQ.m4a tmp/ZhhGkqjCNsM.mp4 multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "C:\Users\user\Anaconda3\envs\dcase2019\lib\multiprocessing\pool.py", line 119, in worker result = (True, func(*args, *kwds)) File "C:\Users\user\Anaconda3\envs\dcase2019\lib\multiprocessing\pool.py", line 44, in mapstar return list(map(args)) File "C:\Users\user\Google 雲端硬碟\NCKU_NETDB\SED\DCASE2019_task4\baseline\download_data.py", line 82, in download_file start=float(segment_start), stop=float(segment_end)) File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\dcase_util\containers\audio.py", line 775, in load duration=duration File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\librosa\core\audio.py", line 119, in load with audioread.audio_open(os.path.realpath(path)) as input_file: File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\audioread__init__.py", line 116, in audio_open raise NoBackendError() audioread.exceptions.NoBackendError """
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "download_data.py", line 218, in
Hope this information can be helpful.
Yi-Wei Cheng b3326023@gap.kmu.edu.tw 於 2019年6月13日 週四 下午10:59寫道:
Here is my dcase environment module list:
Name Version Build Channel
astroid 2.2.5 pypi_0 pypi audioread 2.1.8 pypi_0 pypi blas 1.0 mkl certifi 2019.3.9 py36_0 conda-forge cffi 1.12.3 pypi_0 pypi chardet 3.0.4 pypi_0 pypi colorama 0.4.1 pypi_0 pypi cycler 0.10.0 pypi_0 pypi dcase-util 0.2.10 pypi_0 pypi decorator 4.4.0 pypi_0 pypi ffmpeg 1.4 pypi_0 pypi future 0.17.1 pypi_0 pypi h5py 2.9.0 py36h5e291fa_0 hdf5 1.10.4 h7ebc959_0 icc_rt 2019.0.0 h0cc432a_1 idna 2.8 pypi_0 pypi intel-openmp 2019.4 245 isort 4.3.20 pypi_0 pypi joblib 0.13.2 pypi_0 pypi kiwisolver 1.1.0 pypi_0 pypi lazy-object-proxy 1.4.1 pypi_0 pypi librosa 0.6.3 pypi_0 pypi llvmlite 0.29.0 pypi_0 pypi matplotlib 3.1.0 pypi_0 pypi mccabe 0.6.1 pypi_0 pypi mkl 2019.4 245 mkl_fft 1.0.12 py36h14836fe_0 mkl_random 1.0.2 py36h343c172_0 numba 0.44.0 pypi_0 pypi numpy 1.16.4 py36h19fb1c0_0 numpy-base 1.16.4 py36hc3f5095_0 pandas 0.24.2 py36ha925a31_0 pip 19.1.1 py36_0 pycparser 2.19 pypi_0 pypi pydot-ng 2.0.0 pypi_0 pypi pylint 2.3.1 pypi_0 pypi pyparsing 2.4.0 pypi_0 pypi pyreadline 2.1 py36_1 python 3.6.8 h9f7ef89_7 python-dateutil 2.8.0 py36_0 python-magic 0.4.15 pypi_0 pypi pytz 2019.1 py_0 pyyaml 5.1.1 pypi_0 pypi requests 2.22.0 pypi_0 pypi resampy 0.2.1 pypi_0 pypi scikit-learn 0.21.2 pypi_0 pypi scipy 1.2.1 py36h29ff71c_0 sed-eval 0.2.1 pypi_0 pypi setuptools 41.0.1 py36_0 six 1.12.0 py36_0 soundfile 0.10.2 pypi_0 pypi sqlite 3.28.0 he774522_0 tqdm 4.32.1 pypi_0 pypi typed-ast 1.4.0 pypi_0 pypi urllib3 1.25.3 pypi_0 pypi validators 0.13.0 pypi_0 pypi vc 14.1 h0510ff6_4 vs2015_runtime 14.15.26706 h3a45250_4 wheel 0.33.4 py36_0 wincertstore 0.2 py36h7fe50ca_0 wrapt 1.11.1 pypi_0 pypi youtube-dl 2019.6.8 pypi_0 pypi zlib 1.2.11 h62dcd97_3
I will try to delete the tmp folder! Thx!
turpaultn notifications@github.com 於 2019年6月13日 週四 下午10:57寫道:
Hi, Could you tell which version of audioread, ffmpeg and dcase_util you're using please ? (the full conda list could be even better)
I have no problem downloading on my side, so I don't think it is a corrupted file (you should delete tmp folder between two attempts, sometimes it comes from that)
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/turpaultn/DCASE2019_task4/issues/8?email_source=notifications&email_token=ADVKWCNC42YN7OFLBHF5WRTP2JN4BA5CNFSM4HXYO4S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXT67ZA#issuecomment-501739492, or mute the thread https://github.com/notifications/unsubscribe-auth/ADVKWCJOKVIZHBDXVES3ZUTP2JN4BANCNFSM4HXYO4SQ .
Maybe I found the possible problem! This file ZhhGkqjCNsM.mp4 caused the error, I cannot load it with librosa.load(). But this is strange, because I can open this file via my normal video player(potplayer), so this file is not corrupted. So I still do not know how to solve this problem...
Yi-Wei Cheng b3326023@gap.kmu.edu.tw 於 2019年6月13日 週四 下午11:22寫道:
I just modified the download code slightly to print every downloading filename, so that I can see which file cause this error, and I got the following error:
(dcase2019) C:\Users\user\Google 雲端硬碟\NCKU_NETDB\SED\DCASE2019_task4\baseline>python download_data.py INFO - Download_data INFO -
Once database is downloaded, do not forget to check your missing_files
INFO - You can change N_JOBS and CHUNK_SIZE to increase the download with more processes. INFO - Validation data 0%|▏ | 1/517 [00:08<1:13:38, 8.56s/it] 2%|█▋ | 11/517 [00:08<50:35, 6.00s/it] INFO - YU5udL6UCigk_100.000_110.000.wav INFO - list index out of range WARNING: Unknown codec unknown WARNING: Unknown codec unknown WARNING: Unknown codec unknown WARNING: Unknown codec unknown 6%|████▊ | 31/517 [00:13<34:36, 4.27s/it]tmp/W7gn2todzbg.webm 8%|██████▍ | 41/517 [00:17<24:41, 3.11s/it]tmp/BiNbPvdPthY.webm tmp/LHY9L1MjQvQ.webm tmp/NyIzowSEuqQ.m4a tmp/ZhhGkqjCNsM.mp4 multiprocessing.pool.RemoteTraceback: """ Traceback (most recent call last): File "C:\Users\user\Anaconda3\envs\dcase2019\lib\multiprocessing\pool.py", line 119, in worker result = (True, func(*args, *kwds)) File "C:\Users\user\Anaconda3\envs\dcase2019\lib\multiprocessing\pool.py", line 44, in mapstar return list(map(args)) File "C:\Users\user\Google 雲端硬碟\NCKU_NETDB\SED\DCASE2019_task4\baseline\download_data.py", line 82, in download_file start=float(segment_start), stop=float(segment_end)) File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\dcase_util\containers\audio.py", line 775, in load duration=duration File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\librosa\core\audio.py", line 119, in load with audioread.audio_open(os.path.realpath(path)) as input_file: File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\audioread__init__.py", line 116, in audio_open raise NoBackendError() audioread.exceptions.NoBackendError """
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "download_data.py", line 218, in
base_dir_missing_files=base_missing_files_folder) File "download_data.py", line 158, in download for val in tqdm(p.imap_unordered(download_file_alias, filenames, chunk_size), total=len(filenames)): File "C:\Users\user\Anaconda3\envs\dcase2019\lib\site-packages\tqdm_tqdm.py", line 1005, in iter for obj in iterable: File "C:\Users\user\Anaconda3\envs\dcase2019\lib\multiprocessing\pool.py", line 347, in return (item for chunk in result for item in chunk) File "C:\Users\user\Anaconda3\envs\dcase2019\lib\multiprocessing\pool.py", line 735, in next raise value audioread.exceptions.NoBackendError Hope this information can be helpful.
Yi-Wei Cheng b3326023@gap.kmu.edu.tw 於 2019年6月13日 週四 下午10:59寫道:
Here is my dcase environment module list:
Name Version Build Channel
astroid 2.2.5 pypi_0 pypi audioread 2.1.8 pypi_0 pypi blas 1.0 mkl certifi 2019.3.9 py36_0 conda-forge cffi 1.12.3 pypi_0 pypi chardet 3.0.4 pypi_0 pypi colorama 0.4.1 pypi_0 pypi cycler 0.10.0 pypi_0 pypi dcase-util 0.2.10 pypi_0 pypi decorator 4.4.0 pypi_0 pypi ffmpeg 1.4 pypi_0 pypi future 0.17.1 pypi_0 pypi h5py 2.9.0 py36h5e291fa_0 hdf5 1.10.4 h7ebc959_0 icc_rt 2019.0.0 h0cc432a_1 idna 2.8 pypi_0 pypi intel-openmp 2019.4 245 isort 4.3.20 pypi_0 pypi joblib 0.13.2 pypi_0 pypi kiwisolver 1.1.0 pypi_0 pypi lazy-object-proxy 1.4.1 pypi_0 pypi librosa 0.6.3 pypi_0 pypi llvmlite 0.29.0 pypi_0 pypi matplotlib 3.1.0 pypi_0 pypi mccabe 0.6.1 pypi_0 pypi mkl 2019.4 245 mkl_fft 1.0.12 py36h14836fe_0 mkl_random 1.0.2 py36h343c172_0 numba 0.44.0 pypi_0 pypi numpy 1.16.4 py36h19fb1c0_0 numpy-base 1.16.4 py36hc3f5095_0 pandas 0.24.2 py36ha925a31_0 pip 19.1.1 py36_0 pycparser 2.19 pypi_0 pypi pydot-ng 2.0.0 pypi_0 pypi pylint 2.3.1 pypi_0 pypi pyparsing 2.4.0 pypi_0 pypi pyreadline 2.1 py36_1 python 3.6.8 h9f7ef89_7 python-dateutil 2.8.0 py36_0 python-magic 0.4.15 pypi_0 pypi pytz 2019.1 py_0 pyyaml 5.1.1 pypi_0 pypi requests 2.22.0 pypi_0 pypi resampy 0.2.1 pypi_0 pypi scikit-learn 0.21.2 pypi_0 pypi scipy 1.2.1 py36h29ff71c_0 sed-eval 0.2.1 pypi_0 pypi setuptools 41.0.1 py36_0 six 1.12.0 py36_0 soundfile 0.10.2 pypi_0 pypi sqlite 3.28.0 he774522_0 tqdm 4.32.1 pypi_0 pypi typed-ast 1.4.0 pypi_0 pypi urllib3 1.25.3 pypi_0 pypi validators 0.13.0 pypi_0 pypi vc 14.1 h0510ff6_4 vs2015_runtime 14.15.26706 h3a45250_4 wheel 0.33.4 py36_0 wincertstore 0.2 py36h7fe50ca_0 wrapt 1.11.1 pypi_0 pypi youtube-dl 2019.6.8 pypi_0 pypi zlib 1.2.11 h62dcd97_3
I will try to delete the tmp folder! Thx!
turpaultn notifications@github.com 於 2019年6月13日 週四 下午10:57寫道:
Hi, Could you tell which version of audioread, ffmpeg and dcase_util you're using please ? (the full conda list could be even better)
I have no problem downloading on my side, so I don't think it is a corrupted file (you should delete tmp folder between two attempts, sometimes it comes from that)
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/turpaultn/DCASE2019_task4/issues/8?email_source=notifications&email_token=ADVKWCNC42YN7OFLBHF5WRTP2JN4BA5CNFSM4HXYO4S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXT67ZA#issuecomment-501739492, or mute the thread https://github.com/notifications/unsubscribe-auth/ADVKWCJOKVIZHBDXVES3ZUTP2JN4BANCNFSM4HXYO4SQ .
Well, you're environment is not the same setup the same way it is for me, so it could be the problem. In the conda_create_environment.sh file, it uses conda-forge instead of pypi for ffmpeg for example, could you try this ?
conda create -y -n dcase2019 python=3.6
source activate dcase2019
conda install -y pandas h5py scipy
conda install -y pytorch torchvision cudatoolkit=9.0 -c pytorch # for gpu install (or cpu in MAC)
# conda install pytorch-cpu torchvision-cpu -c pytorch (cpu linux)
conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge
conda install -y ffmpeg -c conda-forge
pip install dcase_util
pip install sed-eval
So at least we can discard some problems, and check the only difference we have.
PS: I can load the mp4 file with librosa here, that's why I'm asking for that.
When I carefully following the conda_create_environment.sh to reinstall the environment on my windows 10, I found that I got an issue at conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge, the issue is shown below:
(dcase2019) C:\Users\user\Google 雲端硬碟\NCKU_NETDB\SED\DCASE2019_task4\baseline\tmp>conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge Collecting package metadata: done Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
Current channels:
To search for alternate channels that may provide the conda package you're looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
The error is: PackagesNotFoundError: The following packages are not available from current channels:
It's so strange...😥
turpaultn notifications@github.com 於 2019年6月14日 週五 上午12:52寫道:
Well, you're environment is not the same setup the same way it is for me, so it could be the problem. In the conda_create_environment.sh file, it uses conda-forge instead of pypi for ffmpeg for example, could you try this ?
conda create -y -n dcase2019 python=3.6 source activate dcase2019 conda install -y pandas h5py scipy conda install -y pytorch torchvision cudatoolkit=9.0 -c pytorch # for gpu install (or cpu in MAC)
conda install pytorch-cpu torchvision-cpu -c pytorch (cpu linux)
conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge conda install -y ffmpeg -c conda-forge
pip install dcase_util pip install sed-eval
So at least we can discard some problems, and check the only difference we have.
PS: I can load the mp4 file with librosa here, that's why I'm asking for that.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/turpaultn/DCASE2019_task4/issues/8?email_source=notifications&email_token=ADVKWCLYH5ZXN4KOIZZFC5TP2J3LHA5CNFSM4HXYO4S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXUKNSQ#issuecomment-501786314, or mute the thread https://github.com/notifications/unsubscribe-auth/ADVKWCLBVAXJU4ATCE2NSWTP2J3LHANCNFSM4HXYO4SQ .
Thank you! I use my Ubuntu system, and carefully reinstall the environment, finally I can finished the download program with no problem!
Yi-Wei Cheng b3326023@gap.kmu.edu.tw 於 2019年6月14日 週五 上午10:08寫道:
When I carefully following the conda_create_environment.sh to reinstall the environment on my windows 10, I found that I got an issue at conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge, the issue is shown below:
(dcase2019) C:\Users\user\Google 雲端硬碟\NCKU_NETDB\SED\DCASE2019_task4\baseline\tmp>conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge Collecting package metadata: done Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- pysoundfile -> libsndfile
- pysoundfile -> libgcc-ng[version='>=7.3.0']
Current channels:
- https://conda.anaconda.org/conda-forge/win-64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/win-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/free/win-64
- https://repo.anaconda.com/pkgs/free/noarch
- https://repo.anaconda.com/pkgs/r/win-64
- https://repo.anaconda.com/pkgs/r/noarch
- https://repo.anaconda.com/pkgs/msys2/win-64
- https://repo.anaconda.com/pkgs/msys2/noarch
To search for alternate channels that may provide the conda package you're looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
The error is: PackagesNotFoundError: The following packages are not available from current channels:
It's so strange...😥
turpaultn notifications@github.com 於 2019年6月14日 週五 上午12:52寫道:
Well, you're environment is not the same setup the same way it is for me, so it could be the problem. In the conda_create_environment.sh file, it uses conda-forge instead of pypi for ffmpeg for example, could you try this ?
conda create -y -n dcase2019 python=3.6 source activate dcase2019 conda install -y pandas h5py scipy conda install -y pytorch torchvision cudatoolkit=9.0 -c pytorch # for gpu install (or cpu in MAC)
conda install pytorch-cpu torchvision-cpu -c pytorch (cpu linux)
conda install -y pysoundfile librosa youtube-dl tqdm -c conda-forge conda install -y ffmpeg -c conda-forge
pip install dcase_util pip install sed-eval
So at least we can discard some problems, and check the only difference we have.
PS: I can load the mp4 file with librosa here, that's why I'm asking for that.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/turpaultn/DCASE2019_task4/issues/8?email_source=notifications&email_token=ADVKWCLYH5ZXN4KOIZZFC5TP2J3LHA5CNFSM4HXYO4S2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXUKNSQ#issuecomment-501786314, or mute the thread https://github.com/notifications/unsubscribe-auth/ADVKWCLBVAXJU4ATCE2NSWTP2J3LHANCNFSM4HXYO4SQ .
Solved, Thanks
I've set the environment as mentioned in README, install all the dependencies, but when I run the download_data.py, and error,
audioread.NoBackendError
is occurred.I have tried on windows 10 and Ubuntu 16.04.5 LTS, but I got the same error. I have install ffmpeg, and I can use librosa to load my own .wav file with no error, so I think the error may caused by file corrupted.
Hope someone help me, please!