Closed jjjanicehuang closed 3 months ago
I thiink it's a "WARNING" so the presence of the message doens't impact the preprocessing of the dataset. You can simply ignore that. No need to change backend from the default or update the torchaudio version
0%| | 0/24107 [00:00<?, ?it/s]--2024-05-09 21:43:21-- https://raw.githubusercontent.com/lingjzhu/CharsiuG2P/main/dicts/nl.tsv Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2024-05-09 21:43:21 ERROR 404: Not Found.
then how about this? the process just stuck with all the errors and warnings that I mentioned above.
For lauguage code, pls follow the format used in charisug2p https://github.com/lingjzhu/CharsiuG2P/tree/main/dicts
Dataset size: 24107 0%| | 0/24107 [00:00<?, ?it/s]--2024-05-10 08:23:29-- https://raw.githubusercontent.com/lingjzhu/CharsiuG2P/main/dicts/dut.tsv Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 787737 (769K) [text/plain] Saving to: ‘dut.tsv’
dut.tsv 100%[==========================================================>] 769.27K --.-KB/s in 0.02s
2024-05-10 08:23:29 (43.4 MB/s) - ‘dut.tsv’ saved [787737/787737]
/home/ict/miipher_env/lib/python3.10/site-packages/torchaudio/backend/sox_io_backend.py:416: UserWarning: File-like object support in sox_io backend is deprecated, and will be removed in v2.1. See https://github.com/pytorch/audio/issues/2950 for the detail.Please migrate to the new dispatcher, or use soundfile backend. warnings.warn(_deprecation_message) formats: mp3 can't encode MPEG audio (layer I, II or III) to 16-bit 0%| | 0/24107 [00:07<?, ?it/s]
Thanks, I think the dict problem solved, but it still stopped out of the backend problem.
Can you show me the error message?, I think what you have here is a WARNING message
Can you show me the config file you use
Thanks, the issue resolved
Can you share how you resolve this issue with the community please?
Sure, regarding the warning issue about the backend, I replaced the sox_io with soundfile at env/lib/python3.10/site-packages/torchaudio/functional/functional.py and add one line of torchaudio.set_audio_backend("soundfile") at the dataset processing script.
Regarding the warning of "mp3 can't encode...", I still didn't get why this was happening because all my original dataset was in format of .flac, anyways I coverted all the flac audios into wav files at the end.
But as you observed, they are just warnings, so the actual fatal issue that I faced is about the suspension of the preprocessing, which was caused by my "break" command in the preprocessor.py while debugging. The preprocess.py finally works after I commented that out.
Thanks for all your help so far :)
Thank you for the detailed explanation 😃 Closing this issue
Hi Wataru, I have a question regarding the dependencies, when I run preprocess.py, the terminal shows errors of the following:
/home/ict/miipher_env/lib/python3.10/site-packages/torchaudio/backend/sox_io_backend.py:416: UserWarning: File-like object support in sox_io backend is deprecated, and will be removed in v2.1. See https://github.com/pytorch/audio/issues/2950 for the detail.Please migrate to the new dispatcher, or use soundfile backend. warnings.warn(_deprecation_message) formats: mp3 can't encode MPEG audio (layer I, II or III) to 16-bit
I tried to set the backend to soundfile by setting: torchaudio.set_audio_backend("soundfile"), but it's not working. I was suggested to adopt backend to "ffmpeg" instead, which may need to upgrade torchaudio first. But if I upgrade torchaudio, then there's an error showing that it's incompatible with miipher 0.1.0:
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lightning-vocoders 0.1.0 requires torch~=2.0.1, but you have torch 2.3.0 which is incompatible. lightning-vocoders 0.1.0 requires torchaudio~=2.0.2, but you have torchaudio 2.3.0 which is incompatible. miipher 0.1.0 requires torchaudio~=2.0.2, but you have torchaudio 2.3.0 which is incompatible.
Appreciate thoughts or solutions if any?