About dependencies

[jjjanicehuang]

Hi Wataru, I have a question regarding the dependencies, when I run preprocess.py, the terminal shows errors of the following:

/home/ict/miipher_env/lib/python3.10/site-packages/torchaudio/backend/sox_io_backend.py:416: UserWarning: File-like object support in sox_io backend is deprecated, and will be removed in v2.1. See https://github.com/pytorch/audio/issues/2950 for the detail.Please migrate to the new dispatcher, or use soundfile backend. warnings.warn(_deprecation_message) formats: mp3 can't encode MPEG audio (layer I, II or III) to 16-bit

I tried to set the backend to soundfile by setting: torchaudio.set_audio_backend("soundfile"), but it's not working. I was suggested to adopt backend to "ffmpeg" instead, which may need to upgrade torchaudio first. But if I upgrade torchaudio, then there's an error showing that it's incompatible with miipher 0.1.0:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts. lightning-vocoders 0.1.0 requires torch~=2.0.1, but you have torch 2.3.0 which is incompatible. lightning-vocoders 0.1.0 requires torchaudio~=2.0.2, but you have torchaudio 2.3.0 which is incompatible. miipher 0.1.0 requires torchaudio~=2.0.2, but you have torchaudio 2.3.0 which is incompatible.

Appreciate thoughts or solutions if any?

[Wataru-Nakata]

I thiink it's a "WARNING" so the presence of the message doens't impact the preprocessing of the dataset. You can simply ignore that. No need to change backend from the default or update the torchaudio version

[jjjanicehuang]

0%| | 0/24107 [00:00<?, ?it/s]--2024-05-09 21:43:21-- https://raw.githubusercontent.com/lingjzhu/CharsiuG2P/main/dicts/nl.tsv Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.108.133, 185.199.109.133, 185.199.110.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.108.133|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2024-05-09 21:43:21 ERROR 404: Not Found.

then how about this? the process just stuck with all the errors and warnings that I mentioned above.

[Wataru-Nakata]

For lauguage code, pls follow the format used in charisug2p https://github.com/lingjzhu/CharsiuG2P/tree/main/dicts

[jjjanicehuang]

Dataset size: 24107 0%| | 0/24107 [00:00<?, ?it/s]--2024-05-10 08:23:29-- https://raw.githubusercontent.com/lingjzhu/CharsiuG2P/main/dicts/dut.tsv Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.108.133, 185.199.109.133, ... Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 787737 (769K) [text/plain] Saving to: ‘dut.tsv’

dut.tsv 100%[==========================================================>] 769.27K --.-KB/s in 0.02s

2024-05-10 08:23:29 (43.4 MB/s) - ‘dut.tsv’ saved [787737/787737]

/home/ict/miipher_env/lib/python3.10/site-packages/torchaudio/backend/sox_io_backend.py:416: UserWarning: File-like object support in sox_io backend is deprecated, and will be removed in v2.1. See https://github.com/pytorch/audio/issues/2950 for the detail.Please migrate to the new dispatcher, or use soundfile backend. warnings.warn(_deprecation_message) formats: mp3 can't encode MPEG audio (layer I, II or III) to 16-bit 0%| | 0/24107 [00:07<?, ?it/s]

Thanks, I think the dict problem solved, but it still stopped out of the backend problem.

[Wataru-Nakata]

Can you show me the error message?, I think what you have here is a WARNING message

[jjjanicehuang]

[Wataru-Nakata]

Can you show me the config file you use

[jjjanicehuang]

Thanks, the issue resolved

[Wataru-Nakata]

Can you share how you resolve this issue with the community please?

[jjjanicehuang]

Sure, regarding the warning issue about the backend, I replaced the sox_io with soundfile at env/lib/python3.10/site-packages/torchaudio/functional/functional.py and add one line of torchaudio.set_audio_backend("soundfile") at the dataset processing script.

Regarding the warning of "mp3 can't encode...", I still didn't get why this was happening because all my original dataset was in format of .flac, anyways I coverted all the flac audios into wav files at the end.

But as you observed, they are just warnings, so the actual fatal issue that I faced is about the suspension of the preprocessing, which was caused by my "break" command in the preprocessor.py while debugging. The preprocess.py finally works after I commented that out.

Thanks for all your help so far :)

[Wataru-Nakata]

Thank you for the detailed explanation 😃 Closing this issue