Preprocessing error while doing with own dataset

[sabby993]

Hello there. I was trying to do the preprocessing steps for my own dataset which is New Zealand English and I ran into the following error: 2020-10-06 00:13:48.426319: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1 [nltk_data] Downloading package averaged_perceptron_tagger to [nltk_data] /root/nltk_data... [nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip. [nltk_data] Downloading package cmudict to /root/nltk_data... [nltk_data] Unzipping corpora/cmudict.zip. 2020-10-06 00:13:53,547 (preprocess:368) INFO: Selected 'ljspeech' processor. 2020-10-06 00:13:54,793 (preprocess:407) INFO: Training items: 788 2020-10-06 00:13:54,793 (preprocess:408) INFO: Validation items: 42 [Preprocessing train]: 14% 110/788 [02:01<09:29, 1.19it/s]2020-10-06 00:15:56,611 (preprocess:290) WARNING: akl_nz_cw_0669 causes clipping. It is better to reconsider global gain scale value. [Preprocessing train]: 22% 170/788 [02:49<08:43, 1.18it/s]2020-10-06 00:16:45,013 (preprocess:290) WARNING: akl_nz_cw_0589 causes clipping. It is better to reconsider global gain scale value. [Preprocessing train]: 44% 349/788 [05:12<05:44, 1.27it/s]Traceback (most recent call last): File "/usr/local/bin/tensorflow-tts-preprocess", line 8, in sys.exit(preprocess()) File "/usr/local/lib/python3.6/dist-packages/tensorflow_tts/bin/preprocess.py", line 441, in preprocess for result, mel, energy, f0, features in train_map: File "/usr/lib/python3.6/multiprocessing/pool.py", line 347, in return (item for chunk in result for item in chunk) File "/usr/lib/python3.6/multiprocessing/pool.py", line 735, in next raise value RuntimeError: Error opening './nz_cw/wavs/\ufeffakl_nz_cw_0001.wav': System error. [Preprocessing train]: 44% 349/788 [05:12<06:33, 1.12it/s]

I modeled my dataset according to the LJ_Speech dataset and I am running the ljspeech processor to preprocess. But, I am running into this error with a file name which is not the correct filename (correct filename is akl_nz_cw_001.wav) and the preprocessing is done for the other 349 wav files and it gets stuck on the first file in the dataset (akl_nz_cw_001.wav) while the code did the preprocess in a random order on 349 other files.

Can you please let me know what to do. Thank you.

[dathudeptrai]

@sabby993 can you try to load that file by manual :)).

import numpy as np
np.load('./nz_cw/wavs/\ufeffakl_nz_cw_0001.wav')

[sabby993]

@dathudeptrai There is no file called ufeffakl_nz_cw_0001.wav in my dataset. The first file is akl_nz_cw_0001.wav and that is my confusion here because there is no file in my dataset with that name. I modeled everything in my dataset according to the LJspeech dataset and I have made the metadata.csv the same structure as LJSpeech has.

[dathudeptrai]

@sabby993 maybe that file exist in metadata.csv but not in wav folder :D

[NekiMedo]

Is the \ufeff part of that path meant to be a Unicode character? zero width no-break space Looks like someone is inserting that char.

[sabby993]

@dathudeptrai I have checked...there isn't a file name of that sort in the metadata.csv file

[sabby993]

I was able to get it working because my metadata.csv file had some unexpected spaces and after I removed them manually the preprocess worked.