search

CSTR-Edinburgh / merlin

This is now the official location of the Merlin project.
http://www.cstr.ed.ac.uk/projects/merlin/
Apache License 2.0
1.31k stars 441 forks source link

Error While Synthesizing audio for new Sentences #129

Closed uvaidya closed 7 years ago

uvaidya commented 7 years ago

I wanted to use trained model obtained by running run_demo.sh/run_full_voice.sh for the purpose of synthesizing audio for new sentences. I followed Issue #28 . Towards doing above, i followed the steps mentioned in "Steps to follow above" (in #28 ) but while running merlin_synthesis.sh, i get following error :- "2017-04-06 05:54:15,554 DEBUG param_generation: acoustic_decomposition for 1 files 2017-04-06 05:54:15,554 INFO param_generation: processing 1 of 1: /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/egs/slt_arctic/s1/experiments/slt_arctic_demo/test_synthesis/wav/own_01.cmp 2017-04-06 05:54:15,554 DEBUG param_generation: feature: mgc checking shape of features over here in generation ... (0, 180) 2017-04-06 05:54:15,554 DEBUG param_generation: starting MLParameterGeneration.generation checking shape of features over here in generation ...*** Traceback (most recent call last): File "/media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/src/run_merlin.py", line 1175, in main_function(cfg) File "/media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/src/run_merlin.py", line 943, in main_function generator.acoustic_decomposition(gen_file_list, cfg.cmp_dim, cfg.out_dimension_dict, cfg.file_extension_dict, var_file_dict, do_MLPG=cfg.do_MLPG, cfg=cfg) File "/media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/src/frontend/parameter_generation.py", line 161, in acoustic_decomposition gen_features = mlpg_algo.generation(current_features, var, out_dimension_dict[feature_name]/3) File "/media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/src/frontend/mlpg_fast.py", line 122, in generation var_frames[0, 1] = 100000000000; IndexError: index 0 is out of bounds for axis 0 with size 0 Lock freed " I am attaching logs generated while running merlin_synthesis.py command_line_logs.txt

I created a directory by the name of txt in path mentioned by ydwang above, and created a test file by the name of own_01.txt which has "Summer vacations has started." as a text in it.

Please help me with this issue, by letting me know what i am missing.

Thanks Utkarsh Vaidya

seblemaguer commented 7 years ago

have you check "/media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/egs/slt_arctic/s1/experiments/slt_arctic_demo/test_synthesis/wav/own_01.cmp"? is it empty ?

uvaidya commented 7 years ago

Hi Sebaistein,

Yes, own_01.cmp size is 0 bytes. It is empty.

uvaidya commented 7 years ago

" _2017-04-06 05:54:13,917 INFO labels : loaded /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/egs/slt_arctic/s1/experiments/slt_arctic_demo/test_synthesis/gen-lab/own_01.lab, 0 labels 2017-04-06 05:54:13,917 DEBUG labels : made label matrix of 0 frames x 425 labels WARNING: no silence found! 2017-04-06 05:54:13,917 DEBUG acoustic_norm: MinMaxNormalisation created for feature dimension of 425 2017-04-06 05:54:13,917 INFO acoustic_norm: Loaded min max values from the trained data for feature dimension of 425 2017-04-06 05:54:13,938 DEBUG labels : HTS-derived input feature dimension is 416 + 9 = 425 2017-04-06 05:54:13,938 INFO main : label dimension is 425 2017-04-06 05:54:13,938 INFO main : generating from DNN 2017-04-06 05:54:13,938 DEBUG dnn_generation: Starting dnn_generation 2017-04-06 05:54:15,463 INFO dnn_generation: generating 1 of 1: /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/egs/slt_arctic/s1/experiments/slt_arctic_demo/acoustic_model/data/nn_no_silence_lab_norm_425/own_01.lab 2017-04-06 05:54:15,552 DEBUG dnn_generation: saved to /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/egs/slt_arctic/s1/experiments/slt_arctic_demo/test_synthesis/wav/own01.cmp 2017-04-06 05:54:15,554 DEBUG main : denormalising generated output using method MVN "

own_01.lab created by using create_labels_from_text.sh is also empty.In the above log snippet info log is there showing that own_01.lab has zero labels.Is this from where problem got started ? While running run_demo.sh, all the labels files that are their for training/testing data have non zero label count in their .lab files.

seblemaguer commented 7 years ago

I think it is yes because if there is nothing in the .lab it can't go further. You should try to understand why the labels are empty

uvaidya commented 7 years ago

Thanks Sebaistein. I went ahead to understand why .lab is empy. Reason for the same was "_dl_audio@dl-powerbox:/media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/egs/slt_arctic/s1$ ./merlin_synthesis.sh Step 1: creating label files from text... creating a scheme file from text directory ****check utt text output {'own_01': 'Summer vacations have started.'} generating utts from scheme file converting festival utts to labels... own_01 /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/misc/scripts/frontend/festival_utt_to_lab/make_labels: line 32: gawk: command not found /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/misc/scripts/frontend/festival_utt_to_lab/make_labels: line 33: gawk: command not found own_02 /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/misc/scripts/frontend/festival_utt_to_lab/make_labels: line 32: gawk: command not found /media/dl_audio/hdd2/Utkarsh/tts_exp/merlin/misc/scripts/frontend/festival_utt_to_lab/make_labels: line 33: gawk: command not found normalizing label files for merlin... own01.lab " gawk was missing, it is required by frontend for processing, when i installed (sudo apt-get install gawk). .lab files were no more empy and accordingly Index error went away.

Audio is getting synthesized for new sentences, but synthesized audio quality is bit bad, reason for same can be, i trained model using run_demo.sh and dataset size for demo is less. Now i am running run_full_voice.sh to train model then will check synthesized audio quality for new sentence.

uvaidya commented 7 years ago

With gawk installed, issue got fixed for me. Closing this one here. Thanks

li-xx-5 commented 6 years ago

hi friend,i met the erro gawk:fatal:cannot open file 'experiments/slt_arctic_demo'/test_synthesis/prompt-lab/tmp for reading how to solve the problem, thanks

arif334 commented 5 years ago

@doctor-xiang It's been a while. Did you solve your problem? I am facing the same problem. In case you solved yours, please share the solution with us.

li-xx-5 commented 5 years ago

ok

发件人："arif334" notifications@github.com 发送日期：2019-01-08 17:08:19 收件人：CSTR-Edinburgh/merlin merlin@noreply.github.com 抄送人：doctor-xiang xiang.li@dlssa.com,Mention mention@noreply.github.com 主题：Re: [CSTR-Edinburgh/merlin] Error While Synthesizing audio for new Sentences (#129)

@doctor-xiang It's been a while. Did you solve your problem? I am facing the same problem. In case you solved yours, please share the solution with us.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.