New Dataset

[jdbermeol]

Hi, So everything worked perfectly with your pre-process Vctk. Now I want to test with Nancy data set. I'm using the script you suggested, but I have 2 questions:

1. When I run the script I get 2 files on the norm_info folder: label_norm_HTS_420.dat and norm_info_mgc_lf0_vuv_bap_63_MVN.dat. Based on the shape the correct file is norm_info_mgc_lf0_vuv_bap_63_MVN.dat, but I want to be sure.

2. In order to combine both datasets, should I have to run the script for each speaker and them combine somehow the norms file, or should I put all data in one folder and process it?

Thanks.

[jdbermeol]

Hi again, Also I check .npz files and they don't contain several files that are in your Vctk data:

• audio_norminfo
• code2char
• text_features
• code2phone
• text_norminfo Do you know how to build it?

[jayavanth]

Uncomment the required arrays here

[karandwivedi42]

@jdbermeol @jayavanth did you find an answer to the original question:

In order to combine both datasets, should I have to run the script for each speaker and them combine somehow the norms file, or should I put all data in one folder and process it?

[jdbermeol]

@karandwivedi42 No, I don't know the answer yet. Also, the script used to work but now I get this error:

Traceback (most recent call last): File "/home/ubuntu/loop/preprocessing/latest_features/merlin/src/run_merlin.py", line 1175, in main_function(cfg) File "/home/ubuntu/loop/preprocessing/latest_features/merlin/src/run_merlin.py", line 693, in main_function acoustic_worker.prepare_nn_data(in_file_list_dict, nn_cmp_file_list, cfg.in_dimension_dict, cfg.out_dimension_dict) File "/home/ubuntu/loop/preprocessing/latest_features/merlin/src/frontend/acoustic_base.py", line 122, in prepare_nn_data self.prepare_data(in_file_list_dict, out_file_list, in_dimension_dict, out_dimension_dict) File "/home/ubuntu/loop/preprocessing/latest_features/merlin/src/frontend/acoustic_composition.py", line 126, in prepare_data features, frame_number = io_funcs.load_binary_file_frame(in_file_name, in_feature_dim) File "/home/ubuntu/loop/preprocessing/latest_features/merlin/src/io_funcs/binary_io.py", line 64, in load_binary_file_frame fid_lab = open(file_name, 'rb') IOError: [Errno 2] No such file or directory: '/home/ubuntu/loop/preprocessing/latest_features/merlin/egs/build_your_own_voice/s1/experiments/my_new_voice/acoustic_model/data/mgc/*.mgc' + echo 'All successfull!! Your demo voice is ready :)' All successfull!! Your demo voice is ready :) Feature extraction complete! Traceback (most recent call last): File "extract_features.py", line 1411, in save_numpy_features() File "extract_features.py", line 853, in save_numpy_features shutil.copy2(audio_norm_source, audio_norm_dest) File "/home/ubuntu/miniconda2/envs/loop/lib/python2.7/shutil.py", line 130, in copy2 copyfile(src, dst) File "/home/ubuntu/miniconda2/envs/loop/lib/python2.7/shutil.py", line 82, in copyfile with open(src, 'rb') as fsrc: IOError: [Errno 2] No such file or directory: '/home/ubuntu/loop/preprocessing/latest_features/final_acoustic_data/norm_info_mgc_lf0_vuv_bap_63_MVN.dat'

[jdbermeol]

@adampolyak, @ytaigman. Hi, So I have been able to run the extract_features script for a speaker on the VCTK dataset. However, each run is going to create a norm_infor folder. So I go back to my original question. Is there a way to combine the output of each norm_info folder. Or should I create a big folder with all samples and run the script using that folder?

[adampolyak]

1. norm_info_mgc_lf0_vuv_bap_63_MVN.dat is indeed the correct norm file.

2. Both ways are valid. The norm file contains mean and std of the dataset - see the generation code. You can merge the statistics or just run the script on the merged folder.

[jackchinor]

@jdbermeol I want to train my own data set, but I found it too complicate to do it. Should I firstly run the install_tts.py? and then run the extract_features script? when I run the install_tts.py , it occurred an error as follow: Traceback (most recent call last): File "install_tts.py", line 174, in pe(untar_cmd) File "install_tts.py", line 114, in pe for line in execute(cmd, shell=shell): File "install_tts.py", line 107, in execute raise subprocess.CalledProcessError(return_code, cmd) subprocess.CalledProcessError: Command '['tar', 'xzf', '/tmp/kastner/speech_synthesis/speech_tools-2.4-release.tar.gz']' returned non-zero exit status 2

I don't know how to fix it . Could you help me with it? really appreciate

[dengbingfeng]

@jackchinor in the install_tts.py, you can find you need to download kk_all_deps.tar.gz or install some tools first

[jackchinor]

@dengbingfeng I created the kk_all_deps.tar.gz, but run the install_tts.py file, it doesn't work, an error occured

[jackchinor]

@jdbermeol finally ,I generate .npz files , but some of the .npy files are not contained, just the same with you. 1.audio_norminfo 2.code2char 3.text_features 4.code2phone 5.text_norminfo Can you tell me how to build them?Really appreciate...

[jdbermeol]

@jackchinor You will need to uncomment this line: https://gist.github.com/kastnerkyle/cc0ac48d34860c5bb3f9112f4d9a0300#file-extract_feats-py-L1034

You will see that the missing matrices are also commented, so you will need to uncommented too. The only one that needs to remain commented is the code2speaker.

[jackchinor]

@jdbermeol I see, thank you so much.

[ankitmishra262]

I am trying to run extract_feats.py on the complete VCTK dataset by following the advice from this comment above, to put all the wav files and text in one big directory and run the script on them.

Before I get to the part of save_dict saving all features, I'm getting the following error.

Feature extraction complete!
Traceback (most recent call last):
    File "extract_feats.py", line 1440 in <module>
        save_numpy_features()
    File "extract_feats.py", line 1020 in save_numpy_features
        assert phonemes[0] == 'pau'
IndexError: tuple index out of range

My best guess is that the phonemes tuple is not being created properly. Any suggestions or am I making some common mistake?

[jdbermeol]

hi @ankitmishra262, great question, same happens to me, I could not solve it, I have to restrict my self to the subsample of speaker Facebook team use in the paper.

[jaxlinksync]

I'm having a problem in running this https://gist.github.com/kastnerkyle/001a58a58d090658ee5350cb6129f857 to run extract_feats.py. Any ideas where I can download the kk_all_deps.tar.gz?

[jaxlinksync]

@jackchinor how did you solve this?

[jaxlinksync]

I created my own kk_all_deps.tar.gz but now encountered a new error.

Makefile:102: recipe for target 'ch_lab' failed
config/rules/targets.mak:55: recipe for target 'main' failed
Traceback (most recent call last):
  File "install_tts.py", line 183, in <module>
    pe(make_cmd)
  File "install_tts.py", line 114, in pe
    for line in execute(cmd, shell=shell):
  File "install_tts.py", line 107, in execute
    raise subprocess.CalledProcessError(return_code, cmd)
subprocess.CalledProcessError: Command '['make', '-j', '4']' returned non-zero exit status 2

[ankitmishra262]

Trying running make externally, that is, from the terminal for the libraries that are failing in this way. This generally happens when make fails due to some dependency issues. Subprocess wasn't printing those outputs for me.

[jaxlinksync]

Hi @ankitmishra262, what ubuntu are you using and the GPU? mine is ubuntu 16, and cuda 8.0

[ankitmishra262]

I'm running ubuntu 14.04 with cuda 8.0 @jaxlinksync

[jaxlinksync]

Hi @jdbermeol,

how did you solve this? https://github.com/facebookresearch/loop/issues/11#issuecomment-328837495 I now came up with the same error.

[jaxlinksync]

Hi @ankitmishra262 Everything went smoothly until I run the extract_feats.py. I have this error

Traceback (most recent call last):
  File "extract_feats.py", line 1406, in <module>
    save_numpy_features()
  File "extract_feats.py", line 853, in save_numpy_features
    shutil.copy2(audio_norm_source, audio_norm_dest)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/home/jax/latest_features/final_acoustic_data/norm_info_mgc_lf0_vuv_bap_63_MVN.dat'

[jackchinor]

@jaxlinksync I solved the problem by manually make and install the package tool.

[hepower]

@jaxlinksync , may I know how you fix the below make issue when running install_tts.py?

_Makefile:102: recipe for target 'ch_lab' failed config/rules/targets.mak:55: recipe for target 'main' failed Traceback (most recent call last): File "install_tts.py", line 183, in pe(make_cmd) File "install_tts.py", line 114, in pe for line in execute(cmd, shell=shell): File "install_tts.py", line 107, in execute raise subprocess.CalledProcessError(returncode, cmd) subprocess.CalledProcessError: Command '['make', '-j', '4']' returned non-zero exit status 2

when I try to execute 'make' in the dir directly , some error happened:

_Making in directory ./siod ... Making in directory ./rxp ... Making in directory ./audio ... Making in directory ./utils ... Making in directory ./base_class ... Making in directory base_class/string ... Making in directory base_class/inst_tmpl ... Making in directory ./ling_class ... Making in directory ./speech_class ... Making in directory ./sigpr ... Making in directory sigpr/pda ... Making in directory ./stats ... Making in directory stats/wagon ... Making in directory stats/kalman_filter ... Making in directory ./grammar ... Making in directory grammar/scfg ... Making in directory grammar/wfst ... Making in directory grammar/ngram ... Making in directory ./intonation ... Making in directory intonation/tilt ... Making in directory ./lib ... Making in directory ./main ... gcc -O3 -Wall -o ch_lab ch_lab_main.o -L../lib -lestools -L../lib -lestbase -L../lib -leststring -lcurses -ldl -lncurses -lm -lstdc++ /usr/bin/ld: cannot find -lcurses /usr/bin/ld: cannot find -lncurses collect2: error: ld returned 1 exit status Makefile:102: recipe for target 'ch_lab' failed make[1]: [ch_lab] Error 1 config/rules/targets.mak:55: recipe for target 'main' failed make: [main] Error 2_

[hepower]

solved by: Install ncurses lib sudo apt-get install libncurses5-dev libncursesw5-dev Ubuntu 16

[hepower]

@jdbermeol , @ankitmishra262 , Have you successfully extract the features from vctk data and train a speakable model? I use extrct_feats.py to deal with the same data set got from download_data.sh , but the shape of feature['audio_feature'] is different with the ones downloaded.

when I used the feature files extracted locally to train a model , it could not speak, only output silences. Do you have the same problem?

[jdbermeol]

@hepower I was able to make it work, but it was not easy. Beside of doing manually all the setup of libraries. I had to use the same voice actors as they use in the download_data.sh, when I try to do it with the complete VCTK dataset it did not work.

[hepower]

@jdbermeol are you training the speakable model with the feature files downloaded with download_data.sh or you create them yourself locally with extract_feats.py?

When I extract the same actors locally, I notice that the shape of the features are not same with the downloaded one.

About your issue. I suggest you do not deal with the complete VCTK dataset, you may change some actors to see if it works.

[pfriesch]

I have the exact same issue as in https://github.com/facebookresearch/loop/issues/11#issuecomment-337762040

somehow the norm_info does not get generated, but I can't find out why. I can't even find where it would be generated.

@jaxlinksync did you get it to work?

@jackchinor what do you mean by make the package tools manually. Which tools do you mean SPTK,WORLD or merlin?

(I am on mac btw)

[ghostcow]

@jdbermeol how did you solve this problem: https://github.com/facebookresearch/loop/issues/11#issuecomment-328837495 ??

[jdbermeol]

@ghostcow I left this project a few months, and my memory is really bad, something important I remember is that Facebook team was not using all the speakers on the Dataset. When I restrict myself to the subsample of speaker Facebook team use in the paper I was able to fix some of the issues. Not sure if exactly that one, but it was helpfull.

[niravpatel2008]

@jaxlinksync how did you fixed below issue?

Traceback (most recent call last):
  File "extract_feats.py", line 1406, in <module>
    save_numpy_features()
  File "extract_feats.py", line 853, in save_numpy_features
    shutil.copy2(audio_norm_source, audio_norm_dest)
  File "/usr/lib/python2.7/shutil.py", line 130, in copy2
    copyfile(src, dst)
  File "/usr/lib/python2.7/shutil.py", line 82, in copyfile
    with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: '/home/jax/latest_features/final_acoustic_data/norm_info_mgc_lf0_vuv_bap_63_MVN.dat'

[haoqi]

@jaxlinksync how did you solve the following error you metioned above, I also tried run make on terminal, but it didn't work

Makefile:102: recipe for target 'ch_lab' failed config/rules/targets.mak:55: recipe for target 'main' failed Traceback (most recent call last): File "install_tts.py", line 183, in pe(make_cmd) File "install_tts.py", line 114, in pe for line in execute(cmd, shell=shell): File "install_tts.py", line 107, in execute raise subprocess.CalledProcessError(return_code, cmd) subprocess.CalledProcessError: Command '['make', '-j', '4']' returned non-zero exit status 2

[wanshun123]

I had the subprocess.CalledProcessError: Command '['make', '-j', '4']' returned non-zero exit status 2 error and it seems that installing ncurses with sudo apt-get install libncurses5-dev libncursesw5-dev (per above) solved it. Now in running install_tts.py I get the following error instead:

Traceback (most recent call last):
  File "install_tts.py", line 362, in <module>
    pe(configure_cmd)
  File "install_tts.py", line 116, in pe
    for line in execute(cmd, shell=shell):
  File "install_tts.py", line 107, in execute
    raise subprocess.CalledProcessError(return_code, cmd)
subprocess.CalledProcessError: Command '['./configure', '--with-fest-search-path=/home/paperspace/voiceloop/test/speech_synthesis/festival/examples', '--with-sptk-search-path=/home/paperspace/voiceloop/test/speech_synthesis/SPTK-3.9/bin', '--with-hts-search-path=/home/paperspace/voiceloop/test/speech_synthesis/htk/HTKTools', '--with-hts-engine-search-path=/home/paperspace/voiceloop/test/speech_synthesis/hts_engine_API-1.10/bin']' returned non-zero exit status 1

[wanshun123]

Command had to be with sudo (sudo python2 install_tts.py)