HMM Voice Building

[ahoenemann]

Hi everyone,

I try to build a new voice with the VoiceImport tool. I could extract all features but now I have a problem with the Data Preparation for the training process (HMMVoiceMakeData). I don't know if it is a problem with the path or something.

console output:

---

Opening GUI....... Loading database layout: Reading config file /home/voice/database.config Reading external binaries config file /home/marytts/lib/external/externalBinaries.config Loading basename list from file /home/voice/basenames.lst Found 366 files in basename list Loading features from file /home/voice/mary/features.txt Running: cd /home/voice/hts/data make mgc

make: *\ No rule to make target `mgc'. Stop. java.lang.Exception: The component HMMVoiceMakeData produced the following exception: at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:297) Caused by: java.lang.RuntimeException: computation failed on file [/home/voice/]! Command line was: [cd /home/voice/hts/data make mgc ]. at marytts.util.io.General.launchBatchProc(General.java:609) at marytts.tools.voiceimport.HMMVoiceMakeData.compute(HMMVoiceMakeData.java:184) at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:294)

---

There is no mgc in the data folder...

[psibre]

Did you run the HMMVoiceDataPreparation, HMMVoiceConfigure, and HMMVoiceFeatureSelection steps successfully? What happens when you cd /home/voice/hts/data && make mgc manually?

[ahoenemann]

Yes, HMMVoiceDataPreparation, HMMVoiceConfigure, and HMMVoiceFeatureSelection were successful.

If I do it manually it is the same problem: make: *\ No rule to make target `mgc'. Stop.

[ahoenemann]

If I only set makeCMPMARY and makeSTR how it is written in the tutorial I have the same problem with make str-mary

make str-mary make: *\ No rule to make target `str-mary'. Stop.

---

HMMVoiceMakeData.makeCMPMARY 1 HMMVoiceMakeData.makeLABELMARY 0 HMMVoiceMakeData.makeLF0 0 HMMVoiceMakeData.makeLIST 0 HMMVoiceMakeData.makeMGC 0 HMMVoiceMakeData.makeQUESTIONSMARY 0 HMMVoiceMakeData.makeSCP 0 HMMVoiceMakeData.makeSTR 1

---

[seblemaguer]

Can you paste the makefile in "hts/data" between triple reverse quote ?

[ahoenemann]

Hi

thanks, I will try it but I started again the whole process and now I have different problems and it seems that my lab files aren't right. So, I'm very confuse because I have different lab directories (ehmm/htk/main) and I do not know what need what. I think, I should only have one lab directory, right? The last lab output is from the HTKLabeler and it has this format:

---

# 0.225 125 0.27 125 ? 0.42 125 a~ 0.46 125 g 0.56 125 a: 0.69 125 Z 0.715 125 @ 0.81 125 m 1.0 125 a~ 1.09 125 ? 1.14 125 I 1.22 125 m 1.31 125 R 1.38 125 e: 1.47 125 l 1.62 125 E: 1.77 125 Z 1.96 125 a~ 2.07 125 R 2.15 125 @ 2.16 125 2.285 125 k 2.355 125 a 2.455 125 n 2.48 125 d 2.535 125 a 2.625 125 s 2.715 125 p 2.78 125 O 2.81 125 6 2.905 125 t 2.975 125 R 3.005 125 a 3.1 125 @ 3.115 125 t 3.325 125 pf 3.375 125 l 3.39 125 u: 3.42 125 E 3.54 125 k 3.585 125 @ 3.745 125 n 4.255 125 _

---

Is this the right format?

Because now I got this error if I start PhoneFeatureFileWriter:

---

Featurefile writer started. Number of units : 19260 java.lang.Exception: The component PhoneFeatureFileWriter produced the following exception: at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:297) Caused by: java.lang.IllegalArgumentException: No int value known for string [edge] at marytts.util.string.IntStringTranslator.get(IntStringTranslator.java:90) at marytts.features.FeatureDefinition.getFeatureIndex(FeatureDefinition.java:928) at marytts.features.FeatureDefinition.createEdgeFeatureVector(FeatureDefinition.java:1416) at marytts.tools.voiceimport.PhoneFeatureFileWriter.writeUnitFeaturesTo(PhoneFeatureFileWriter.java:152) at marytts.tools.voiceimport.PhoneFeatureFileWriter.compute(PhoneFeatureFileWriter.java:125) at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:294)

---

Maybe someone can help me? I tried so many....

[psibre]

Those files need to be in Xwaves format, with one line per phone and three fields per line: end time, an unused number, and the phone label. The LabelPauseDeleter should handle this conversion for you.

[ahoenemann]

Ok! Yes, this lab files are in the phonelab direktoriy, right? So, I don't know where the problem is. What does that mean 'No int value known for string [edge]' in the error message (s. my previous comment)? Do you have an idea? Sorry for the annoyances but I need the voice for an experiment... !

[ahoenemann]

This error happens if I start the HMMVoiceConfigure. Maybe I have a wrong version of HTS/HTK? Because I have no HMGenS on my computer but HSGen...

****************************************************************************
checking for swab... /usr/local/SPTK/bin//swab
checking for sox... /usr/bin/sox
checking for HMGenS... :
configure: error: Can't find HMGenS of HTS
java.lang.Exception: The component HMMVoiceConfigure produced the following exception: 
    at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:297)
Caused by: java.lang.RuntimeException: Configure computation failed on file [/home/ahoenemann/pvoice/]!
Command line was: [cd /home/ahoenemann/pvoice/hts
/home/ahoenemann/pvoice/hts/configure --with-tcl-search-path=/usr/bin --with-sptk-search-path=/usr/local/SPTK/bin/ --with-hts-search-path=/home/ahoenemann/petra_voice/hts --with-hts-engine-search-path=/usr/bin --with-sox-search-path=/usr/bin SPEAKER=petra DATASET=/home/ahoenemann/pvoice/hts/data LOWERF0=110 UPPERF0=280 VER=1 QNUM=001 FRAMELEN=400 FRAMESHIFT=80 WINDOWTYPE=1 NORMALIZE=1 FFTLEN=512 FREQWARP=0.42 GAMMA=0 MGCORDER=34 STRORDER=5 STRFILTERNAME=filters/mix_excitation_5filters_99taps_16Kz.txt LNGAIN=1 SAMPFREQ=16000 NSTATE=5 NITER=5].
    at marytts.util.io.General.launchBatchProc(General.java:609)
    at marytts.tools.voiceimport.HMMVoiceConfigure.compute(HMMVoiceConfigure.java:425)
    at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:294)
*************************************************

[psibre]

Maybe I have a wrong version of HTS/HTK?

I don't know, which version do you have?

[ahoenemann]

HTS-2.2_for_HTK-3.4.1.tar.bz2

[seblemaguer]

Are you sure you have patched HTK ? HTK has already HSGen. HMGenS is added by the HTS patch

[ahoenemann]

No, I'm not sure. How can I check it? Or how can I do it? I have no idea of htk or hts patch

[seblemaguer]

Do you remember how did you install HTS? Can you go to this directory: $MARY_BASE/lib/external Do you see a directory named "sw" ? If yes can you indicate the architecture ? If not, when you execute the command "bash ./check_install_external_programs.sh" when you are in the directory $MARY_BASE/lib/external, what is it indicated ?

[psibre]

You can apply the patch manually if you're in your HTK source directory, like so:

$ patch -p1 < HTS-2.2_for_HTK-3.4.1.patch

Note that you must have HDecode unpacked there as well, otherwise the patch will fail.

[ahoenemann]

No, I can't see the 'sw' directory.

I got this output after execute bash ./check_install_external_programs.sh

check_programs.sh: check the neccessary programs (and versions) to run the HTS-HTK demo for MARY 4.0 beta Usage: ./check_install_external_programs.sh [options:-check|-install][paths (if not in PATH): path-to-awk|path-to-perl|path-to-tclsh|path-to-bc|path-to-sox|path-to-htk|path-to-htsengine|path-to-sptk] options: -check : just check programs in PATH or in the paths provided by the user -install: check programs in PATH or in the paths provided by the user and try to install the missing ones

[ahoenemann]

Ok, I tried to install HDecode and I got the following error:

ranlib HTKLiblv.a
make[1]: Leaving directory `/home/ahoenemann/htk/HTKLib'
(cd HTKLVRec && make all) \
      || case "" in *k*) fail=yes;; *) exit 1;; esac;
make[1]: Entering directory `/home/ahoenemann/htk/HTKLVRec'
make[1]: *** No rule to make target `HDecode.c', needed by `HDecode.orig.o'.  Stop.
make[1]: Leaving directory `/home/ahoenemann/htk/HTKLVRec'
make: *** [hdecode] Error 1
**********************************************************

and I tried the patch -p1 < HTS-2.2_for_HTK-3.4.1.patch
and I git this error;

ahoenemann@ubuntu:~/htk$  patch -p1 < HTS-2.2_for_HTK-3.4.1.patch
can't find file to patch at input line 4
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff -pbBEwurNa --ignore-matching-lines='.*vc_id.*' HTK-3.4.1/HTKLVRec/HDecode.c HTS-2.2_for_HTK-3.4.1/HTKLVRec/HDecode.c
|--- HTK-3.4.1/HTKLVRec/HDecode.c   2009-03-13 03:55:15.000000000 +0900
|+++ HTS-2.2_for_HTK-3.4.1/HTKLVRec/HDecode.c   2011-02-10 17:23:06.000000000 +0900
--------------------------
File to patch: 

[seblemaguer]

Hello,

I think you have corrupted your htk installation. Let's proceed manually :

1. Check if your version of htk and hdecode archives is 3.4.1
2. delete your htk directory (/home/ahoenemann/htk/)
3. unpack the htk archive into your home directory
4. unpack the hdecode archive into your home directory
5. rename the "htk-XX" (where X stands for anything) to "hts"
6. go to ~/hts
7. copy the file HTS-2.2_for_HTK-3.4.1.patch to the current directory
8. apply the patch using this command "patch -p1 < HTS-2.2_for_HTK-3.4.1.patch"

ahoenemann writes:

Ok, I tried to install HDecode and I got the following error:

ranlib HTKLiblv.a make[1]: Leaving directory /home/ahoenemann/htk/HTKLib' (cd HTKLVRec && make all) \ || case "" in k) fail=yes;; ) exit 1;; esac; make[1]: Entering directory/home/ahoenemann/htk/HTKLVRec' make[1]: ** No rule to make target HDecode.c', needed byHDecode.orig.o'. Stop. make[1]: Leaving directory `/home/ahoenemann/htk/HTKLVRec' make: *\ [hdecode] Error 1

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

and I tried the patch -p1 < HTS-2.2_for_HTK-3.4.1.patch and I git this error;

ahoenemann@ubuntu:~/htk$ patch -p1 < HTS-2.2_for_HTK-3.4.1.patch can't find file to patch at input line 4 Perhaps you used the wrong -p or --strip option?

The text leading up to this was:

|diff -pbBEwurNa --ignore-matching-lines='.vc_id.' HTK-3.4.1/HTKLVRec/HDecode.c HTS-2.2_for_HTK-3.4.1/HTKLVRec/HDecode.c |--- HTK-3.4.1/HTKLVRec/HDecode.c 2009-03-13 03:55:15.000000000 +0900

|+++ HTS-2.2_for_HTK-3.4.1/HTKLVRec/HDecode.c 2011-02-10 17:23:06.000000000 +0900

File to patch:

— Reply to this email directly or view it on GitHub.*

Dr. Sébastien Le Maguer Postdoctorate researcher

Saarland University Campus C7.4 - room 2.03 D-66123 Saarbrücken Germany

phone : +49-681-302-70030 Mail: slemaguer@coli.uni-saarland.de website : http://sites.google.com/site/seblemaguer

[ahoenemann]

Ok. This worked. I will see now how it works with the voiceimport tool. I will start all again.

[ahoenemann]

Ok. Now at the point were I was for a few weeks. If I execute HMMVoiceMakeData I got the error: '''''''''''''''''''''''''''''''''''''''''''''''''' Saving extra features for training HMMs to /home/ahoenemann/Petra/mary/hmmFeatures.txt Running: cd /home/ahoenemann/Petra/hts/data make str-mary

make: *\ No rule to make target `str-mary'. Stop. java.lang.Exception: The component HMMVoiceMakeData produced the following exception: at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:297) Caused by: java.lang.RuntimeException: computation failed on file [/home/ahoenemann/Petra/]! Command line was: [cd /home/ahoenemann/Petra/hts/data make str-mary ]. at marytts.util.io.General.launchBatchProc(General.java:609) at marytts.tools.voiceimport.HMMVoiceMakeData.compute(HMMVoiceMakeData.java:192) at marytts.tools.voiceimport.DatabaseImportMain$8.run(DatabaseImportMain.java:294)

---

Because there is no str-mary in this directory and also not in another directory?

[psibre]

Could you please paste (in triple backticks!) the output of

$ cd /home/ahoenemann/Petra/hts/data
$ make -pn

[ahoenemann]

The problem ist that there is no makefile in the directory!? Only makefile.in. If I executed make -pn there are many information about my system (it seems so!!) and information such as (I have no idea what triple backticks means??):

# Not a target:
.sym:
#  Implicit rule search has not been done.
#  Modification time never checked.
#  File has not been updated.

# Not a target:
.f:
#  Implicit rule search has not been done.
#  Modification time never checked.
#  File has not been updated.
#  commands to execute (built-in):
    $(LINK.f) $^ $(LOADLIBES) $(LDLIBS) -o $@

# Not a target:
.f.o:
#  Implicit rule search has not been done.
#  Modification time never checked.
#  File has not been updated.
#  commands to execute (built-in):
    $(COMPILE.f) $(OUTPUT_OPTION) $<

# files hash-table stats:
# Load=69/1024=7%, Rehash=0, Collisions=276/1520=18%
# VPATH Search Paths

# No `vpath' search paths.

# No general (`VPATH' variable) search path.

# # of strings in strcache: 0
# # of strcache buffers: 0
# strcache size: total = 0 / max = 0 / min = 4096 / avg = 0
# strcache free: total = 0 / max = 0 / min = 4096 / avg = 0

# Finished Make data base on Mon Dec 15 10:06:04 2014

[ahoenemann]

Oh sorry. I don't know why the previous comment has such big letters.

[psibre]

Oh sorry. I don't know why the previous comment has such big letters.

Because of the missing triple backticks. Fixed that for you. See https://guides.github.com/features/mastering-markdown/ for help.

[ahoenemann]

So, what do you think about the makefile? I can't build anything. I think you don't need the ouput of make -pn?

[psibre]

Are you sure that all of the preceding steps explained in the wiki have completed without problems?

[ahoenemann]

yes it should be but I will try again...

[ahoenemann]

Ok it seems that I have no chance. I install alll programs, I checked it with you script check_install_external_programs.sh. I have only 19 files for test. I did all steps from the wiki but only one doesn't work: make mgc lf0 str-mary cmp-mary list scp and alos with HMMVoiceMakeData. There is the same error.

[psibre]

I have only 19 files for test.

Wait a minute... are you saying that you want to train the voice on only 19 utterances? Why not use the full data?

[ahoenemann]

Because the EHMMLabeler need much time and I have a better overview If there is an error I thought. I know Its not necessary to start the EHHMLabeler again but because I do not know where the problem is, I start often the tool at the beginning to be sure that all is new. I know it is not enough to build a good voice but at first I want to know if the voiceimport tool and my environment works with my files. I can now create a unit selection voice but still not the HMM voice.

[seblemaguer]

Hello,

I find really weird that you not have any makefile in /home/ahoenemann/Petra/hts/data if the configure stage succeed. Can you go the directory /home/ahoenemann/Petra/hts/ and run the configure manually and indicate what happened ?

[ahoenemann]

This happens:
... checking for vstat... /usr/bin/vstat checking for step... /usr/bin/step checking for sopr... /usr/bin/sopr checking for vopr... : configure: error: Can't find vopr of SPTK ... I install the sptk 3.4.1 manually but also with your script. But it seems the sptk path is wrong?
... Programs status (detailed information above): The following paths should be in the PATH variable awk: /usr/bin perl: /usr/bin bc: /usr/bin The following paths are used when running HMMVoiceConfigure tclsh: /usr/bin sox: /usr/bin hts/htk: /usr/local/bin hts_engine: /usr/bin sptk: /home/ahoenemann/marytts/lib/external/bin This path is used when running the EHMMlabeler ehmm: /home/ahoenemann/marytts/lib/external/bin

List of paths in: /home/ahoenemann/marytts/lib/external/externalBinaries.config ...

[psibre]

Because the EHMMLabeler need much time

But you already have manual labels for all 400 of your utterances...?

[seblemaguer]

no the sptk path is good (it manages to find vstat and step) but something crashed during the compilation of sptk. Go to the sptk directory and try to compile again using make.

ahoenemann writes:

This happens:

... checking for vstat... /usr/bin/vstat checking for step... /usr/bin/step checking for sopr... /usr/bin/sopr checking for vopr... : configure: error: Can't find vopr of SPTK ... I install the sptk 3.4.1 manually but also with your script. But it seems the sptk path is wrong?

... Programs status (detailed information above): The following paths should be in the PATH variable awk: /usr/bin perl: /usr/bin bc: /usr/bin The following paths are used when running HMMVoiceConfigure tclsh: /usr/bin sox: /usr/bin hts/htk: /usr/local/bin hts_engine: /usr/bin sptk: /home/ahoenemann/marytts/lib/external/bin This path is used when running the EHMMlabeler ehmm: /home/ahoenemann/marytts/lib/external/bin

List of paths in: /home/ahoenemann/marytts/lib/external/externalBinaries.config ...

— Reply to this email directly or view it on GitHub.*

Dr. Sébastien Le Maguer Postdoctorate researcher

Saarland University Campus C7.4 - room 2.03 D-66123 Saarbrücken Germany

phone : +49-681-302-70030 Mail: slemaguer@coli.uni-saarland.de website : http://sites.google.com/site/seblemaguer

[ahoenemann]

I do not need the automatic labeling? I thougth because our files has the wrong format? So, what steps I have to do with our labeling?

[ahoenemann]

Yes I did and it look good. It is installed at /usr/local/SPTK.

[psibre]

I do not need the automatic labeling? I thougth because our files has the wrong format? So, what steps I have to do with our labeling?

EHMMLabeler will never produce results that are as good as Donata's manual labels. Just convert them with a perl one-liner or something.

[psibre]

sptk: /home/ahoenemann/marytts/lib/external/bin

That's your problem right there.

[ahoenemann]

So, the path is wrong?

[ahoenemann]

I have no idea of perl...

[psibre]

So, the path is wrong?

Yes, judging from your earlier comment

[psibre]

I have no idea of perl...

Then use python, or R, or praat, or whatever you want.

[ahoenemann]

Thanks for your patience but I'm really confuse about this now...

[seblemaguer]

ok let's try to be more organisated. You are facing 2 problems

1. getting label files. To solve this one, just take the manual label files and transform them using any kind of text manipulation languages. For this point, you have to choose a language and learn it. If you are going to work on TTS later it's a mandatory step....
2. your environment is in an unstable stage as you have tried many ways to install everything. This is confirmed by the fact that the checking tool find some tools in /usr/bin and not  /home/ahoenemann/marytts/lib/external/bin as expected. I'm almost sure that even if we solve the SPTK problem another one will pop up because of this environement issue.

Therefore, I suggest that you will have to install everything manually but you have to be really carefull of the pathes used in the configuration. For example, for SPTK, we see that it's expected to find the binaries into "/home/ahoenemann/marytts/lib/external/bin". Consequently, you have to follow these steps top install :

1. got the SPTK source directory
2. execute the command "./configure --prefix=/home/ahoenemann/marytts/lib/external/" (this command set the path were every sptk will bin installed. binaries are, by convention, placed in the "$prefix/bin" directory 
3. execute the command "make" 
4. execute the commande "make install"

Apply the same procedure for the other tools!

go to your SPTK source directory

[ahoenemann]

ok

[ahoenemann]

Sorry, but I have one question. I've started the HMMVoiceMakeDate from the VoiceImpoirtTool and now after 7 hour I have an error and the process was stopped. How can I start it at a particular point of the process or have I start it again and have to wait 7 hours? I have almost 1700 file. I couldn't find a script for that.

[seblemaguer]

Hello,

have you got the error message ?

Kind regards, Sébastien

ahoenemann writes:

Sorry, but I have one question. I've started the HMMVoiceMakeDate from the VoiceImpoirtTool and now after 7 hour I have an error and the process was stopped. How can I start it at a particular point of the process or have I start it again and have to wait 7 hours? I have almost 1700 file. I couldn't find a script for that.

— Reply to this email directly or view it on GitHub.*

Dr. Sébastien Le Maguer Postdoctorate researcher

Saarland University Campus C7.4 - room 2.03 D-66123 Saarbrücken Germany

phone : +49-681-302-70030 Mail: slemaguer@coli.uni-saarland.de website : http://sites.google.com/site/seblemaguer

[ahoenemann]

Yes, the question folder or files could not be found. Of course because there wasn't a folder. I don't no why. Unfortunately I had no log file, so I can't give you the exact error message. I've started the whole process again with a log file. I use now the makefile instead the GUI. I've built a HMM voice but with a smaller set of files (400), so the voice sounds like micky mouse.

[seblemaguer]

Hello,

sorry for the delay. You need to go to the specific hts directory ("voicedir"/hts/) and run manually the following command:

perl scripts/Training.pl scripts/Config.pm

[ahoenemann]

Hello, no problem. I've already found it. After two days and nights training I hope it will be finished today and it will be sounds good ;-)

[ahoenemann]

Ok, it doesn't work. I've built a voice now but it sounds like micky mouse. What could be the problem? It seems that are only 9 files were taken because it is written at the end the following only for 9 files. Can it be?
... Synthesizing a speech waveform from /home/ahoenemann/pv_full/hts/data/labels/gen/gen_s0084.lab using hts_engine...done. ... I had also many warnigs at the step HAlign (I think) that the input data does not in the 'short' (or something like this). It has something to do with the f0? Otherwise there were no errors.

Can I do something afterwards to improve the voice? I tried it with the client but it doesn't work neither.

Or mybe the recording, the human voice, is not useful?