Holzhaus
commented
8 years ago There is no error visible in your logs.
Closed Thomas301263 closed 5 years ago
Holzhaus
commented
8 years ago There is no error visible in your logs.
Thomas301263
commented
8 years ago Maybe it is somewhere else cause since the last update the keyword jasper won't be recognized in German language. That was different before. I have done: cd ~/jasper git fetch origin git checkout -- . git checkout origin/jasper-dev git checkout origin/feat/multilanguage-support
Holzhaus
commented
8 years ago The feat/multilanguage-support branch doesn't exist anymore, because it has been merged into jasper-dev in e3f2bf530057af596f245782b260a65db96a1658, so you need to use:
$ cd ~/jasper
$ git fetch origin
$ git checkout origin/jasper-devYou can check that via:
$ git rev-parse --verify --short=7 HEADIf you're using the latest version, it should print the same commit id like the uppermost entry on this page, i.e. 7d4d306 at the moment.
Holzhaus
commented
8 years ago If that does not fix your issue, remove all compiled vocabularies to force recompilation:
$ rm -rf ~/.jasper/vocabulariesThen run jasper an check again. If the problem is still there, I need the output of:
$ cat ~/.jasper/vocabularies/de-DE/sphinx/keyword/dictionarygit rev-parse --verify --short=7 HEAD the output is like you wrote: 7d4d306 It seems to be that jasper will not understand the German pocketsphinx anymore. Before it was ok. Here the output of
pi@pi:~/jasper $ cat ~/.jasper/vocabularies/de-DE/sphinx/keyword/dictionary
WELCHES W EH L CH AH Z
WELCHES(2) W EH L CH IY Z
WELCHES(3) W EH L CH AH S
WELCHER W EH L CH ER
WELCHER(2) W EH L K ER
WELCHER(3) V EH L CH ER
ABER EY B ER
ABER(2) AE B ER
ABER(3) AH B ER
WAREN W AO R AH N
WAREN(2) W EH R AH N
WAREN(3) W EH R N
DIE D IY
DIE(2) D AY
DIE(3) D AY IY
DER D ER
DER(2) D EH R
DER(3) D EY ER
IST IH S T
IST(2) AY S T
IST(3) AH S T
SIE S IY
SIE(2) EH S AY IY
SIE(3) S AY IY
SIND S IH N D
SIND(2) S AY N D
SIND(3) S IH N
MIT M IH T
MIT(2) M AH T
MIT(3) M IY T
BIN B IH N
BIN(2) B AH N
BIN(3) B IY N
WIRD W ER D
WIRD(2) W IH R D
WIRD(3) W AY R D
WAS W AA Z
WAS(2) W AH Z
WAS(3) W AA S
SAG S AE G
SAG(2) S AA G
SAG(3) S EY G
JA Y AH
JA(2) JH AH
JA(3) ZH AH
WESHALB W EH SH AO L B
WESHALB(2) W IH SH AO L B
WESHALB(3) W EH SH AE L B
SEID S AY D
SEID(2) S IY D
SEID(3) S AY T
ICH IH CH
ICH(2) IH HH
ICH(3) IH K
NEIN N IY N
NEIN(2) N AY N
NEIN(3) N EY N
DAS D AE S
DAS(2) D AH Z
DAS(3) D AH S
AN AE N
AN(2) AH N
AN(3) AA N
WER W ER
WER(2) W EH R
WER(3) W R
WEIL W AY L
WEIL(2) W IY L
WEIL(3) V AY L
JASPER JH AE S P ER
JASPER(2) Y AH S P ER
JASPER(3) Y AA S P ER
IN IH N
IN(2) AH N
IN(3) AE N
WIESO W IY S OW
WIESO(2) W AY S OW
WIESO(3) V IY S OW
ERST ER S T
ERST(2) EH R S T
ERST(3) EH R AH S T
WAR W AO R
WAR(2) W AA R
WAR(3) W ER
ES EH S
ES(2) IY Z
ES(3) AH S
ER ER
ER(2) EH R
ER(3) IH R
WIE W IY
WIE(2) V IY
WIE(3) W AY
WARUM W AO R AH M
WARUM(2) W EH R AH M
WARUM(3) W ER AH M
IHR IH R
IHR(2) IH
IHR(3) AY R
VON V AA N
VON(2) V AO N
VON(3) V AH N
WERDET W ER D EH T
WERDET(2) W ER D AH T
WERDET(3) W ER D IH T
JETZT JH EH T S T
JETZT(2) Y EH T S T
JETZT(3) JH EH T Z T
WERDEN W ER D AH N
WERDEN(2) W ER D EH N
WERDEN(3) W ER D N
ARBEIT AA R B AY T
ARBEIT(2) AA R B IY AH T
ARBEIT(3) AA R B AY T IY
WIR W ER
WIR(2) W IH R
WIR(3) W AY ER
pi@pi:~/jasper $ Looks normal. What about this?
$ cat ~/.jasper/vocabularies/de-DE/sphinx/keyword/languagemodelIt seems to be normal and runs for the first time. If I say Jasper once all looks ok but if I try to say Uhrzeit or something else Jasper will not understand. Before it was working. Here the language model.
pi@pi:~ $ cat ~/.jasper/vocabularies/de-DE/sphinx/keyword/languagemodel
#############################################################################
## Copyright (c) 1996, Carnegie Mellon University, Cambridge University,
## Ronald Rosenfeld and Philip Clarkson
## Version 3, Copyright (c) 2006, Carnegie Mellon University
## Contributors includes Wen Xu, Ananlada Chotimongkol,
## David Huggins-Daines, Arthur Chan and Alan Black
#############################################################################
=============================================================================
=============== This file was produced by the CMU-Cambridge ===============
=============== Statistical Language Modeling Toolkit ===============
=============================================================================
This is a 3-gram language model, based on a vocabulary of 41 words,
which begins "</s>", "<s>", "ABER"...
This is an OPEN-vocabulary model (type 1)
(OOVs were mapped to UNK, which is treated as any other vocabulary word)
Good-Turing discounting was applied.
1-gram frequency of frequency : 38
2-gram frequency of frequency : 77 0 0 0 0 0 0
3-gram frequency of frequency : 115 0 0 0 0 0 0
1-gram discounting ratios : 0.95
2-gram discounting ratios :
3-gram discounting ratios :
This file is in the ARPA-standard format introduced by Doug Paul.
p(wd3|wd1,wd2)= if(trigram exists) p_3(wd1,wd2,wd3)
else if(bigram w1,w2 exists) bo_wt_2(w1,w2)*p(wd3|wd2)
else p(wd3|w2)
p(wd2|wd1)= if(bigram exists) p_2(wd1,wd2)
else bo_wt_1(wd1)*p_1(wd2)
All probs and back-off weights (bo_wt) are given in log10 form.
Data formats:
Beginning of data mark: \data\
ngram 1=nr # number of 1-grams
ngram 2=nr # number of 2-grams
ngram 3=nr # number of 3-grams
\1-grams:
p_1 wd_1 bo_wt_1
\2-grams:
p_2 wd_1 wd_2 bo_wt_2
\3-grams:
p_3 wd_1 wd_2 wd_3
end of data mark: \end\
\data\
ngram 1=42
ngram 2=78
ngram 3=115
\1-grams:
-2.0830 <UNK> 0.0000
-0.4809 </s> -1.4222
-0.4696 <s> -1.7287
-2.0830 ABER -0.3029
-2.0830 AN -0.3029
-2.0830 ARBEIT -0.3029
-2.0830 BIN -0.3029
-2.0830 DAS -0.3029
-2.0830 DER -0.3029
-2.0830 DIE -0.3029
-2.0830 ER -0.3029
-2.0830 ERST -0.3029
-2.0830 ES -0.3029
-2.0830 ICH -0.3029
-2.0830 IHR -0.3029
-2.0830 IN -0.3029
-2.0830 IST -0.3029
-2.0830 JA -0.3029
-2.0830 JASPER 0.0000
-2.0830 JETZT -0.3029
-2.0830 MIT -0.3029
-2.0830 NEIN -0.3029
-2.0830 SAG -0.3029
-2.0830 SEID -0.3029
-2.0830 SIE -0.3029
-2.0830 SIND -0.3029
-2.0830 VON -0.3029
-2.0830 WAR -0.3029
-2.0830 WAREN -0.3029
-2.0830 WARUM -0.3029
-2.0830 WAS -0.3029
-2.0830 WEIL -0.3029
-2.0830 WELCHER -0.3029
-2.0830 WELCHES -0.3029
-2.0830 WER -0.3029
-2.0830 WERDEN -0.3029
-2.0830 WERDET -0.3029
-2.0830 WESHALB -0.3029
-2.0830 WIE -0.3029
-2.0830 WIESO -0.3029
-2.0830 WIR -0.3029
-2.0830 WIRD -0.3029
\2-grams:
-0.0110 </s> <s> -0.1706
-1.5966 <s> ABER 0.1761
-1.5966 <s> AN 0.1761
-1.5966 <s> ARBEIT 0.1761
-1.5966 <s> BIN 0.1761
-1.5966 <s> DAS 0.1761
-1.5966 <s> DER 0.1761
-1.5966 <s> DIE 0.1761
-1.5966 <s> ER 0.1761
-1.5966 <s> ERST 0.1761
-1.5966 <s> ES 0.1761
-1.5966 <s> ICH 0.1761
-1.5966 <s> IHR 0.1761
-1.5966 <s> IN 0.1761
-1.5966 <s> IST 0.1761
-1.5966 <s> JA 0.1761
-1.5966 <s> JASPER -0.1268
-1.5966 <s> JETZT 0.1761
-1.5966 <s> MIT 0.1761
-1.5966 <s> NEIN 0.1761
-1.5966 <s> SAG 0.1761
-1.5966 <s> SEID 0.1761
-1.5966 <s> SIE 0.1761
-1.5966 <s> SIND 0.1761
-1.5966 <s> VON 0.1761
-1.5966 <s> WAR 0.1761
-1.5966 <s> WAREN 0.1761
-1.5966 <s> WARUM 0.1761
-1.5966 <s> WAS 0.1761
-1.5966 <s> WEIL 0.1761
-1.5966 <s> WELCHER 0.1761
-1.5966 <s> WELCHES 0.1761
-1.5966 <s> WER 0.1761
-1.5966 <s> WERDEN 0.1761
-1.5966 <s> WERDET 0.1761
-1.5966 <s> WESHALB 0.1761
-1.5966 <s> WIE 0.1761
-1.5966 <s> WIESO 0.1761
-1.5966 <s> WIR 0.1761
-1.5966 <s> WIRD 0.1761
-0.1761 ABER </s> 1.3010
-0.1761 AN </s> 1.3010
-0.1761 ARBEIT </s> 1.3010
-0.1761 BIN </s> 1.3010
-0.1761 DAS </s> 1.3010
-0.1761 DER </s> 1.3010
-0.1761 DIE </s> 1.3010
-0.1761 ER </s> 1.3010
-0.1761 ERST </s> 1.3010
-0.1761 ES </s> 1.3010
-0.1761 ICH </s> 1.3010
-0.1761 IHR </s> 1.3010
-0.1761 IN </s> 1.3010
-0.1761 IST </s> 1.3010
-0.1761 JA </s> 1.3010
-0.1761 JETZT </s> 1.3010
-0.1761 MIT </s> 1.3010
-0.1761 NEIN </s> 1.3010
-0.1761 SAG </s> 1.3010
-0.1761 SEID </s> 1.3010
-0.1761 SIE </s> 1.3010
-0.1761 SIND </s> 1.3010
-0.1761 VON </s> 1.3010
-0.1761 WAR </s> 1.3010
-0.1761 WAREN </s> 1.3010
-0.1761 WARUM </s> 1.3010
-0.1761 WAS </s> 1.3010
-0.1761 WEIL </s> 1.3010
-0.1761 WELCHER </s> 1.3010
-0.1761 WELCHES </s> 1.3010
-0.1761 WER </s> 1.3010
-0.1761 WERDEN </s> 1.3010
-0.1761 WERDET </s> 1.3010
-0.1761 WESHALB </s> 1.3010
-0.1761 WIE </s> 1.3010
-0.1761 WIESO </s> 1.3010
-0.1761 WIR </s> 1.3010
-0.1761 WIRD </s> 1.3010
\3-grams:
-1.5911 </s> <s> ABER
-1.5911 </s> <s> AN
-1.5911 </s> <s> ARBEIT
-1.5911 </s> <s> BIN
-1.5911 </s> <s> DAS
-1.5911 </s> <s> DER
-1.5911 </s> <s> DIE
-1.5911 </s> <s> ER
-1.5911 </s> <s> ERST
-1.5911 </s> <s> ES
-1.5911 </s> <s> ICH
-1.5911 </s> <s> IHR
-1.5911 </s> <s> IN
-1.5911 </s> <s> IST
-1.5911 </s> <s> JA
-1.5911 </s> <s> JASPER
-1.5911 </s> <s> JETZT
-1.5911 </s> <s> MIT
-1.5911 </s> <s> NEIN
-1.5911 </s> <s> SAG
-1.5911 </s> <s> SEID
-1.5911 </s> <s> SIE
-1.5911 </s> <s> SIND
-1.5911 </s> <s> VON
-1.5911 </s> <s> WAR
-1.5911 </s> <s> WAREN
-1.5911 </s> <s> WARUM
-1.5911 </s> <s> WAS
-1.5911 </s> <s> WEIL
-1.5911 </s> <s> WELCHER
-1.5911 </s> <s> WELCHES
-1.5911 </s> <s> WERDEN
-1.5911 </s> <s> WERDET
-1.5911 </s> <s> WESHALB
-1.5911 </s> <s> WIE
-1.5911 </s> <s> WIESO
-1.5911 </s> <s> WIR
-1.5911 </s> <s> WIRD
-0.3010 <s> ABER </s>
-0.3010 <s> AN </s>
-0.3010 <s> ARBEIT </s>
-0.3010 <s> BIN </s>
-0.3010 <s> DAS </s>
-0.3010 <s> DER </s>
-0.3010 <s> DIE </s>
-0.3010 <s> ER </s>
-0.3010 <s> ERST </s>
-0.3010 <s> ES </s>
-0.3010 <s> ICH </s>
-0.3010 <s> IHR </s>
-0.3010 <s> IN </s>
-0.3010 <s> IST </s>
-0.3010 <s> JA </s>
-0.3010 <s> JASPER </s>
-0.3010 <s> JETZT </s>
-0.3010 <s> MIT </s>
-0.3010 <s> NEIN </s>
-0.3010 <s> SAG </s>
-0.3010 <s> SEID </s>
-0.3010 <s> SIE </s>
-0.3010 <s> SIND </s>
-0.3010 <s> VON </s>
-0.3010 <s> WAR </s>
-0.3010 <s> WAREN </s>
-0.3010 <s> WARUM </s>
-0.3010 <s> WAS </s>
-0.3010 <s> WEIL </s>
-0.3010 <s> WELCHER </s>
-0.3010 <s> WELCHES </s>
-0.3010 <s> WER </s>
-0.3010 <s> WERDEN </s>
-0.3010 <s> WERDET </s>
-0.3010 <s> WESHALB </s>
-0.3010 <s> WIE </s>
-0.3010 <s> WIESO </s>
-0.3010 <s> WIR </s>
-0.3010 <s> WIRD </s>
-0.3010 ABER </s> <s>
-0.3010 AN </s> <s>
-0.3010 ARBEIT </s> <s>
-0.3010 BIN </s> <s>
-0.3010 DAS </s> <s>
-0.3010 DER </s> <s>
-0.3010 DIE </s> <s>
-0.3010 ER </s> <s>
-0.3010 ERST </s> <s>
-0.3010 ES </s> <s>
-0.3010 ICH </s> <s>
-0.3010 IHR </s> <s>
-0.3010 IN </s> <s>
-0.3010 IST </s> <s>
-0.3010 JA </s> <s>
-0.3010 JETZT </s> <s>
-0.3010 MIT </s> <s>
-0.3010 NEIN </s> <s>
-0.3010 SAG </s> <s>
-0.3010 SEID </s> <s>
-0.3010 SIE </s> <s>
-0.3010 SIND </s> <s>
-0.3010 VON </s> <s>
-0.3010 WAR </s> <s>
-0.3010 WAREN </s> <s>
-0.3010 WARUM </s> <s>
-0.3010 WAS </s> <s>
-0.3010 WEIL </s> <s>
-0.3010 WELCHER </s> <s>
-0.3010 WELCHES </s> <s>
-0.3010 WER </s> <s>
-0.3010 WERDEN </s> <s>
-0.3010 WERDET </s> <s>
-0.3010 WESHALB </s> <s>
-0.3010 WIE </s> <s>
-0.3010 WIESO </s> <s>
-0.3010 WIR </s> <s>
-0.3010 WIRD </s> <s>
\end\
pi@pi:~ $ This also seems fine. Does recognition work when running:
pocketsphinx_continuous -hmm /home/pi/voxforge-de-r20141117/model_parameters/voxforge.cd_cont_3000 -dict ~/.jasper/vocabularies/de-DE/sphinx/default/dictionary -lm ~/.jasper/vocabularies/de-DE/sphinx/default/languagemodel -inmic yesIt is doing something but not what am saying. As I said it already was running before. SSH connection will not work with raspi cause of security. Therefore another program has to be installed. I have installed teamviewer on another laptop. That you can use to try to connect. Teamfiewer ID = 538257853 Password = 8200
Thomas301263
commented
8 years ago btw. English language is working.
Holzhaus
commented
8 years ago Sorry, I don't have teamviewer installed. What FST model are you using?
Holzhaus
commented
8 years ago Also, it would be helpful if you could post the output when if it's actually compiling the vocabulary, i.e. the output of:
$ rm -rf ~/.jasper/vocabularies
$ ./jasper.py --debugHere is the file you like to get. I put it in a separate file cause it is a bit larger then others
Holzhaus
commented
8 years ago Hmm there is no problem at all during compilation. What does happen if you actually say something?
Thomas301263
commented
8 years ago Three times I said Jasper till the beep, then I said "Uhrzeit", then again "Jasper" then "Zeit", then "Jasper" then "Wie spät ist es" Any time he said that jasper doesn't understand it. Here again the file: jasper.txt
Holzhaus
commented
8 years ago Okay, then there is not program error at all.
Recognition doesn't work properly. because:
The reason why you didn't notice that before was because there were only a few words in that model and Pocketsphinx kinda recognized them "by chance". I added more words to compile into that model in commit 841f41e, and now the bad recognition rate is much more noticeable.
Thus, I reverted that change. I also added X-SAMPA support, so that you can use a german FST model instead of an english one:
pocketsphinx:
hmm_dir: '/path/to/voxforge-de-r20141117/model_parameters/voxforge.cd_cont_3000'
fst_model: '/path/to/dict-xsampa.fst'
fst_model_alphabet: 'xsampa'Got the dict-xsampa.fst for German and that voxforge model and added it. If am talking now to Jasper it recognizes nothing as I have seen in the debug output.
Thomas301263
commented
8 years ago Here is a new output of jasper. new.txt
Holzhaus
commented
8 years ago Did you add fst_model_alphabet: 'xsampa'?
Thomas301263
commented
8 years ago I added in my folder ..../de dict-xsampa.arpa ....corpus .....fst Something else I haven't found.
Holzhaus
commented
8 years ago No, i was referring to your profile.yml.
Thomas301263
commented
8 years ago My profile.yml
language: 'de-DE'
carrier: ''
first_name: Thomas
gmail_password: ''
location: Ingolstadt
phone_number: ''
prefers_email: false
stt_engine: sphinx
pocketsphinx:
hmm_dir: '/home/pi/voxforge-de-r20141117/model_parameters/voxforge.cd_cont_3000'
fst_model: '/home/pi/de/dict-xsampa.fst'
fst_model_alphabet: 'xsampa'
# fst_model: '../phonetisaurus/g014b2b.fst' #optional
# hmm_dir: '/usr/local/share/pocketsphinx/model/hmm/en_US/hub4wsj_sc_8k' #optional
# hmm_dir: '/home/pi/pocketsphinx-5prealpha/test/data/tidigits/hmm'
# hmm_dir: '/home/pi/pocketsphinx-0.8/model/hmm/en_US/hub4wsj_sc_8k'
# hmm_dir: '/home/pi/voxforge-de-r20141117/model_parameters/voxforge.cd_cont_3000'
tts_engine: pico-tts
#pico-tts:
# language: 'de-DE'
timezone: Europe/Berlin
input_device: 'default'
output_device: 'default'For testings I changed the language back to English. As long as I am not using my own modules it seems to be all is working fine. If I copy my modules in the folder ~/.jasper/plugin/speechhandler and start jasper then my modules are not working. After I start your command
./compile_translations.sh my modules are working but then jasper for example hardly understand if I say TIME. It recognize POWER or NEWS instead of time.
G10DRAS
commented
8 years ago Hello Holzhaus, Though not 100% accurate but German Acoustic Model works well for me.
One question here: Where did you get XSAMPA_TO_ARPABET_MAPPING ?
I am trying to generate a German FST model which will come with inbuilt support for XSAMPA_TO_ARPABET_MAPPING.
XSAMPA_TO_ARPABET_MAPPING = {
'p': 'P',
'b': 'B',
't': 'T',
'd': 'D',
'k': 'K',
'g': 'G',
'?': 'Q',............
Thomas301263
commented
8 years ago Hi G10DRAS,
may I ask which language model you are using cause in my case it will not work. How does your profil.yml looks like? I am trying to get the German language working but without many success. Now I am playing with Simon and there is a possibility to train the acoustic Model and it will work. Is there a possibility to convert that model that it can be used together with Jasper?
Regards Thomas
G10DRAS
commented
8 years ago Thomas See the threadd below
I have generated an German FST model which works with German Acoustic Model (voxforge.cd_cont_3000 from http://goofy.zamia.org/voxforge/de/voxforge-de-r20141117.tgz). No code change required in Jasper.
Download it from https://www.dropbox.com/s/vbkf8qjvsf67b66/voxforge-de.fst?dl=0
See how it works...........
G10DRAS
commented
8 years ago In profile.yml specify the path to hmm dir and fst as usual.
pocketsphinx: hmm_dir: '/path/to/voxforge-de-r20141117/model_parameters/voxforge.cd_cont_3000' fst_model: '/path/to/downloaded.fst'
Thomas301263
commented
8 years ago I have updated Jasper before and using that files you wrote to run Jasper in German language. Jasper stops now since 5 hours on the same step:
DEBUG:sphinx_1_0_0.sphinxvocab:Getting phonemes for 39 words...
DEBUG:sphinx_1_0_0.g2p:Converting 39 words to phonemesTestings with English language before was working.
G10DRAS
commented
8 years ago Could you try previous version of Jasper (Jasper v1 or jasper-main) with german acoustic model and my FST. My FST model was compiled using best available dictionary which contains only 23000 german words. I am away from my RPI2 for next 2 days hence can not figure out where is the problem exactly. But i tested it in the past with almost 50 WORDS and my jasper (Jasper v1) works very well.
Thomas301263
commented
8 years ago You are right with version v1 of Jasper it works with me as well till there were some updates and I try to find out now why this version is not working. v1 of Jasper has the problem of misunderstanding of my spoken words so I go further steps and try if there will be a solution. I know German language is not supported that much then English that's why I am using Simon in parallel to train Simon till it works and later I will try to compile Simon *.sbm that I can use it for Jasper. Hope there is a way doing that.
andweber
commented
8 years ago I have the same issue with the dev-branch as mentioned by Thomas301263. Jasper V1 not tested. g2p conversion stumbles over the words:
ES ICH IN IST SIE
leaving these words out and the g2p conversion is able to finish. It works fine with dict-xsampa.fst. Thus it seems to be a problem of g2p with the fst model of G10DRAS (https://www.dropbox.com/s/vbkf8qjvsf67b66/voxforge-de.fst?dl=0)
edit one more remark - the recognition results seem to be better with the model of G10DRAS - but its just a feeling.
G10DRAS
commented
8 years ago could you try below config with jasper dev-branch (comment out fst_model_alphabet: 'xsampa' line )
pocketsphinx: hmm_dir: '/path/to/voxforge-de-r20141117/model_parameters/voxforge.cd_cont_3000' fst_model: '/path/to/downloaded_dropbox.fst'
Thomas301263
commented
8 years ago That's I already made without success.
andweber
commented
8 years ago @Thomas301263: if you like to get it running, go to: ./jasper/data/standard_phrases/de-DE.txt and delete the words mentioned above.
@G10DRAS: its not a problem with an option. Inline conversion of the phonemes is done after the g2p conversion call. You get the same result directly calling phonetisaurus:
phonetisaurus-g2p --model=/path/to/downloaded_dropbox.fst --input=SIE --nbest=3 is broken.
phonetisaurus-g2p --model=/path/to/downloaded_dropbox.fst --input=DAS --nbest=3 is working
and
phonetisaurus-g2p --model=/path/to/dict-xsempa.fst --input=SIE --nbest=3 is working
I have not yet realy understood what the content of *.fst is. This might be a problem with phonetisaurus or with the model. I'm using phonetisaurus 0.7.8 - as mentioned in the install instructions of jasper - but I will check with a master build from github.
edit it seems to be a bug in phonetisaurus 0.7.8, current git hub version of phonetisaurus (GitRevision: 09651e) is working with. But the interface changed, thus jasper needs to adapted.
G10DRAS
commented
8 years ago Ok What I did just now with Jasper on RPi2B. I took 38 German Words (Keyword Phrases) + Use My German FST voxforge-de.fst + HMM voxforge.cd_cont_3000 + Pocketsphinx-5prealpha (Compiled and installed from src downloaded from Github) + Inbuilt Mic of Logicool c270 Webcam
ABER AN ARBEIT BIN DAS DER DIE ER ERST ES ICH IHR IN IST JA JASPER JETZT MIT NEIN SAG SEID SIE SIND VON WAR WAREN WARUM WAS WEIL WELCHER WELCHES WERDEN WERDET WESHALB WIE WIESO WIR WIRD
I got result as follows (I can say results are 60% accurate, though I am not a native German speaker)
Transcribed:=======> '' Transcribed:=======> 'WERDET' Transcribed:=======> '' Transcribed:=======> 'SIND SIE' Transcribed:=======> '' Transcribed:=======> 'WERDEN' Transcribed:=======> '' Transcribed:=======> 'WELCHES' Transcribed:=======> 'WIR' Transcribed:=======> 'WERDET' Transcribed:=======> 'ER' Transcribed:=======> 'ER' Transcribed:=======> 'ER' Transcribed:=======> 'NEIN' Transcribed:=======> 'SEID' Transcribed:=======> '' Transcribed:=======> 'SEID SIE' Transcribed:=======> 'JA' Transcribed:=======> 'IST WIR' Transcribed:=======> '' Transcribed:=======> 'NEIN' Transcribed:=======> 'NEIN' Transcribed:=======> 'ER' Transcribed:=======> '' Transcribed:=======> 'IN' Transcribed:=======> '' Transcribed:=======> 'IST' Transcribed:=======> 'SIE' Transcribed:=======> 'SEID' Transcribed:=======> '' Transcribed:=======> '' Transcribed:=======> 'WERDEN' Transcribed:=======> 'JETZT' Transcribed:=======> 'ES WAS' Transcribed:=======> '' Transcribed:=======> 'JETZT' Transcribed:=======> 'WELCHER' Transcribed:=======> 'ER' Transcribed:=======> 'JASPER' Transcribed:=======> '' Transcribed:=======> 'IST' Transcribed:=======> 'ER' Transcribed:=======> '' Transcribed:=======> 'NEIN' Transcribed:=======> 'WEIL' Transcribed:=======> '' Transcribed:=======> 'WEIL' Transcribed:=======> 'JA' Transcribed:=======> 'WERDEN' Transcribed:=======> '' Transcribed:=======> 'IST' Transcribed:=======> 'IST' Transcribed:=======> 'ER' Transcribed:=======> '' Transcribed:=======> 'IN' Transcribed:=======> '' Transcribed:=======> 'WIE' Transcribed:=======> 'WIR' Transcribed:=======> '' Transcribed:=======> 'MIT' Transcribed:=======> 'WIESO' Transcribed:=======> '' Transcribed:=======> 'WELCHER' Transcribed:=======> 'ER' Transcribed:=======> 'IST'
Try installing Pocketsphinx-5prealpha and see if it improves recognization rate.
Thomas301263
commented
8 years ago I tried your solution and you are right that the recognition rate is about 60%. That is to less for me cause Jasper should control my flat and if he do a lot of wrong things then maybe lights, power heating system, rollerblinds and others will not work correctly. I am using now 2 Raspi's 2b with different solutions. One of them is Jasper and the other one is Simon. Raspi 2b together with Simon will work well and the recognition rate is about 95% cause I can train Simon as I want. That doesn't mean that I am not working with Jasper anymore but I am trying to get the best solution for my smart home controller which work perfect but control everything so there should be a very high recognition rate.
G10DRAS
commented
8 years ago If you know how to Train / Adapt an Acoustic Model for CMUSphinx, you can try that, and use that Trained / Adapted Acoustic Model with Jasper. You will get a lot better accuracy with Jasper too.
See links below: http://cmusphinx.sourceforge.net/wiki/tutorialam http://cmusphinx.sourceforge.net/wiki/tutorialadapt
or at least try to generate mllr_matrix and use that with Pocketsphinx in Jasper.
For voice controlled home automation, it is too much if you say "TURN ON GROUND FLOOR KICHEN TABLE LIGHT" another solution is use shortcut, like SEVEN --- for Bedroom Light TEN --- for Kichen Light example command: TURN ON SEVEN SWITCH OFF TEN HEATER ON SEVEN
Thomas301263
commented
8 years ago Thanks I will try to do it that way you wrote but I like to get the voice command in another way. I like to say "Strom Schlafzimmer einschalten" and then there should be a reaction. If there is a way to get the sphinx training from Simon to Jasper then it will be good. To train Simon is really simple and quick and if it can be transformed in a format for Jasper then it will be more then ok. Now am trying your way of training.
G10DRAS
commented
8 years ago I am not sure if you can use trained model from simon into jasper, could you attached a sample file? There is another application I.L.A which do the same thing as Simon do i.e Train a Model on the fly. https://sites.google.com/site/ilavoiceassistant/ I check the code but sorce code is not available for it (only binaries.). Take alook if it works for you.
Thomas301263
commented
8 years ago I will take a look and try. Thanks Normally the file is basemodel.sbm but I cannot place it here so I made .txt at the end. Just rename it and then you should have it. basemodel.sbm.txt
G10DRAS
commented
8 years ago attached sbm file is a compressed file like zip file, I opened it with 7-zip and it contains all Acoustic model files. Try uncompressed model with Jasper, It should work.
Thomas301263
commented
8 years ago Now I tried your way and basically Jasper understood my words I am talking but the problem is if am using long phrases like "Jasper Rollo Wohnzimmer Fenster Ost schliessen" then Jasper struggle. There should be a timer where pauses between words can be defined. Now I am running Jasper and Simon on Raspi 2 and 3 in parallel and Simon is better if I am using long phrases. The main reason that I need long phrases is that I am trying to get something which support me and making things for me easier. That could not be done by short phrases. At the end there should be a system for elder people which can use it supporting them in difficult situation. Example: Somebody is not able to take a phone cause he cannot move (maybe a stroke) but he is able to talk so he can say "Computer I need emergency help" The computer is connecting that person to somebody else by phone or by mail ..... etc. telling them that there is a emergency.
andweber
commented
8 years ago @Thomas301263: looks like I'm trying to get something similar running - just curiosity, does your basemodel.sbm includes an updated trained model? I tried that too, but my basemodel remained untrained.
Concerning the recognition: Are you using a language model corresponding to your input?
G10DRAS
commented
8 years ago @Thomas301263 Seems Simon is using JSGF file with PocketSphinx Decoder and not using LM file.
"-hmm", model.data(),
"-jsgf", grammar.data(),
"-dict", dict.data(),
"-samprate", samprate.data(),Do some experiment using _pocketsphinxcontinue .
See the difference, if you feel JSGF file is better, configure your Jasper's PocketSphinx Decoder to use JSGF file and see how Jasper perform.
in stt.py you need to do something like below:
# psConfig.set_string('-lm', "</path/to/lm_file>")
psConfig.set_string('-jsgf', "</path/to/jsgf_file>")@andweber I have not used a base model and I was starting from scratch with Simon. I trained every word about 30-40 times build my phrases like sentences trained them as well and Simon is understanding me well. Then I took the .sbm file uncompress it and replaced the files from Jasper. Now I will try the way G10DRAS suggest me and we will found a way.
Thomas301263
commented
8 years ago @G10DRAS Now there was time trying your suggestion. For me it seems to be that you are using Jasper V1. I am using Jasper V2 and there is no stt.py file anymore. I do not know where I have to set .jsgf or .lm files in that case. I know in the old version there was a stt.py file.
andweber
commented
8 years ago the right spot is in plugins/stt/pocketsphinx-stt/sphinxplugin.py line 90ff
if self._pocketsphinx_v5:
# Pocketsphinx v5
config = pocketsphinx.Decoder.default_config()
config.set_string('-hmm', hmm_dir)
config.set_string('-jsgf', jsgf_path) # <--- replaced line with lm
config.set_string('-dict', dict_path)
config.set_string('-logfn', self._logfile)
self._decoder = pocketsphinx.Decoder(config)
else:
# Pocketsphinx v4 or sooner
self._decoder = pocketsphinx.Decoder(
hmm=hmm_dir, logfn=self._logfile, jsgf=jsgf_path, dict=dict_path)`
For Pocketsphinx v4 I only guessed the code. Instead of jsgf_path insert the correct path for you.
Thomas301263
commented
8 years ago @andweber Thanks for that information. I tried it immediately but there is another error now and I do not know where to set it.
File "/home/pi/jasper/plugins/stt/pocketsphinx-stt/sphinxplugin.py", line 94, in __init__
config.set_string('-jsgf', jsgf_path)
NameError: global name 'jsgf_path' is not defined
Since the last update Jasper is not supporting German language correctly with pocketsphinx. English language so far is ok. Jasper with German language:
Jasper with English language: