search

jasperproject / jasper-client

Client code for Jasper voice computing platform
MIT License
4.53k stars 1.01k forks source link

Change Jasper Language #599

Open odouglsantos opened 7 years ago

odouglsantos commented 7 years ago

Hi, my name is Doug. I'm from Brazil and I'm using Project Jasper as part of the college completion work. And I would like to know if it is possible to add command in Portuguese, for me no problem in using it in English, but the bank that will evaluate will not accept. For example when I say "Jasper, what time is it?" In Portuguese it would be "Jasper Que horas são?". Is there such a possibility?

G10DRAS commented 7 years ago

It depends on STT you are using with Jasper. Also you need to handle Portuguese words in Jasper Modules. For example HORAS in place of TIME. And Jasper also need a Portuguese TTS, so that it will speake out "São 2:30 pm agora."

odouglsantos commented 7 years ago

I understood, thank you for the information. I researched and realized that Julius allows you to create a phonetic phoneline, but it is very complicated. I tried to use Google TTS, but it gave 403 error .. I believe it is related to the Token to develop but I did not understand how to use it.

G10DRAS commented 7 years ago

I dont think Julius support Portuguese. Jasper supports Google Speech API, it doesnt support Google Speech Cloud. So generate API Key accordingly. And I think google stop accepting httpsrequest for its Speech API, hence you need to change code little bit in stt.py

                # ('https', 'www.google.com', '/speech-api/v2/recognize', '',
                ('http', 'www.google.com', '/speech-api/v2/recognize', '',

and you need to change language from en-us to pt-br

paoloantinori commented 7 years ago

Hi, how can I create a Google Speech API key?

https://console.developers.google.com

lists only Google Cloud Speech API

G10DRAS commented 7 years ago

https://console.developers.google.com/apis/private

paoloantinori commented 7 years ago

Thank you, but only Google Cloud Speech API is listed when filtering the apis list for speech

G10DRAS commented 7 years ago

speech

You need to select Speech API

paoloantinori commented 7 years ago

Thank you, this were the steps I was missing: http://www.chromium.org/developers/how-tos/api-keys

On 8 March 2017 at 10:39, ジ10ドラス notifications@github.com wrote:

[image: speech] https://cloud.githubusercontent.com/assets/14088095/23698329/337330c6-0411-11e7-83b0-8fa24a25e198.jpg

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/jasperproject/jasper-client/issues/599#issuecomment-284993872, or mute the thread https://github.com/notifications/unsubscribe-auth/ABcz2syg1petFbNASm0DygUkjCYsA6Plks5rjndogaJpZM4MTIIO .

rhumphan commented 7 years ago

Hi, have you succeeded with this? I've tried to change it but nothing work >.<

geovannimp commented 7 years ago

I think jasper would have an i18n lib where could be possibe configure a default language and modules have language files.

chaitanya11 commented 7 years ago

Use google translation api and embed English text from google to jasper

G10DRAS commented 7 years ago

@geovannikun what you are looking for is already supported by jasoer-dev version. take a look into the code.

wrboliveira commented 7 years ago

Olá amigo deu certo inserir o pt-br no jasper se sim como o fez

G10DRAS commented 7 years ago

try google-stt and google-tts

wrboliveira commented 7 years ago

fia configuração desta forma mas ainda obtenho erro! stt_engine: google tts_engine: google-espeech

wrboliveira commented 7 years ago

Preciso usar o jasper com o portuguese-brasil, estou com dificuldade para configurar o ptbr , o meu jasper já esta funcionando mas somente com sphinx, quando muda para google ou espeak ele da erro.

E já forcei o pt-br no stt.py mas sem sucesso.

G10DRAS commented 7 years ago

Try following config in profile for google STT and TTS

stt_engine: google
tts_engine: google-tts
google-tts:
  language: pt
keys:
  GOOGLE_SPEECH: "<google api key>"
wrboliveira commented 7 years ago

Obrigado desta forma que você postou eu consegui a comunicação com google!!! Obrigado

Audio= Ingles Escrita= portugues

Preciso que ele funcione assim. 👍

Audio= Portugues - brasil

Escrita= portugues - brasil

Quero montar um assistente pessoal para automação e integração com minha home assim preciso que seja em portugues brasileiro.

Queria usar outro motor de fala que fosse offline mas nemhum que eu vi me da suporte para o brasil a voz e escrita brasileira, se souber como posso andar com esse projeto dentro do modelo brasileiro, agradeço a ajuda.

Mesmo assim muito obrigado pela ajuda!!!!!! G10DRAS

odouglsantos commented 7 years ago

Tente colocar stt_google: google-stt. Infelizmente não existe nenhum modelo de voz open souce com suporte ao pt-br. O que indico é você criar seu modelo, existe um projeto chamado Fala Brasil da UFPA, talvez te ajude a ter um norte para conseguir utilizar offline, eles já disponibilizam um modelo acústico recomendo que crie o seu. http://labvis.ufpa.br/falabrasil/simonbr.php

wrboliveira commented 7 years ago

Legal, dei uma olhada no http://labvis.ufpa.br/falabrasil/simonbr.php e segui o procedimento mas a arquitetura não é compatível, o pacote deb é i386.

Você já usou o Simonbr? se sim como o fez?

G10DRAS commented 7 years ago

SimonBR looks good, see if you can use Brazillian Acoustic Model 'Constituicao.cd_cont_1000_16' with Jasper. They also build Language Model (LaPSLM v1.0) and dictionary (UFPAdic3.0). What you also need is offline version of grapheme-phoneme converter (LaPSG2P) to use with Jasper. If all this working Jasper I think you are good to go offline with Jasper for pt-br.

wrboliveira commented 7 years ago

Olá G10DRAS Fiz a busca dos arquivos necessários - Constituicao.cd_cont_1000_16 - LaPSLM v1.0 - UFPAdic3.0 - (LaPSG2P --> Esse não funciona!!!)

Montei as pastas .....

/usr/share/pocketsphinx/model/hmm/pt_BR/brasil /usr/share/pocketsphinx/model/lm/pt_BR

/home/pi/sphinxbase-0.8/pocketsphinx-0.8/model/lm/pt_BR ---> /nessa pasta falta o arquivo (.DMP) não sei como gerar esse arquivo?

/home/pi/sphinxbase-0.8/pocketsphinx-0.8/model/hmm/pt_BR/brasil

Editei os arquivos (makefile) inserindo os caminhos.

Coloquei o caminho no profile.yml

stt_engine: sphinx language: pt_BR pockectsphinx: fst_model: home/phonetisaurus/g014b2b.fst hmm_dir: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil

O jasper funciona mas não fala comigo!!!!

A mensagem do inicio ele fala mas em en_US!!!!!

//////////////////////////////////////////////////////////////////////

O que falta? Como gerar o arquivo .DMP? Existe algum procedimento correto para a instalação da linguagem no sphinx?

G10DRAS commented 7 years ago

@wrboliveira, It is better if you can try Acoustic Model, Dictionary and Language Model first with pocketsphinx_coninuous, if it works, then we can say all this work with Jasper.

We also need a FST model for brasilian dictionary, g014b2b.fst will not work in this case. I have generated one FST model for German dict in the past, let me see if I can do the same for Brasilian dict.

wrboliveira commented 7 years ago

@G10DRAS , Ok vou estudar aqui a forma que você mostrou. (pocketsphinx_coninuous)

Obrigado pela colaboração!!

wrboliveira commented 7 years ago

Olá @G10DRAS teve algum resultado com FST DICT BRASILEIRO....

Estou tentando usar o (pocketsphinx_coninuous) mas acho que etou um pouco perdido!!! O Jasper funciona mas não transcreve e nem fala!!!!

Apenas a mensagem do começo!!!!

Bom se tiver alguma novidade ou um tuto de como usar o (pocketsphinx_coninuous)!!!!

G10DRAS commented 7 years ago

well you can change and try following command:

pocketsphinx_continuous -hmm <path to acoustic model dir> -lm <path to language model file> -dict <path to dictionary file> -samprate 16000 -inmic yes -adcdev plughw:1,0

wrboliveira commented 7 years ago

Olá @G10DRAS obrigado pelo retorno,

Esse comando que você me mandou ele cria os arquivos?

quais os caminhos corretos para os arquivos?
wrboliveira commented 7 years ago

pi@raspberrypi:~/pocketsphinx/model $ pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil -lm /usr/share/pocketsphinx/model/lm/pt_BR -dict /usr/share/pocketsphinx/model/lm/pt_BR -samprate 16000 -inmic yes -adcdev plughw:1,0 INFO: cmd_ln.c(691): Parsing command line: pocketsphinx_continuous \ -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil \ -lm /usr/share/pocketsphinx/model/lm/pt_BR \ -dict /usr/share/pocketsphinx/model/lm/pt_BR \ -samprate 16000 \ -inmic yes \ -adcdev plughw:1,0

Current configuration: [NAME] [DEFLT] [VALUE] -adcdev plughw:1,0 -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -argfile
-ascale 20.0 2.000000e+01 -aw 1 1 -backtrace no no -beam 1e-48 1.000000e-48 -bestpath yes yes -bestpathlw 9.5 9.500000e+00 -bghist no no -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -compallsen no no -debug 0 -dict /usr/share/pocketsphinx/model/lm/pt_BR -dictcase no no -dither no no -doublebw no no -ds 1 1 -fdict
-feat 1s_c_d_dd 1s_c_d_dd -featparams
-fillprob 1e-8 1.000000e-08 -frate 100 100 -fsg
-fsgusealtpron yes yes -fsgusefiller yes yes -fwdflat yes yes -fwdflatbeam 1e-64 1.000000e-64 -fwdflatefwid 4 4 -fwdflatlw 8.5 8.500000e+00 -fwdflatsfwin 25 25 -fwdflatwbeam 7e-29 7.000000e-29 -fwdtree yes yes -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil -infile
-input_endian little little -jsgf
-kdmaxbbi -1 -1 -kdmaxdepth 0 0 -kdtree
-latsize 5000 5000 -lda
-ldadim 0 0 -lextreedump 0 0 -lifter 0 0 -lm /usr/share/pocketsphinx/model/lm/pt_BR -lmctl
-lmname default default -logbase 1.0001 1.000100e+00 -logfn
-logspec no no -lowerf 133.33334 1.333333e+02 -lpbeam 1e-40 1.000000e-40 -lponlybeam 7e-29 7.000000e-29 -lw 6.5 6.500000e+00 -maxhmmpf -1 -1 -maxnewoov 20 20 -maxwpf -1 -1 -mdef
-mean
-mfclogdir
-min_endfr 0 0 -mixw
-mixwfloor 0.0000001 1.000000e-07 -mllr
-mmap yes yes -ncep 13 13 -nfft 512 512 -nfilt 40 40 -nwpen 1.0 1.000000e+00 -pbeam 1e-48 1.000000e-48 -pip 1.0 1.000000e+00 -pl_beam 1e-10 1.000000e-10 -pl_pbeam 1e-5 1.000000e-05 -pl_window 0 0 -rawlogdir
-remove_dc no no -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03 -smoothspec no no -svspec
-time no no -tmat
-tmatfloor 0.0001 1.000000e-04 -topn 4 4 -topn_beam 0 0 -toprule
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -usewdphones no no -uw 1.0 1.000000e+00 -var
-varfloor 0.0001 1.000000e-04 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wbeam 7e-29 7.000000e-29 -wip 0.65 6.500000e-01 -wlen 0.025625 2.562500e-02

INFO: cmd_ln.c(691): Parsing command line: \ -alpha 0.97 \ -doublebw no \ -nfilt 40 \ -ncep 13 \ -lowerf 133.33334 \ -upperf 6855.4976 \ -nfft 512 \ -wlen 0.0256 \ -transform legacy \ -feat 1s_c_d_dd \ -agc none \ -cmn current \ -varnorm no

Current configuration: [NAME] [DEFLT] [VALUE] -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -dither no no -doublebw no no -feat 1s_c_d_dd 1s_c_d_dd -frate 100 100 -input_endian little little -lda
-ldadim 0 0 -lifter 0 0 -logspec no no -lowerf 133.33334 1.333333e+02 -ncep 13 13 -nfft 512 512 -nfilt 40 40 -remove_dc no no -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -smoothspec no no -svspec
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wlen 0.025625 2.560000e-02

INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/feat.params INFO: feat.c(713): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none' INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0 INFO: mdef.c(517): Reading model definition: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/mdef INFO: bin_mdef.c(179): Allocating 96626 8 bytes (754 KiB) for CD tree INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/transition_matrices INFO: acmod.c(121): Attempting to use SCHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: acmod.c(123): Attempting to use PTHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: ptm_mgau.c(792): Number of codebooks exceeds 256: 1117 INFO: acmod.c(125): Falling back to general multi-stream GMM computation INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: ms_senone.c(149): Reading senone mixture weights: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/mixture_weights INFO: ms_senone.c(200): Truncating senone logs3(pdf) values by 10 bits INFO: ms_senone.c(207): Not transposing mixture weights in memory INFO: ms_senone.c(266): Read mixture weights for 1117 senones: 1 features x 16 codewords INFO: ms_senone.c(320): Mapping senones to individual codebooks INFO: ms_mgau.c(141): The value of topn: 4 INFO: dict.c(317): Allocating 4099 20 bytes (80 KiB) for word entries INFO: dict.c(332): Reading main dictionary: /usr/share/pocketsphinx/model/lm/pt_BR INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(335): 0 words read INFO: dict.c(341): Reading filler dictionary: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/noisedict INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(344): 3 words read INFO: dict2pid.c(396): Building PID tables for dictionary INFO: dict2pid.c(404): Allocating 39^3 * 2 bytes (115 KiB) for word-initial triphones INFO: dict2pid.c(131): Allocated 18408 bytes (17 KiB) for word-final triphones INFO: dict2pid.c(195): Allocated 18408 bytes (17 KiB) for single-phone word triphones INFO: ngram_model_arpa.c(77): No \data\ mark in LM file ERROR: "ngram_search.c", line 208: Failed to read language model file: /usr/share/pocketsphinx/model/lm/pt_BR

wrboliveira commented 7 years ago

@G10DRAS

Verifique veja se estou errando em algo por favor.

sudo pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil -lm /usr/share/pocketsphinx/model/lm/pt_BR/3gram.lm -dict /usr/share/pocketsphinx/model/lm/pt_BR -samprate 16000 -inmic yes -adcdev plughw:1
INFO: cmd_ln.c(691): Parsing command line: pocketsphinx_continuous \ -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil \ -lm /usr/share/pocketsphinx/model/lm/pt_BR/3gram.lm \ -dict /usr/share/pocketsphinx/model/lm/pt_BR \ -samprate 16000 \ -inmic yes \ -adcdev plughw:1

Current configuration: [NAME] [DEFLT] [VALUE] -adcdev plughw:1 -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -argfile
-ascale 20.0 2.000000e+01 -aw 1 1 -backtrace no no -beam 1e-48 1.000000e-48 -bestpath yes yes -bestpathlw 9.5 9.500000e+00 -bghist no no -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -compallsen no no -debug 0 -dict /usr/share/pocketsphinx/model/lm/pt_BR -dictcase no no -dither no no -doublebw no no -ds 1 1 -fdict
-feat 1s_c_d_dd 1s_c_d_dd -featparams
-fillprob 1e-8 1.000000e-08 -frate 100 100 -fsg
-fsgusealtpron yes yes -fsgusefiller yes yes -fwdflat yes yes -fwdflatbeam 1e-64 1.000000e-64 -fwdflatefwid 4 4 -fwdflatlw 8.5 8.500000e+00 -fwdflatsfwin 25 25 -fwdflatwbeam 7e-29 7.000000e-29 -fwdtree yes yes -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil -infile
-input_endian little little -jsgf
-kdmaxbbi -1 -1 -kdmaxdepth 0 0 -kdtree
-latsize 5000 5000 -lda
-ldadim 0 0 -lextreedump 0 0 -lifter 0 0 -lm /usr/share/pocketsphinx/model/lm/pt_BR/3gram.lm -lmctl
-lmname default default -logbase 1.0001 1.000100e+00 -logfn
-logspec no no -lowerf 133.33334 1.333333e+02 -lpbeam 1e-40 1.000000e-40 -lponlybeam 7e-29 7.000000e-29 -lw 6.5 6.500000e+00 -maxhmmpf -1 -1 -maxnewoov 20 20 -maxwpf -1 -1 -mdef
-mean
-mfclogdir
-min_endfr 0 0 -mixw
-mixwfloor 0.0000001 1.000000e-07 -mllr
-mmap yes yes -ncep 13 13 -nfft 512 512 -nfilt 40 40 -nwpen 1.0 1.000000e+00 -pbeam 1e-48 1.000000e-48 -pip 1.0 1.000000e+00 -pl_beam 1e-10 1.000000e-10 -pl_pbeam 1e-5 1.000000e-05 -pl_window 0 0 -rawlogdir
-remove_dc no no -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03 -smoothspec no no -svspec
-time no no -tmat
-tmatfloor 0.0001 1.000000e-04 -topn 4 4 -topn_beam 0 0 -toprule
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -usewdphones no no -uw 1.0 1.000000e+00 -var
-varfloor 0.0001 1.000000e-04 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wbeam 7e-29 7.000000e-29 -wip 0.65 6.500000e-01 -wlen 0.025625 2.562500e-02

INFO: cmd_ln.c(691): Parsing command line: \ -alpha 0.97 \ -doublebw no \ -nfilt 40 \ -ncep 13 \ -lowerf 133.33334 \ -upperf 6855.4976 \ -nfft 512 \ -wlen 0.0256 \ -transform legacy \ -feat 1s_c_d_dd \ -agc none \ -cmn current \ -varnorm no

Current configuration: [NAME] [DEFLT] [VALUE] -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -dither no no -doublebw no no -feat 1s_c_d_dd 1s_c_d_dd -frate 100 100 -input_endian little little -lda
-ldadim 0 0 -lifter 0 0 -logspec no no -lowerf 133.33334 1.333333e+02 -ncep 13 13 -nfft 512 512 -nfilt 40 40 -remove_dc no no -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -smoothspec no no -svspec
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wlen 0.025625 2.560000e-02

INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/feat.params INFO: feat.c(713): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none' INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0 INFO: mdef.c(517): Reading model definition: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/mdef INFO: bin_mdef.c(179): Allocating 96626 8 bytes (754 KiB) for CD tree INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/transition_matrices INFO: acmod.c(121): Attempting to use SCHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: acmod.c(123): Attempting to use PTHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: ptm_mgau.c(792): Number of codebooks exceeds 256: 1117 INFO: acmod.c(125): Falling back to general multi-stream GMM computation INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: ms_senone.c(149): Reading senone mixture weights: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/mixture_weights INFO: ms_senone.c(200): Truncating senone logs3(pdf) values by 10 bits INFO: ms_senone.c(207): Not transposing mixture weights in memory INFO: ms_senone.c(266): Read mixture weights for 1117 senones: 1 features x 16 codewords INFO: ms_senone.c(320): Mapping senones to individual codebooks INFO: ms_mgau.c(141): The value of topn: 4 INFO: dict.c(317): Allocating 4099 20 bytes (80 KiB) for word entries INFO: dict.c(332): Reading main dictionary: /usr/share/pocketsphinx/model/lm/pt_BR INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(335): 0 words read INFO: dict.c(341): Reading filler dictionary: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/noisedict INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(344): 3 words read INFO: dict2pid.c(396): Building PID tables for dictionary INFO: dict2pid.c(404): Allocating 39^3 * 2 bytes (115 KiB) for word-initial triphones INFO: dict2pid.c(131): Allocated 18408 bytes (17 KiB) for word-final triphones INFO: dict2pid.c(195): Allocated 18408 bytes (17 KiB) for single-phone word triphones INFO: ngram_model_arpa.c(477): ngrams 1=64966, 2=3515726, 3=2359696 INFO: ngram_model_arpa.c(135): Reading unigrams INFO: ngram_model_arpa.c(516): 64966 = #unigrams created INFO: ngram_model_arpa.c(195): Reading bigrams .....................................................INFO: ngram_model_arpa.c(533): 3515726 = #bigrams created INFO: ngram_model_arpa.c(534): 35817 = #prob2 entries INFO: ngram_model_arpa.c(542): 11679 = #bo_wt2 entries INFO: ngram_model_arpa.c(292): Reading trigrams ERROR: "ngram_model_arpa.c", line 351: Trigrams not in bigram order ERROR: "ngram_model_dmp.c", line 121: Wrong magic header size number a5c6461: /usr/share/pocketsphinx/model/lm/pt_BR/3gram.lm is not a dump file ERROR: "ngram_search.c", line 208: Failed to read language model file: /usr/share/pocketsphinx/model/lm/pt_BR/3gram.lm

wrboliveira commented 7 years ago

@G10DRAS Olá acredito que houve uma evolução aqui rsrs de uma olhada o que acha Como devo prosseguir agora?

sudo pocketsphinx_continuous -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil -lm /usr/share/pocketsphinx/model/lm/pt_BR/lapsam.sorted.lm -dict /usr/share/pocketsphinx/model/lm/pt_BR -samprate 16000 -inmic yes -adcdev plughw:1,0 INFO: cmd_ln.c(691): Parsing command line: pocketsphinx_continuous \ -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil \ -lm /usr/share/pocketsphinx/model/lm/pt_BR/lapsam.sorted.lm \ -dict /usr/share/pocketsphinx/model/lm/pt_BR \ -samprate 16000 \ -inmic yes \ -adcdev plughw:1,0

Current configuration: [NAME] [DEFLT] [VALUE] -adcdev plughw:1,0 -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -argfile
-ascale 20.0 2.000000e+01 -aw 1 1 -backtrace no no -beam 1e-48 1.000000e-48 -bestpath yes yes -bestpathlw 9.5 9.500000e+00 -bghist no no -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -compallsen no no -debug 0 -dict /usr/share/pocketsphinx/model/lm/pt_BR -dictcase no no -dither no no -doublebw no no -ds 1 1 -fdict
-feat 1s_c_d_dd 1s_c_d_dd -featparams
-fillprob 1e-8 1.000000e-08 -frate 100 100 -fsg
-fsgusealtpron yes yes -fsgusefiller yes yes -fwdflat yes yes -fwdflatbeam 1e-64 1.000000e-64 -fwdflatefwid 4 4 -fwdflatlw 8.5 8.500000e+00 -fwdflatsfwin 25 25 -fwdflatwbeam 7e-29 7.000000e-29 -fwdtree yes yes -hmm /usr/share/pocketsphinx/model/hmm/pt_BR/brasil -infile
-input_endian little little -jsgf
-kdmaxbbi -1 -1 -kdmaxdepth 0 0 -kdtree
-latsize 5000 5000 -lda
-ldadim 0 0 -lextreedump 0 0 -lifter 0 0 -lm /usr/share/pocketsphinx/model/lm/pt_BR/lapsam.sorted.lm -lmctl
-lmname default default -logbase 1.0001 1.000100e+00 -logfn
-logspec no no -lowerf 133.33334 1.333333e+02 -lpbeam 1e-40 1.000000e-40 -lponlybeam 7e-29 7.000000e-29 -lw 6.5 6.500000e+00 -maxhmmpf -1 -1 -maxnewoov 20 20 -maxwpf -1 -1 -mdef
-mean
-mfclogdir
-min_endfr 0 0 -mixw
-mixwfloor 0.0000001 1.000000e-07 -mllr
-mmap yes yes -ncep 13 13 -nfft 512 512 -nfilt 40 40 -nwpen 1.0 1.000000e+00 -pbeam 1e-48 1.000000e-48 -pip 1.0 1.000000e+00 -pl_beam 1e-10 1.000000e-10 -pl_pbeam 1e-5 1.000000e-05 -pl_window 0 0 -rawlogdir
-remove_dc no no -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -sendump
-senlogdir
-senmgau
-silprob 0.005 5.000000e-03 -smoothspec no no -svspec
-time no no -tmat
-tmatfloor 0.0001 1.000000e-04 -topn 4 4 -topn_beam 0 0 -toprule
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -usewdphones no no -uw 1.0 1.000000e+00 -var
-varfloor 0.0001 1.000000e-04 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wbeam 7e-29 7.000000e-29 -wip 0.65 6.500000e-01 -wlen 0.025625 2.562500e-02

INFO: cmd_ln.c(691): Parsing command line: \ -alpha 0.97 \ -doublebw no \ -nfilt 40 \ -ncep 13 \ -lowerf 133.33334 \ -upperf 6855.4976 \ -nfft 512 \ -wlen 0.0256 \ -transform legacy \ -feat 1s_c_d_dd \ -agc none \ -cmn current \ -varnorm no

Current configuration: [NAME] [DEFLT] [VALUE] -agc none none -agcthresh 2.0 2.000000e+00 -alpha 0.97 9.700000e-01 -ceplen 13 13 -cmn current current -cmninit 8.0 8.0 -dither no no -doublebw no no -feat 1s_c_d_dd 1s_c_d_dd -frate 100 100 -input_endian little little -lda
-ldadim 0 0 -lifter 0 0 -logspec no no -lowerf 133.33334 1.333333e+02 -ncep 13 13 -nfft 512 512 -nfilt 40 40 -remove_dc no no -round_filters yes yes -samprate 16000 1.600000e+04 -seed -1 -1 -smoothspec no no -svspec
-transform legacy legacy -unit_area yes yes -upperf 6855.4976 6.855498e+03 -varnorm no no -verbose no no -warp_params
-warp_type inverse_linear inverse_linear -wlen 0.025625 2.560000e-02

INFO: acmod.c(246): Parsed model-specific feature parameters from /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/feat.params INFO: feat.c(713): Initializing feature stream to type: '1s_c_d_dd', ceplen=13, CMN='current', VARNORM='no', AGC='none' INFO: cmn.c(142): mean[0]= 12.00, mean[1..12]= 0.0 INFO: mdef.c(517): Reading model definition: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/mdef INFO: bin_mdef.c(179): Allocating 96626 8 bytes (754 KiB) for CD tree INFO: tmat.c(205): Reading HMM transition probability matrices: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/transition_matrices INFO: acmod.c(121): Attempting to use SCHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: acmod.c(123): Attempting to use PTHMM computation module INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: ptm_mgau.c(792): Number of codebooks exceeds 256: 1117 INFO: acmod.c(125): Falling back to general multi-stream GMM computation INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/means INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(198): Reading mixture gaussian parameter: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/variances INFO: ms_gauden.c(292): 1117 codebook, 1 feature, size: INFO: ms_gauden.c(294): 16x39 INFO: ms_gauden.c(354): 702 variance values floored INFO: ms_senone.c(149): Reading senone mixture weights: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/mixture_weights INFO: ms_senone.c(200): Truncating senone logs3(pdf) values by 10 bits INFO: ms_senone.c(207): Not transposing mixture weights in memory INFO: ms_senone.c(266): Read mixture weights for 1117 senones: 1 features x 16 codewords INFO: ms_senone.c(320): Mapping senones to individual codebooks INFO: ms_mgau.c(141): The value of topn: 4 INFO: dict.c(317): Allocating 4099 20 bytes (80 KiB) for word entries INFO: dict.c(332): Reading main dictionary: /usr/share/pocketsphinx/model/lm/pt_BR INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(335): 0 words read INFO: dict.c(341): Reading filler dictionary: /usr/share/pocketsphinx/model/hmm/pt_BR/brasil/noisedict INFO: dict.c(211): Allocated 0 KiB for strings, 0 KiB for phones INFO: dict.c(344): 3 words read INFO: dict2pid.c(396): Building PID tables for dictionary INFO: dict2pid.c(404): Allocating 39^3 * 2 bytes (115 KiB) for word-initial triphones INFO: dict2pid.c(131): Allocated 18408 bytes (17 KiB) for word-final triphones INFO: dict2pid.c(195): Allocated 18408 bytes (17 KiB) for single-phone word triphones INFO: ngram_model_arpa.c(477): ngrams 1=65722, 2=2053001, 3=5843214 INFO: ngram_model_arpa.c(135): Reading unigrams INFO: ngram_model_arpa.c(516): 65722 = #unigrams created INFO: ngram_model_arpa.c(195): Reading bigrams ...............................INFO: ngram_model_arpa.c(533): 2053001 = #bigrams created INFO: ngram_model_arpa.c(534): 45385 = #prob2 entries INFO: ngram_model_arpa.c(542): 8016 = #bo_wt2 entries INFO: ngram_model_arpa.c(292): Reading trigrams .........................................................................................INFO: ngram_model_arpa.c(555): 5843214 = #trigrams created INFO: ngram_model_arpa.c(556): 47577 = #prob3 entries INFO: ngram_search_fwdtree.c(99): 0 unique initial diphones INFO: ngram_search_fwdtree.c(147): 0 root, 0 non-root channels, 4 single-phone words INFO: ngram_search_fwdtree.c(186): Creating search tree INFO: ngram_search_fwdtree.c(191): before: 0 root, 0 non-root channels, 4 single-phone words INFO: ngram_search_fwdtree.c(326): after: max nonroot chan increased to 128 ERROR: "ngram_search_fwdtree.c", line 336: No word from the language model has pronunciation in the dictionary INFO: ngram_search_fwdtree.c(338): after: 0 root, 0 non-root channels, 3 single-phone words INFO: ngram_search_fwdflat.c(156): fwdflat: min_ef_width = 4, max_sf_win = 25 INFO: continuous.c(371): pocketsphinx_continuous COMPILED ON: Aug 12 2017, AT: 20:06:37

Warning: Could not find Capture element READY....

G10DRAS commented 7 years ago

Ok I have downloaded AM,LM, Dict from falabrasil. Orignal site is down it seems. And it works.....

pocketsphinx_continuous -hmm constituicao.cd_cont_1000 -lm 3gram.lm -dict constituicao.dic -samprate 16000 -inmic yes -adcdev plughw:1,0 -logfn /dev/null

um guia
é um
ilya
brava num
o abandono
amato ao ar
cada povo
análise e um abandono
e deu um
velho
e hélio
méier
rio
em rio
a daimler
criam view um views
view

For Jasper to work with Brasilian Model, all you need is a FST model train using brasilan dict constituicao.dic

G10DRAS commented 7 years ago

Finally, G2P Model (FST) for Brasilian Dict is ready. Download it from g2p

If you use it with Jasper, you need small code changes in Jasper else test it with Phonetisaurus as follow

phonetisaurus-g2p -model constituicao.fst -words yes -isfile yes -input constituicaotest.txt --output constituicaotest.dict

wrboliveira commented 7 years ago

Olá @G10DRAS Obrigado pela contribuição acredito que será de bom uso por todos que precisarem deste pt_BR.

Gostaria de uma ajuda para inserir este modelo no jasper!!! Quais mudanças devo efetuar no código Jasper?
Como inserir os arquivos e onde inserir os arquivos?

Prefiro perguntar para não fazer algo errado, você consegue me orientar por favor!

G10DRAS commented 7 years ago

how about pocketsphinx_continuous, was it worked for you ?

Brasilian model will work in similar fashion as US model in Jasper.

profile.yml will looks like:

stt_engine: sphinx
language: pt_BR
pockectsphinx:
  fst_model: /home/brasil/constituicao.fst
  hmm_dir: /home/brasil/constituicao.cd_cont_1000

and you need code changes in jasper-client/client/g2p.py line 15-16

# PATTERN = re.compile(r'^(?P<word>.+)\t(?P<precision>\d+\.\d+)\t<s> ' +
# r'(?P<pronounciation>.*) </s>', re.MULTILINE)

replace it with

 PATTERN = re.compile(
        r'^(?P<word>.+)\t(?P<precision>\d+\.\d+)\t' +
        r'(?P<pronounciation>.*)', 
        re.MULTILINE)

and line 63 # result[word].append(pronounc) replace it with result[word].append(pronounc.lower())

Thats it. (I guess)

and yes, replace English WORDS in jasper modules with Brailian one. For example HORAS in place of TIME and so on.....

Run Jasper in debug mode. Test... Check Log.... Debug Code.... Test ..................

wrboliveira commented 7 years ago

Ok muito obrigado irei trabalhar aqui e vou postar o resultado.

Novamente obrigado.

G10DRAS commented 7 years ago

Ok, let me know how it goes.

wrboliveira commented 7 years ago

obtenho esse erro. ja revisei o procedimento mas não consegui resolver o erro.

pi@raspberrypi:~/jasper $ python jasper.py --debug

G10DRAS commented 7 years ago

Looks like some formatting issue with you profile.yml. Test it here http://www.yamllint.com/

wrboliveira commented 7 years ago

Teste após alterar com o profile.yml http://www.yamllint.com/

Já revisei as alterações antes dos erros está conforme orientação.

Meu profile funcionava antes de alterar g2P.py.

Após alteração de g2p.py esse erro persiste.

pi@raspberrypi:~/jasper $ python jasper.py --debug

G10DRAS commented 7 years ago

where is your profile.yml ?

wrboliveira commented 7 years ago

Meu profile.yml carrier: '' first_name: Willian gmail_address: gmail_password: last_name: Oliveira location: brasil phone_number: prefers_email: true timezone: America/Sao_Paulo stt_engine: sphinx pockectsphinx: fst_model: /home/pi/brasil/constituiçao.fst hmm_dir: /usr/share/pocketsphinx/model/hmm/pt_BR/constituicao.cd_cont_1000

wrboliveira commented 7 years ago

Olá @G10DRAS Bom o que eu descobri foi.

Eu rodei populate.py e inseri novamente as informações e o jasper funcionou o profile.yml não estava com as informações do file path hmm e fst.

Quando eu coloco o file path no profile.yml o erro volta a acontecer.

///////////////////////////////////////////////////////////////////////////////////////// st_model: /home/pi/brasil/constituiçao.fst hmm_dir: /usr/share/pocketsphinx/model/hmm/pt_BR/constituicao.cd_cont_1000 ///////////////////////////////////////////////////////////////////////////////////////// erro profile.yml

Acredito que esteja acontecendo algum erro na decodificação mas não consigo encontrar um local exato para alterar.

O que você acha ?

wrboliveira commented 7 years ago

Bom @G10DRAS estas forma minhas ultimas tentativas em compilar o novo modelo. Acredito que sua ajuda é necessária. rsrs Segue --debug

pi@raspberrypi:~/jasper $ python jasper.py --debug

wrboliveira commented 7 years ago

i@raspberrypi:~/jasper $ python jasper.py

  • JASPER - THE TALKING COMPUTER *
  • (c) 2015 Shubhro Saha, Charlie Marsh & Jan Holthuis *

    WARNING:root:tts_engine not specified in profile, defaulting to 'espeak-tts' ERROR:client.g2p:Command 'phonetisaurus-g2p --model=/home/pi/brasil/constituicao.fst --input=/tmp/tmpuoApeS.g2p --words --isfile' return with exit status -6 ERROR:client.vocabcompiler:Fatal compilation Error occured, cleaning up... Traceback (most recent call last): File "/home/pi/jasper/client/vocabcompiler.py", line 163, in compile self._compile_vocabulary(phrases) File "/home/pi/jasper/client/vocabcompiler.py", line 271, in _compile_vocabulary self._compile_dictionary(vocabulary, self.dictionary_file) File "/home/pi/jasper/client/vocabcompiler.py", line 321, in _compile_dictionary phonemes = g2pconverter.translate(words) File "/home/pi/jasper/client/g2p.py", line 132, in translate output = self._translate_words(words) File "/home/pi/jasper/client/g2p.py", line 121, in _translate_words nbest=self.nbest) File "/home/pi/jasper/client/g2p.py", line 58, in execute raise OSError("Command execution failed") OSError: Command execution failed ERROR:root:Error occured! Traceback (most recent call last): File "jasper.py", line 146, in app = Jasper() File "jasper.py", line 109, in init stt_passive_engine_class.get_passive_instance(), File "/home/pi/jasper/client/stt.py", line 48, in get_passive_instance return cls.get_instance('keyword', phrases) File "/home/pi/jasper/client/stt.py", line 40, in get_instance vocabulary.compile(phrases) File "/home/pi/jasper/client/vocabcompiler.py", line 171, in compile raise e OSError: Command execution failed

wrboliveira commented 7 years ago

Bom jasper inicia com o novo idioma mas não responde e não transcreve

~/jasper $ python jasper.py --debug

  • JASPER - THE TALKING COMPUTER *
  • (c) 2015 Shubhro Saha, Charlie Marsh & Jan Holthuis *

    DEBUG:client.diagnose:Checking network connection to server 'www.google.com'... DEBUG:client.diagnose:Network connection working DEBUG:main:Trying to read config file: '/home/pi/.jasper/profile.yml' DEBUG:client.diagnose:Checking python import 'pocketsphinx'... DEBUG:client.diagnose:Python module 'pocketsphinx' found: '/usr/local/lib/python2.7/dist-packages/pocketsphinx.so' WARNING:root:tts_engine not specified in profile, defaulting to 'espeak-tts' DEBUG:client.diagnose:Checking executable 'aplay'... DEBUG:client.diagnose:Executable 'aplay' found: '/usr/bin/aplay' DEBUG:client.diagnose:Checking executable 'espeak'... DEBUG:client.diagnose:Executable 'espeak' found: '/usr/bin/espeak' DEBUG:client.vocabcompiler:compiled_revision is '7dac5eef56d256443a0a2389ce11be23170bfc7c' DEBUG:client.brain:Looking for modules in: '/home/pi/jasper/client/modules' DEBUG:client.brain:Found module 'Aniversario' with words: ['ANIVERSARIO'] DEBUG:client.brain:Found module 'Clima' with words: ['CLIMA', 'HOJE', 'AMANHA'] DEBUG:client.brain:Found module 'Gmail' with words: ['EMAIL', 'ENTRADA'] DEBUG:client.brain:Found module 'HN' with words: ['HACKER', 'NEWS', 'SIM', 'NAO', 'PRIMEIRO', 'SEGUNDO', 'TERCEIRO'] DEBUG:client.brain:Found module 'Horas' with words: ['HORAS'] DEBUG:client.brain:Found module 'Kodi2' with words: ['SWITCH', 'ON', 'OFF', 'TV', 'PLAY'] DEBUG:client.brain:Found module 'MPDControl' with words: ['MUSIC', 'SPOTIFY'] DEBUG:client.brain:Found module 'News' with words: ['NEWS', 'SIM', 'NAO', 'PRIMEIRO', 'SEGUNDO', 'TERCEIRO'] DEBUG:client.brain:Found module 'Notificaçao' with words: ['FACEBOOK', 'NOTIFICATION'] DEBUG:client.brain:Found module 'Piadas' with words: ['JOKE', 'KNOCK KNOCK'] DEBUG:client.brain:Found module 'Unclear' with words: [] DEBUG:client.brain:Found module 'Vida' with words: ['SIGNIFICADO', 'DA', 'VIDA'] DEBUG:client.brain:Found module 'Weather' with words: ['CLIMA', 'HOJE', 'AMANHA'] DEBUG:client.brain:Found module 'kodi' with words: ['MEDIA', 'BACK', 'PLAY', 'PAUSE', 'STOP', 'SELECT', 'INFO', 'UP', 'DOWN', 'START'] DEBUG:client.vocabcompiler:compiled_revision is 'fddd539dea1977bc984418e3679d8fb630f51bb4' INFO:client.mic:Initializing PyAudio. ALSA/Jack error messages that pop up during this process are normal and can usually be safely ignored. INFO:client.mic:Initialization of PyAudio completed. DEBUG:client.tts:Saying 'Como Posso Ajudar, willian?' with 'espeak-tts' DEBUG:client.tts:Executing espeak -v default+m3 -p 40 -s 160 -w /tmp/tmp358vjQ.wav 'Como Posso Ajudar, willian?' DEBUG:client.tts:Executing aplay -D plughw:1,0 /tmp/tmp358vjQ.wav DEBUG:client.tts:Output was: 'Playing WAVE '/tmp/tmp358vjQ.wav' : Signed 16 bit Little Endian, Rate 22050 Hz, Mono ' DEBUG:client.brain:Looking for modules in: '/home/pi/jasper/client/modules' DEBUG:client.brain:Found module 'Aniversario' with words: ['ANIVERSARIO'] DEBUG:client.brain:Found module 'Clima' with words: ['CLIMA', 'HOJE', 'AMANHA'] DEBUG:client.brain:Found module 'Gmail' with words: ['EMAIL', 'ENTRADA'] DEBUG:client.brain:Found module 'HN' with words: ['HACKER', 'NEWS', 'SIM', 'NAO', 'PRIMEIRO', 'SEGUNDO', 'TERCEIRO'] DEBUG:client.brain:Found module 'Horas' with words: ['HORAS'] DEBUG:client.brain:Found module 'Kodi2' with words: ['SWITCH', 'ON', 'OFF', 'TV', 'PLAY'] DEBUG:client.brain:Found module 'MPDControl' with words: ['MUSIC', 'SPOTIFY'] DEBUG:client.brain:Found module 'News' with words: ['NEWS', 'SIM', 'NAO', 'PRIMEIRO', 'SEGUNDO', 'TERCEIRO'] DEBUG:client.brain:Found module 'Notificaçao' with words: ['FACEBOOK', 'NOTIFICATION'] DEBUG:client.brain:Found module 'Piadas' with words: ['JOKE', 'KNOCK KNOCK'] DEBUG:client.brain:Found module 'Unclear' with words: [] DEBUG:client.brain:Found module 'Vida' with words: ['SIGNIFICADO', 'DA', 'VIDA'] DEBUG:client.brain:Found module 'Weather' with words: ['CLIMA', 'HOJE', 'AMANHA'] DEBUG:client.brain:Found module 'kodi' with words: ['MEDIA', 'BACK', 'PLAY', 'PAUSE', 'STOP', 'SELECT', 'INFO', 'UP', 'DOWN', 'START'] INFO:apscheduler.scheduler:Scheduler started DEBUG:apscheduler.scheduler:Looking for jobs to run DEBUG:apscheduler.scheduler:No jobs; waiting until a job is added INFO:apscheduler.scheduler:Added job "Notifier.gather" to job store "default" INFO:client.conversation:Starting to handle conversation with keyword 'JASPER'. DEBUG:client.conversation:Started listening for keyword 'JASPER' DEBUG:apscheduler.scheduler:Looking for jobs to run DEBUG:apscheduler.scheduler:Next wakeup is due at 2017-08-24 03:58:55.263628+00:00 (in 29.981209 seconds) INFO:client.stt:Transcribed: [''] DEBUG:client.conversation:Stopped listening for keyword 'JASPER' INFO:client.conversation:Nothing has been said or transcribed. DEBUG:client.conversation:Started listening for keyword 'JASPER' INFO:client.stt:Transcribed: [''] DEBUG:client.conversation:Stopped listening for keyword 'JASPER' INFO:client.conversation:Nothing has been said or transcribed. DEBUG:client.conversation:Started listening for keyword 'JASPER' INFO:client.stt:Transcribed: [''] DEBUG:client.conversation:Stopped listening for keyword 'JASPER' INFO:client.conversation:Nothing has been said or transcribed. DEBUG:client.conversation:Started listening for keyword 'JASPER' INFO:client.stt:Transcribed: [''] DEBUG:client.conversation:Stopped listening for keyword 'JASPER' INFO:client.conversation:Nothing has been said or transcribed. DEBUG:client.conversation:Started listening for keyword 'JASPER' ^CTraceback (most recent call last): File "jasper.py", line 151, in app.run() File "jasper.py", line 121, in run conversation.handleForever() File "/home/pi/jasper/client/conversation.py", line 31, in handleForever threshold, transcribed = self.mic.passiveListen(self.persona) File "/home/pi/jasper/client/mic.py", line 141, in passiveListen data = stream.read(CHUNK) File "/usr/lib/python2.7/dist-packages/pyaudio.py", line 605, in read return pa.read_stream(self._stream, num_frames) KeyboardInterrupt INFO:apscheduler.scheduler:Scheduler has been shut down Error in `python': free(): invalid next size (fast): 0x00c3f570 Abortado

G10DRAS commented 7 years ago

WoW. Now check mic setting. Test recording with arecord. alsamixerfor sound level. ~/.asound.rc for ALSA default card setting etc....

Did you test brasilian model with pocketsphinx_continuous ?

wrboliveira commented 7 years ago

OK, vou verificar as config do mic !!!!!

Me oriente a fazer o teste do pocketsphinx_continuous .......

wrboliveira commented 7 years ago

Novo erro quando rodei o --debug ele trava aqui e não sai disso. jasper $ ./jasper.py --debug

  • JASPER - THE TALKING COMPUTER *
  • (c) 2015 Shubhro Saha, Charlie Marsh & Jan Holthuis *

    DEBUG:client.diagnose:Checking network connection to server 'www.google.com'... DEBUG:client.diagnose:Network connection working DEBUG:main:Trying to read config file: '/home/pi/.jasper/profile.yml' DEBUG:client.diagnose:Checking python import 'pocketsphinx'... DEBUG:client.diagnose:Python module 'pocketsphinx' found: '/usr/local/lib/python2.7/dist-packages/pocketsphinx.so' WARNING:root:tts_engine not specified in profile, defaulting to 'espeak-tts' DEBUG:client.diagnose:Checking executable 'aplay'... DEBUG:client.diagnose:Executable 'aplay' found: '/usr/bin/aplay' DEBUG:client.diagnose:Checking executable 'espeak'... DEBUG:client.diagnose:Executable 'espeak' found: '/usr/bin/espeak' INFO:client.vocabcompiler:Starting compilation... DEBUG:client.vocabcompiler:Compiling languagemodel... DEBUG:client.vocabcompiler:Creating vocab file: '/tmp/tmpqPX8tM.vocab' DEBUG:cmuclmtk:text2wfreq : Reading text from standard input... DEBUG:cmuclmtk:text2wfreq : Done. DEBUG:cmuclmtk:Command 'text2wfreq -hash 1000000 -verbosity 2' returned with exit code '0'. DEBUG:cmuclmtk:wfreq2vocab : Will generate a vocabulary containing the most DEBUG:cmuclmtk:frequent 20000 words. Reading wfreq stream from stdin... DEBUG:cmuclmtk:wfreq2vocab : Done. DEBUG:cmuclmtk:Command 'wfreq2vocab -verbosity 2 -records 1000000' returned with exit code '0'. DEBUG:client.vocabcompiler:Creating languagemodel file: '/home/pi/.jasper/vocabularies/pocketsphinx-vocabulary/keyword/languagemodel' DEBUG:cmuclmtk:text2idngram DEBUG:cmuclmtk:Vocab : /tmp/tmpqPX8tM.vocab DEBUG:cmuclmtk:Output idngram : /tmp/tmpAm_8yb.idngram DEBUG:cmuclmtk:N-gram buffer size : 100 DEBUG:cmuclmtk:Hash table size : 2000000 DEBUG:cmuclmtk:Temp directory : cmuclmtk-UWg37K DEBUG:cmuclmtk:Max open files : 20 DEBUG:cmuclmtk:FOF size : 10 DEBUG:cmuclmtk:n : 3 DEBUG:cmuclmtk:Initialising hash table... DEBUG:cmuclmtk:Reading vocabulary... DEBUG:cmuclmtk:Allocating memory for the n-gram buffer... DEBUG:cmuclmtk:Reading text into the n-gram buffer... DEBUG:cmuclmtk:20,000 n-grams processed for each ".", 1,000,000 for each line. DEBUG:cmuclmtk:Sorting n-grams... DEBUG:cmuclmtk:Writing sorted n-grams to temporary file cmuclmtk-UWg37K/1 DEBUG:cmuclmtk:Merging 1 temporary files... DEBUG:cmuclmtk:2-grams occurring: N times > N times Sug. -spec_num value DEBUG:cmuclmtk:0 94 104 DEBUG:cmuclmtk:1 82 12 22 DEBUG:cmuclmtk:2 8 4 14 DEBUG:cmuclmtk:3 3 1 11 DEBUG:cmuclmtk:4 0 1 11 DEBUG:cmuclmtk:5 0 1 11 DEBUG:cmuclmtk:6 0 1 11 DEBUG:cmuclmtk:7 0 1 11 DEBUG:cmuclmtk:8 0 1 11 DEBUG:cmuclmtk:9 0 1 11 DEBUG:cmuclmtk:10 0 1 11 DEBUG:cmuclmtk:3-grams occurring: N times > N times Sug. -spec_num value DEBUG:cmuclmtk:0 114 125 DEBUG:cmuclmtk:1 107 7 17 DEBUG:cmuclmtk:2 4 3 13 DEBUG:cmuclmtk:3 3 0 10 DEBUG:cmuclmtk:4 0 0 10 DEBUG:cmuclmtk:5 0 0 10 DEBUG:cmuclmtk:6 0 0 10 DEBUG:cmuclmtk:7 0 0 10 DEBUG:cmuclmtk:8 0 0 10 DEBUG:cmuclmtk:9 0 0 10 DEBUG:cmuclmtk:10 0 0 10 DEBUG:cmuclmtk:text2idngram : Done. DEBUG:cmuclmtk:Command 'text2idngram -vocab /tmp/tmpqPX8tM.vocab -idngram /tmp/tmpAm_8yb.idngram -buffer 100 -hash 2000000 -files 20 -verbosity 2 -n 3 -fof_size 10' returned with exit code '0'. DEBUG:cmuclmtk:Warning : OOV fraction specified, but will not be used, since vocab type is not 2. DEBUG:cmuclmtk:n : 3 DEBUG:cmuclmtk:Input file : /tmp/tmpAm_8yb.idngram (binary format) DEBUG:cmuclmtk:Output files : DEBUG:cmuclmtk:ARPA format : /home/pi/.jasper/vocabularies/pocketsphinx-vocabulary/keyword/languagemodel DEBUG:cmuclmtk:Vocabulary file : /tmp/tmpqPX8tM.vocab DEBUG:cmuclmtk:Cutoffs : DEBUG:cmuclmtk:2-gram : 0 3-gram : 0 DEBUG:cmuclmtk:Vocabulary type : Open - type 1 DEBUG:cmuclmtk:Minimum unigram count : 0 DEBUG:cmuclmtk:Zeroton fraction : 1 DEBUG:cmuclmtk:Counts will be stored in two bytes. DEBUG:cmuclmtk:Count table size : 65535 DEBUG:cmuclmtk:Discounting method : Good-Turing DEBUG:cmuclmtk:Discounting ranges : DEBUG:cmuclmtk:1-gram : 1 2-gram : 7 3-gram : 7 DEBUG:cmuclmtk:Memory allocation for tree structure : DEBUG:cmuclmtk:Allocate 100 MB of memory, shared equally between all n-gram tables. DEBUG:cmuclmtk:Back-off weight storage : DEBUG:cmuclmtk:Back-off weights will be stored in four bytes. DEBUG:cmuclmtk:Reading vocabulary. DEBUG:cmuclmtk:read_wlist_into_siht: a list of 49 words was read from "/tmp/tmpqPX8tM.vocab". DEBUG:cmuclmtk:read_wlist_into_array: a list of 49 words was read from "/tmp/tmpqPX8tM.vocab". DEBUG:cmuclmtk:WARNING: appears as a vocabulary item, but is not labelled as a DEBUG:cmuclmtk:context cue. DEBUG:cmuclmtk:Allocated space for 3571428 2-grams. DEBUG:cmuclmtk:Allocated space for 8333333 3-grams. DEBUG:cmuclmtk:table_size 50 DEBUG:cmuclmtk:Allocated 57142848 bytes to table for 2-grams. DEBUG:cmuclmtk:Allocated (2+33333332) bytes to table for 3-grams. DEBUG:cmuclmtk:Processing id n-gram file. DEBUG:cmuclmtk:20,000 n-grams processed for each ".", 1,000,000 for each line. DEBUG:cmuclmtk:Calculating discounted counts. DEBUG:cmuclmtk:Warning : 1-gram : Discounting range is 1; setting P(zeroton)=P(singleton). DEBUG:cmuclmtk:Discounted value : 0.97 DEBUG:cmuclmtk:Warning : 2-gram : GT statistics are out of range; lowering cutoff to 6. DEBUG:cmuclmtk:Warning : 2-gram : GT statistics are out of range; lowering cutoff to 5. DEBUG:cmuclmtk:Warning : 2-gram : GT statistics are out of range; lowering cutoff to 4. DEBUG:cmuclmtk:Warning : 2-gram : GT statistics are out of range; lowering cutoff to 3. DEBUG:cmuclmtk:Warning : 2-gram : GT statistics are out of range; lowering cutoff to 2. DEBUG:cmuclmtk:Warning : 3-gram : GT statistics are out of range; lowering cutoff to 6. DEBUG:cmuclmtk:Warning : 3-gram : GT statistics are out of range; lowering cutoff to 5. DEBUG:cmuclmtk:Warning : 3-gram : GT statistics are out of range; lowering cutoff to 4. DEBUG:cmuclmtk:Warning : 3-gram : GT statistics are out of range; lowering cutoff to 3. DEBUG:cmuclmtk:Warning : 3-gram : GT statistics are out of range; lowering cutoff to 2. DEBUG:cmuclmtk:Warning : 3-gram : Some discount values are out of range; DEBUG:cmuclmtk:lowering discounting range to 1. DEBUG:cmuclmtk:Warning : 3-gram : Discounting range of 1 is equivalent to excluding DEBUG:cmuclmtk:singletons. DEBUG:cmuclmtk:Unigrams's discount mass is 0.00782013 (n1/N = 0.258065) DEBUG:cmuclmtk:1 zerotons, P(zeroton) = 0.00782013 P(singleton) = 0.00782014 DEBUG:cmuclmtk:prob[UNK] = 0.00782013 DEBUG:cmuclmtk:Incrementing contexts... DEBUG:cmuclmtk:Calculating back-off weights... DEBUG:cmuclmtk:Writing out language model... DEBUG:cmuclmtk:ARPA-style 3-gram will be written to /home/pi/.jasper/vocabularies/pocketsphinx-vocabulary/keyword/languagemodel DEBUG:cmuclmtk:idngram2lm : Done. DEBUG:cmuclmtk:Command 'idngram2lm -idngram /tmp/tmpAm_8yb.idngram -vocab /tmp/tmpqPX8tM.vocab -vocab_type 1 -oov_fraction 0.5 -min_unicount 0 -verbosity 2 -n 3 -arpa /home/pi/.jasper/vocabularies/pocketsphinx-vocabulary/keyword/languagemodel -bin_input' returned with exit code '0'. DEBUG:client.vocabcompiler:Getting words from vocab file and removing it afterwards... DEBUG:client.vocabcompiler:Starting dictionary... DEBUG:client.vocabcompiler:Getting phonemes for 47 words... DEBUG:client.diagnose:Checking executable 'phonetisaurus-g2p'... DEBUG:client.diagnose:Executable 'phonetisaurus-g2p' found: '/usr/local/bin/phonetisaurus-g2p' DEBUG:client.g2p:Using FST model: '/home/pi/brasil/constituicao.fst' DEBUG:client.g2p:Converting 47 words to phonemes

G10DRAS commented 7 years ago

Below pocketsphinx_continuous command work for me.

pocketsphinx_continuous -hmm constituicao.cd_cont_1000 -lm 3gram.lm -dict constituicao.dic -samprate 16000 -inmic yes -adcdev plughw:1,0 -logfn /dev/null

You need to download this Language Model

also check card index wirh arecord -l and replace, if required, in -adcdev plughw:1,0

odranoelsilel commented 6 years ago

This language model link is not working anymore. Someone who has downloaded previously could post the LaPSLM.rar file somewhere. Thanks.

  • © Githubissues.
  • Githubissues is a development platform for aggregating issues.