search

alumae / kaldi-gstreamer-server

Real-time full-duplex speech recognition server, based on the Kaldi toolkit and the GStreamer framwork.
BSD 2-Clause "Simplified" License
1.07k stars 341 forks source link

re: issues with new update #18

Closed rohithkodali closed 9 years ago

rohithkodali commented 9 years ago

HI Alumae,

I have seen a couple of bugs with the new update. 1)the server cannot process more than one decoding process per worker, after one decoding processed the worker is not accepting connection from server.

2) we are not getting the correct json format for the usage

curl -T 752706.wav "http://localhost:8888/client/dynamic/recognize"

{"status": 0, "hypotheses": [{"utterance": ""}], "id": "6ceaf16d-e754-488f-a2a8-3d9c4e63affa"}

                or

curl -v -T 20141014165818Z.wav -H "Content-Type: audio/x-raw-int; rate=16000" --header "Transfer-Encoding: chunked" --limit-rate 32000 "http://localhost:8888/client/dynamic/recognize"

{"status": 0, "hypotheses": [{"utterance": ""}], "id": "6ceaf16d-e754-488f-a2a8-3d9c4e63affa"}

as seen we are not getting any hypothesis after the new update. But i can see the result on the server.

alumae commented 9 years ago

Are you using the gst-kaldi-nnet2-online plugin? If yes, did you update it's to latest trunk and recompile?

rohithkodali commented 9 years ago

I have updated it, i'm able to get the likelihood scores and confidence scores.

alumae commented 9 years ago

Can you copy/paste you configuration file?

Also, do you see any errors in the server or worker logs?

rohithkodali commented 9 years ago

Here is my configuration file

You have to download TEDLIUM "online nnet2" models in order to use this sample

Run download-tedlium-nnet2.sh in 'test/models' to download them.

use-nnet2: True decoder: use-threaded-decoder: true model : /home/azureuser/speech/server/kaldi-gstreamer-server-master/test/models/english/fisher_nnet_a_gpu_online/final.mdl word-syms : /home/azureuser/speech/server/kaldi-gstreamer-server-master/auxo/lm/words.txt fst : /home/azureuser/speech/server/kaldi-gstreamer-server-master/auxo/lm/HCLG.fst mfcc-config : /home/azureuser/speech/server/kaldi-gstreamer-server-master/test/models/english/fisher_nnet_a_gpu_online/conf/mfcc.conf ivector-extraction-config : /home/azureuser/speech/server/kaldi-gstreamer-server-master/test/models/english/fisher_nnet_a_gpu_online/conf/ivector_extractor.conf max-active: 7000 beam: 15.0 lattice-beam: 6.0 acoustic-scale: 0.083 lm-fst: /home/azureuser/speech/server/kaldi-gstreamer-server-master/auxo/lm/G.fst do-endpointing : True endpoint-silence-phones : "1:2:3:4:5:6:7:8:9:10" traceback-period-in-secs: 0.25 chunk-length-in-secs: 0.25 num-nbest: 10 out-dir: tmp

use-vad: False silence-timeout: 10

Just a sample post-processor that appends "." to the hypothesis

post-processor: perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\1./;'

full-post-processor: ./sample_full_post_processor.py

logging: version : 1 disable_existing_loggers: False formatters: simpleFormater: format: '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s' datefmt: '%Y-%m-%d %H:%M:%S' handlers: console: class: logging.StreamHandler formatter: simpleFormater level: DEBUG root: level: DEBUG handlers: [console]

alumae commented 9 years ago

Please use markdown syntax to make such things readable (https://guides.github.com/features/mastering-markdown/)

My second question was: do you see any errors in the server or worker logs?

rohithkodali commented 9 years ago

this is my server log after a decoding is complete and when i send a new file to decode

INFO 2015-07-01 14:11:47,999 Worker <main.WorkerSocketHandler object at 0x7f21aa3ac210> leaving INFO 2015-07-01 14:11:48,902 ce903546-4d5d-4ae5-bea9-d6bee6151292: Handling on_connection_close() INFO 2015-07-01 14:11:48,903 ce903546-4d5d-4ae5-bea9-d6bee6151292: Closing worker connection INFO 2015-07-01 14:11:49,052 New worker available <main.WorkerSocketHandler object at 0x7f21aa3ac3d0> INFO 2015-07-01 14:12:57,718 Status listener left INFO 2015-07-02 02:54:18,959 0d516e2d-b038-4546-9c79-7c939c9937e7: OPEN: user='none', content='none' INFO 2015-07-02 02:54:18,960 0d516e2d-b038-4546-9c79-7c939c9937e7: Using worker <main.HttpChunkedRecognizeHandler object at 0x7f21aa3acdd0> INFO 2015-07-02 02:55:09,096 0d516e2d-b038-4546-9c79-7c939c9937e7: Handling the end of chunked recognize request INFO 2015-07-02 02:55:09,098 0d516e2d-b038-4546-9c79-7c939c9937e7: yielding... INFO 2015-07-02 02:55:09,099 0d516e2d-b038-4546-9c79-7c939c9937e7: Waiting for final result... q INFO 2015-07-02 03:09:58,502 Worker <main.WorkerSocketHandler object at 0x7f21aa38bbd0> leaving INFO 2015-07-02 03:09:58,503 0d516e2d-b038-4546-9c79-7c939c9937e7: Receiving 'close' from worker INFO 2015-07-02 03:09:58,504 0d516e2d-b038-4546-9c79-7c939c9937e7: Final hyp:
INFO 2015-07-02 03:09:58,507 200 PUT /client/dynamic/recognize (43.249.226.68) 939547.85ms INFO 2015-07-02 03:09:58,507 Everything done INFO 2015-07-02 04:01:09,529 New worker available <main.WorkerSocketHandler object at 0x7f21aa3acf50> INFO 2015-07-02 04:01:28,258 New worker available <main.WorkerSocketHandler object at 0x7f21aa167990> INFO 2015-07-02 05:48:23,666 57914580-bff0-45e3-bb37-f56cf23e5925: OPEN: user='none', content='none' INFO 2015-07-02 05:48:23,667 57914580-bff0-45e3-bb37-f56cf23e5925: Using worker <main.HttpChunkedRecognizeHandler object at 0x7f21aa167e10>

alumae commented 9 years ago

OK, what about the worker log?

rohithkodali commented 9 years ago

2015-07-02 04:01:28 - INFO: decoder2: Setting pipeline to READY 2015-07-02 04:01:28 - INFO: decoder2: Set pipeline to READY 2015-07-02 04:01:28 - INFO: main: Opening websocket connection to master server 2015-07-02 04:01:28 - INFO: main: Opened websocket connection to server

alumae commented 9 years ago

This is the log describing worker startup, not the processing of an actual request.

rohithkodali commented 9 years ago

here is the log after the decoder processed a couple of requests , it is not accepting connection from server after this

2015-07-02 06:04:25 - INFO: decoder2: 1ccfa65f-9f98-4d00-821a-3f2b8c7f8d99: Pipeline received eos signal 2015-07-02 06:04:25 - INFO: decoder2: 1ccfa65f-9f98-4d00-821a-3f2b8c7f8d99: Resetting decoder state 2015-07-02 06:04:25 - INFO: main: 1ccfa65f-9f98-4d00-821a-3f2b8c7f8d99: Sending adaptation state to client... 2015-07-02 06:04:25 - DEBUG: ws4py: Closing message received (1000) '' 2015-07-02 06:04:25 - DEBUG: main: 1ccfa65f-9f98-4d00-821a-3f2b8c7f8d99: Websocket closed() called 2015-07-02 06:04:25 - DEBUG: main: 1ccfa65f-9f98-4d00-821a-3f2b8c7f8d99: Websocket closed() finished 2015-07-02 06:04:26 - INFO: main: Opening websocket connection to master server 2015-07-02 06:04:26 - INFO: main: Opened websocket connection to server

rohithkodali commented 9 years ago

here is my configuration file uploaded http://textuploader.com/3qep

alumae commented 9 years ago

I'm sorry, but I need a worker log from the startup until the 1st request has been processed.

rohithkodali commented 9 years ago

ok i will repeat the issue and will share the complete logs server and worker. can i get your email ID

alumae commented 9 years ago

alumae@gmail.com

rohithkodali commented 9 years ago

Hi Alumae,

the problem mostly seems like this https://github.com/alumae/kaldi-gstreamer-server/issues/10 but now it is like after a timeout instead of first decoding.

alumae commented 9 years ago

The timeout problem was caused by Azure cloud which closes inactive websocket connections.

bourbonkk commented 5 years ago

Hi rohithkodali I am experiencing the same problems you have experienced. Could you tell me how to solve it?

client: {"status": 0, "hypotheses": [{"utterance": ""}], "id": "1a331ae8-cb64-48ce-ba87-d52476a9d5b4"}

master server: INFO 2018-11-13 10:41:44,721 131493e0-9cef-4bbb-8c43-f147b618347d: Final hyp: