search

dialogflow / asr-server

FastCGI support for Kaldi ASR
Apache License 2.0
184 stars 86 forks source link

Low throughput w/ kaldi + asr-server + kaldi #24

Closed milescrabill closed 6 years ago

milescrabill commented 6 years ago

I am testing using nginx -> fastcgi_pass -> asr-server -> kaldi successfully, but the throughput we're getting is very low - on the order of 1-2rq/ps using significantly sized AWS instances.

Do people have similar experiences using this in production? We're running running with the ../asr-server/fcgi-nnet3-decoder --fcgi-socket=:8000 option.

milescrabill commented 6 years ago

We've gotten better throughput since this issue was created (mostly by adjusting thread count and sizing up to instances with more cores). For those that don't know, you can pass fcgi-nnet3-decoder a flag --fcgi-threads-number=${processorcount} where ${processorcount} is an integer.

ghost commented 6 years ago

@milescrabill wondering what throughput you were able to achieve on which AWS instance type -- it'd really help for a rough cost estimate, thank you for your time!

milescrabill commented 6 years ago

@ananda621 We're overspecced for our workloads, but we're using c4.4xlarges.