search

alphacep / vosk-server

WebSocket, gRPC and WebRTC speech recognition server based on Vosk and Kaldi libraries
Apache License 2.0
896 stars 243 forks source link

Server capacity & resources #259

Open aslam-t opened 1 month ago

aslam-t commented 1 month ago

Question (Not an Issue)

Hi team,

I have a question rather than a issue. Please bear with me as I have no knowledge of vosk internals. So, I am hosting a docker instance of vosk-server. My primary use case is to utilize it in a conferencing system as transcriber. It works fine with my initial testing with few users but some clarity on following is still required before I confidently release:

  1. Which resource is most important for vosk-server CPU, RAM or GPU etc.?
  2. How many instances are enough to handle 1000 concurrent sessions?
  3. Do we have a pointer if such info already exists.?

I am only transcribing English for now, no other language, if that matter for my case.

Looking forward to any help.

Thanks

nshmyrev commented 1 month ago

Which resource is most important for vosk-server CPU, RAM or GPU etc.

Memory bandwidth and model size are most restricting factors.

How many instances are enough to handle 1000 concurrent sessions?

10 CPU servers of 48 cores each or 5 GPU servers with RTX4090

zhq1 commented 2 weeks ago

Is there any way to optimize CPU usage? For example, not recognizing the contents of silent packages