search

SEPIA-Framework / sepia-stt-server

SEPIA server to support open-source speech recognition via WebSocket connection.
MIT License
116 stars 21 forks source link
audio-streaming kaldi-asr open-source raspberry-pi sepia-framework speech-recognition websocket whisper

readme

SEPIA Speech-To-Text Server

SEPIA Speech-To-Text (STT) Server is a WebSocket based, full-duplex Python server for realtime automatic speech recognition (ASR) supporting multiple open-source ASR engines. It can receive a stream of audio chunks via the secure WebSocket connection and return transcribed text almost immediately as partial and final results.

One goal of this project is to offer a standardized, secure, realtime interface for all the great open-source ASR tools out there. The server works on all major platforms including single-board devices like Raspberry Pi (4).

NOTE: This is a complete rewrite (2021) of the original STT Server (2018). Code of the old version has been moved to the LEGACY SERVER folder.
If you are using custom models built for the 2018 version you can easily convert them to new models (please ask for details via the issues section).

SEPIA STT Recorder Demo

Features

Integrated ASR Engines

If you want to see additional engines please create a new issue. Pull requests are welcome ;-)

Quick-Start with Python

The easiest way to get started is to use a Docker container for your platform. To install the server yourself please see the code section README or scripts section.

Quick-Start with Docker

Simply pull the latest image (or choose an older one from the archive). The smallest English and German Vosk models and an English Coqui model (w/o scorer) are included:

docker pull sepia/stt-server:latest

Supported platforms:

After the download is finished you can start the container like this: 

sudo docker run --rm --name=sepia-stt -p 20741:20741 -it sepia/stt-server:latest

Test via web interface or Python client

To test the server visit: http://localhost:20741 if you are on the same machine or http://[server-IP]:20741 if you are in the same network (NOTE: custom recordings via microphone will only work using localhost or a HTTPS URL!).

Alternatively you can use the python-client to test your server.

Models

Currently the server supports Vosk ASR models, Coqui-STT models and custom models (see "adapt" section below).

To add new ASR models create a shared volume for your container, place your model inside and update the server config file. The "adapt" section below has a more detailed example, but basically you can:

Included inside the Docker containers are:

Server Settings

Most of the settings can be handled easily via the server.conf settings file. Please check out the file to see whats possible.

ENV variables:

Command line options:

NOTE: Command line options always overrule the settings file but in most scenarios it makes sense to simply create a new settings file and use the -s flag.

ASR Engine Settings

As soon as the server is running you can check the current setup via the HTTP REST interface: http://localhost:20741/settings or the test page (see quick-start above).

Individual settings for the active engine can be changed on-the-fly during the WebSocket 'welcome' event. See the API docs file for more info or check out the 'Engine Settings' section of the test page.

How to use with SEPIA Client

The SEPIA Client will support the new STT server out-of-the-box from version 0.24.0 on. Simply open the client's settings, look for 'ASR engine (STT)' and select SEPIA. The server address will be set automatically relative to your SEPIA Server host. If your SEPIA server proxy has not been updated yet to forward requests to the SEPIA STT-Server you can enter the direct URL via the STT settings page, e.g.: http://localhost:20741 or http://localhost:20726/sepia/stt. The settings will allow you to select a specific ASR model for each client language as well (if you don't want to use the language defaults set by your STT server config).

NOTE: Keep in mind that the client's microphone will only work in a secure environment (that is localhost or HTTPS) and thus the link to your server must be secure as well (e.g. use a real domain and SSL certificate, self-signed SSL or a proxy running on localhost).

Develop your own Client

See the separate API docs file or check out the Javascript client class, the HTML test page or the python-client source-code.

Demo clients:

Using Customized ASR Models

Open-source ASR has improved a lot in the last years but sometimes it makes sense to adapt the models to your own, specific use-case/domain and vocabulary to improve accuracy. Language model adaptation via web GUI is planned for the near future. Until then please check out the following link:

Adapt a model using the Docker image

Before you continue please read the basics about custom model creation on kaldi-adapt-lm if you haven't already. You should at least understand what the 'lmcorpus' folder does and have a 'sentences[lang].txt' ([lang] e.g.: en, de) ready in your language ;-).

If you use one of the newer Docker images (>=August 2021) 'kaldi-adapt-lm' is already integrated and ready for action. You just need to adjust your Docker start command a bit:

The result should look like this:

sudo docker run --rm --name=sepia-stt -p 20741:20741 -it \
    -v [host-models-folder]:/home/admin/sepia-stt/models/my \
    -v [host-share-folder]:/home/admin/sepia-stt/share \
    --env SEPIA_STT_SETTINGS=/home/admin/sepia-stt/share/my.conf \
    sepia/stt-server:latest \
    /bin/bash

Don't start the container yet! First copy your own LM corpus (e.g.: sentences_en.txt) and optionally LM dictionary (e.g.: my_dict_en.txt) to your shared folder on the host machine ([host-share-folder]).

When you are ready do the following:

Finally we need to tell the server where to find the new model:

To use the new model in "production" don't forget to start your Docker container with the -v and --env modifications from now on (drop the '/bin/bash' if you just run the server).