alumae / kaldi-gstreamer-server

Real-time full-duplex speech recognition server, based on the Kaldi toolkit and the GStreamer framwork.
BSD 2-Clause "Simplified" License
1.07k stars 341 forks source link

worker process killed when worker replications reach to 3 #258

Open nullscc opened 2 years ago

nullscc commented 2 years ago

Hi, I encountered a problem, which worker process will be killed himself(sometimes the worker process will disappear totally). I have no idea why this happened.

Logs are showed below, please notice that there are four windows running shell(I'm using tmux). Toppest one run master_server.py, the other three run worker.py.

Screen Shot 2022-04-06 at 6 35 15 PM

When I ran third worker.py successfully, the already running one will Kill himself(showed in the third window in the picture). Meaning that I am only permitted to run at most two worker processes.

I've tried use docker and py3 version to deploy, and the problem is always the same.

It might be helpful to mention that I've ran successfully using the same source code and config.yaml in anather ubuntu system(ubuntu server 20.04). I'm now use Ubuntu Desktop 21.10.

config.yaml:

# You have to download Librispeech "online nnet2" models in order to use this sample
# Run download-librispeech-nnet2.sh in 'test/models' to download them.
use-nnet2: true
decoder:
    # All the properties nested here correspond to the kaldinnet2onlinedecoder GStreamer plugin properties.
    # Use gst-inspect-1.0 ./libgstkaldionline2.so kaldinnet2onlinedecoder to discover the available properties
    nnet-mode: 3 
    use-threaded-decoder: true

    model : tdnnf/final.mdl
    word-syms : tdnnf/graph-base/words.txt
    fst : tdnnf/graph-base/HCLG.fst
    mfcc-config : conf/mfcc_hires.conf
    ivector-extraction-config : conf/ivector_extractor.conf

    max-active: 5000
    beam: 10.0
    lattice-beam: 4.0
    acoustic-scale: 1.0
    do-endpointing : true
    # endpoint-silence-phones : "1:2:3:4:5:6:7:8:9:10:11:12:13:14:15:16:17:18:19:20"
    endpoint-silence-phones : "1:2:3:4:5"
    traceback-period-in-secs: 0.25
    chunk-length-in-secs: 0.2
    num-nbest: 10
    feature-type: mfcc
    frame-subsampling-factor: 3
    #Additional functionality that you can play with:
    #lm-fst:  man_model/G.fst
    #big-lm-const-arpa: man_model/G.carpa
    #phone-syms: man_model/phones.txt
    #word-boundary-file: man_model/word_boundary.int
    #do-phone-alignment: true
out-dir: tmp

use-vad: False
silence-timeout: 10

# Just a sample post-processor that appends "." to the hypothesis
post-processor: perl -npe 'BEGIN {use IO::Handle; STDOUT->autoflush(1);} s/(.*)/\1./;'

# A sample full post processor that add a confidence score to 1-best hyp and deletes other n-best hyps
#full-post-processor: ./sample_full_post_processor.py

logging:
    version : 1
    disable_existing_loggers: False
    formatters:
        simpleFormater:
            format: '%(asctime)s - %(levelname)7s: %(name)10s: %(message)s'
            datefmt: '%Y-%m-%d %H:%M:%S'
    handlers:
        console:
            class: logging.StreamHandler
            formatter: simpleFormater
            level: DEBUG
    root:
        level: DEBUG
        handlers: [console]

Thanks for any help in advance!!!