alumae / kaldi-gstreamer-server

Real-time full-duplex speech recognition server, based on the Kaldi toolkit and the GStreamer framwork.
BSD 2-Clause "Simplified" License
1.07k stars 341 forks source link

issue running threaded nnet2 #11

Closed russlevy closed 9 years ago

russlevy commented 9 years ago

Using the sample_english_nnet2.yaml as is, and then adding use-threaded-decoder : True to the decoder section I am running this on a WAV file that is continuously being written to storage, and I get this error message:

KALDI_ASSERT: at BestPathEnd:lattice-faster-online-decoder.cc:667, failed: NumFramesDecoded() > 0 && "You cannot call BestPathEnd if no frames were decoded."
Stack trace is:
kaldi::KaldiGetStackTrace()
kaldi::KaldiAssertFailure_(char const*, char const*, int, char const*)
kaldi::LatticeFasterOnlineDecoder::BestPathEnd(bool, float*) const
kaldi::LatticeFasterOnlineDecoder::GetBestPath(fst::VectorFst<fst::ArcTpl<fst::LatticeWeightTpl<float> > >*, bool) const
kaldi::SingleUtteranceNnet2DecoderThreaded::GetBestPath(bool, fst::VectorFst<fst::ArcTpl<fst::LatticeWeightTpl<float> > >*, float*) const
.
.
.
/usr/lib/x86_64-linux-gnu/libgstreamer-1.0.so.0(+0x94a16) [0x7fd78e00fa16]
/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x712b8) [0x7fd78fb1f2b8]
/lib/x86_64-linux-gnu/libglib-2.0.so.0(+0x70925) [0x7fd78fb1e925]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x80a5) [0x7fd791d670a5]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7fd791a94cfd]

It alternates between this message and You cannot call FinalizeDecoding() and then call BestPathEnd() with use_final_probs == false (the assertion before it in the file).

Do you have any recommendations on getting this to work?

alumae commented 9 years ago

Does it work if use the non-threaded decoder?

I doubt if it works with a wav file that is continuously being written. Maybe you can somehow stream the audio into GStreamer istead of a wav file?

russlevy commented 9 years ago

It works on the non-threaded decoder I modified the client to read in the following way (basically wait 5 seconds before assuming you're at the end of the file).

On the unthreaded version it works well.

    def try_read(self, f):
        count = 0
        while True:
            for block in iter(lambda: f.read(self.byterate/4), ""):
                self.send_data(block)
                count = 0
            time.sleep(1)
            if count > 5:
                break
            count = count + 1
alumae commented 9 years ago

OK, I'll take a look.

alumae commented 9 years ago

I committed a fix, please try again.

alumae commented 9 years ago

Just to make it clear, the fix was committed to gst-kaldi-nnet2-online -- you have to recompile it.