search

flashlight / text

Text utilities, including beam search decoding, tokenizing, and more, built for use in Flashlight.
MIT License
64 stars 14 forks source link

Online/Streaming mode #56

Closed naymaraq closed 1 year ago

naymaraq commented 1 year ago

Question

Is there any official API/doc/example to use flashlight decoder in streaming mode?

jacobkahn commented 1 year ago

@naymaraq — if you're using the CTC decoder, you can pass data into decodeStep incrementally, and the decoder will behave in a streaming fashion out of the box. Streaming isn't currently supported for the autoregressive decoder, although that may change down the line.

There isn't documentation for streaming as it stands, but I hope to add some soon. As always, feel free to contribute back.