search

fedirz / faster-whisper-server

https://hub.docker.com/r/fedirz/faster-whisper-server
MIT License
477 stars 78 forks source link

streaming transcription: Add example #15

Closed powellnorma closed 4 months ago

powellnorma commented 4 months ago

I first thought that It works in the similar way to chat messages are being when chatting with LLMs. means it would use SSE too. But it just streams raw json. However its not even always a complete json object, sometimes its just part of one. Or am I missing something?

I currently just do:

with requests.post(url, files=files, data=data, stream=True) as r:
    for x in r:
powellnorma commented 4 months ago

Ok, this seems to work:

with requests.post(url, files=files, data=data, stream=True) as r:
    tmp = b""
    for x in r:
        tmp += x
        if tmp[-1] == ord('}'):
            txt = json.loads(tmp)['text']
            print(txt)
fedirz commented 4 months ago

Let me test out the behaviour and get back to you. The first you provided is how it should work

fedirz commented 4 months ago

SSE should work as expected now!