pipecat-ai / pipecat

Open Source framework for voice and multimodal conversational AI
BSD 2-Clause "Simplified" License
3.46k stars 338 forks source link

Send STT transcript to websocket #410

Open see2run opened 3 months ago

see2run commented 3 months ago

Hi, I tried and edited "Websocket-Server" example to only use transport.input(), stt and transport.output() in pipeline, I got my voice transcribed in my bot server console, but front-end html didnt receive any messages from websocket.

I need to send STT transcript to html client with websocket, how to achieve this?

Durgesh92 commented 3 months ago

Have you been able to find anything on this? also whether your websocket server is able to handle multiple parallel connections ?

see2run commented 2 months ago

my goals are exactly the same as yours, send stt transcript to websocket and websocket parallel connections but for now i havent find anything yet

LuckyWind2157 commented 1 month ago

Customize a FrameProcessor class to process TranscriptionFrame。` class InSoundFrameProcessor(FrameProcessor): """send user sound transcription text"""

def __init__(self, websocket: WebSocket):
    super().__init__()
    self._websocket = websocket

async def process_frame(self, frame: Frame, direction: FrameDirection):
    await super().process_frame(frame, direction)
    if isinstance(frame, TranscriptionFrame):
        logger.debug(f"send user sound transcription text,text={frame.text}")
        await self._websocket.send_json({"text": frame.text, "name": "user", "timestamp": frame.timestamp})
    # In case anything else downstream needs it
    await self.push_frame(frame)`