livekit / python-sdks

LiveKit real-time and server SDKs for Python
https://docs.livekit.io
Apache License 2.0
139 stars 40 forks source link

fix: independent event loops & streams #51

Closed keepingitneil closed 1 year ago

keepingitneil commented 1 year ago

Previously, AudioStreams and VideoStreams would emit frames as an event. Here we make them async iterators. This makes working with streams a bit more ergonomic. State specific to a stream can live with the logic of frame handling. For example:

   async def transcribe_loop(stream: AudioStream):

        written_samples = 0
        buffer = np.zeros(WHISPER_SAMPLE_RATE * SECONDS_PER_WHISPER_INFERENCE, dtype=np.float32)
        async for frame_coro in agent_stream:
            frame = await frame_coro
            resampled = frame.remix_and_resample(WHISPER_SAMPLE_RATE, 1)
            data = np.array(resampled.data, dtype=np.float32) / 32768.0
            buffer[written_samples: written_samples + len(data)] = data
            written_samples += len(data)

            if written_samples >= WHISPER_SAMPLE_RATE * SECONDS_PER_WHISPER_INFERENCE:
                res = self.whisper.transcribe(buffer)
                written_samples = 0
                print(res)
theomonnom commented 1 year ago

not safe to merge atm, need to ensure synchronisations are still OK

theomonnom commented 1 year ago

sorry @keepingitneil for stealing your PR with my changes lol

theomonnom commented 1 year ago

Important commit is https://github.com/livekit/client-sdk-python/pull/51/commits/22137062f91c870067a90d18db028d9515acf2d6

keepingitneil commented 1 year ago

sorry @keepingitneil for stealing your PR with my changes lol

haha teamwork!