We aim to boost productivity by offering meeting summarization to our users. By recording audio streams, transcribing them with Whisper, and summarizing the content using a large language model (LLM), users can quickly understand key points and action items without reviewing entire sessions.
Research
What does LiveKit offer?
WebRTC is fantastic for last-mile media delivery, but interoperability with other services can be challenging
LiveKit offers an additional system, Universal Egress, designed to provide universal exports of LiveKit sessions or tracks to a file or stream data.
To keep the load off the SFU and avoid impacting real-time audio or video performance/quality.
Egress design?
Incoming egress requests are written to a queue observed by a pool of egress workers. A worker decides to fulfill a request based on its current load. Here is the overall system architecture:
Web egress for recordings that aren't attached to a single LiveKit room.
Track composite for exporting synchronized tracks of a single participant. (audio and video)
Track egress for exporting individual tracks. (audio or video, not both)
Each type of Egress has different use cases. Room composite Egress is more CPU-intensive, as it runs a full Chrome instance. Track egress is the most computationally efficient option for exporting tracks and is useful for raw track archival or processing streams in real-time.
Track Egress can also forward a track to another server via WebSocket for real-time features like real-time transcription.
How worker join a meeting?
As hidden participants. For composite requests, they connect to the LiveKit room via JavaScript and capture the browser window. For other egress types, they use the SDK.
There is default support for conventional cloud storage (e.g., AWS, GCP, Azure). Files can also be stored locally. Please refer to the GitHub FAQ for more information.
Feature Request
Goal
We aim to boost productivity by offering meeting summarization to our users. By recording audio streams, transcribing them with Whisper, and summarizing the content using a large language model (LLM), users can quickly understand key points and action items without reviewing entire sessions.
Research
What does LiveKit offer?
LiveKit offers an additional system, Universal Egress, designed to provide universal exports of LiveKit sessions or tracks to a file or stream data.
Full documentation
Why Egress is a separate system?
To keep the load off the SFU and avoid impacting real-time audio or video performance/quality.
Egress design?
Incoming egress requests are written to a queue observed by a pool of egress workers. A worker decides to fulfill a request based on its current load. Here is the overall system architecture:
source
Types of Egress?
All types:
Each type of Egress has different use cases. Room composite Egress is more CPU-intensive, as it runs a full Chrome instance. Track egress is the most computationally efficient option for exporting tracks and is useful for raw track archival or processing streams in real-time.
Track Egress can also forward a track to another server via WebSocket for real-time features like real-time transcription.
How worker join a meeting?
As hidden participants. For composite requests, they connect to the LiveKit room via JavaScript and capture the browser window. For other egress types, they use the SDK.
How is it deployed?
The application needs to be deployed alongside the LiveKit server and should utilize the same Redis service (confirmation pending).
Please find Egress configuration described here, also they offer few instructions on how to run the Egress service locally here
Documentation section
At least 4CPUs and 4GB of memory
File storage
There is default support for conventional cloud storage (e.g., AWS, GCP, Azure). Files can also be stored locally. Please refer to the GitHub FAQ for more information.
Resources