bridge: move into substrate

[progrium]

This is to track getting the new bridge code in webrtc-sessions into the substrate repo. I might suggest moving it in as bridge2 for now because there are still a few minor things missing or will be temporarily lost in a full replacement, but lets consider this against what will be gained:

new bridge gains

• support for creating and accessing multiple sessions
• sessions (audio data and annotation/event data) are persisted to disk
• sessions can be continued as selected in sidebar
• multiple audio sources are possible including multiple tabs of same session
• capturing system audio including other tabs is possible
• new session model is simple and open ended making it easy to add new agents/services
• at least 1/2 the code optimized for increased velocity/experimentation/iteration

missing/lost functionality at present time

• translation (regained with #63)
• LLM assistant (regained with #64)
• "drafts", though model supports adding per agent as needed
• representing silence in UI (easy to add)
• webrtc config file (but same underlying stack means possible to add back)

backing services

There are new equivalents of the asr-* and python-llama-cpp services named by functional role (so model/impl can change) based on the existing ones, except they run on CPU and dont have any shared dependency so they are self contained and individually iterable. These include minor DX additions including readmes, resource usage info, and test scripts/programs. They can be started with a docker-compose file.

These services will need:

• substrate cue service descriptions
• flags to run in "gpu" mode
• support for resourcedirs

They are currently made to include models in their container images to be self contained and easy to setup, but this needs to be modified to happen optionally when running in substrate and set up to use resourcedirs.

other considerations

Bridge sessions are stored on disk, so the bridge substrate service will probably also need this to integrate with resourcedirs.

move recommendation

It seems like it would be easy to move the webrtc-sessions/bridge project into the substrate repo as bridge2 during transition. Work could continue to happen running/developing outside substrate system, but with the project source in the substrate repo. The changes/additions needed for this to work inside substrate can be added in parallel to development by somebody more familiar with substrate. Once it can run in/on substrate and state of gains/losses is acceptable, it can be renamed to replace bridge.

[progrium]

This issue has been updated with a rough outline of what needs to be done to run on substrate and a plan to start integrating it while development continues as is.

[ajbouh]

Excellent, thanks for laying out these steps.

I agree putting it in /images/bridge2 would make sense.

There should be a corresponding /defs/bridge2.cue file as well.

backing services

• each should be "flattened" into /images/ and given their own .cue file

resourcedirs

• deterministically specified by their "inputs"
• named by the digest of their inputs
• never modified once created
• are mounted (ro) into the container on start

spaces

For saving data, we'll use substrate's concept of a "space". A space is a directory that is mounted (rw) into the container on start. Each space has a unique (randomly generated) ID.

A service can declare the spaces they use as spawn.parameters in its corresponding .cue file.

[progrium]

Shall we close this? Has everything else needed to be done ended up in new issues?

[progrium]

@ajbouh ^

[ajbouh]

let's add issues for the functionality still missing, then yes sounds good to close

[mgood]

I think this seems safe to close now, right?