whitphx / streamlit-webrtc

Real-time video and audio processing on Streamlit
https://discuss.streamlit.io/t/new-component-streamlit-webrtc-a-new-way-to-deal-with-real-time-media-streams/8669
MIT License
1.43k stars 189 forks source link

Unify input/output API for RECVONLY and SENDONLY modes #1172

Open whitphx opened 1 year ago

whitphx commented 1 year ago

webrtc_streamer has many different I/O types which makes its API complex. We should get rid of some APIs so that both the API and its implementation get simpler.

Input

Currently webrtc_streamer supports 2 input types:

And #731 may also be here.

The source_*_track should be the unified input API, and the player-based API should be removed from webrtc_streamer(). Instead, the developers will manually connect the player object to the input tracks.

Output

Currently webrtc_streamer supports 3 output types:

At least, like the player API above, the recorder API should be removed from webrtc_streamer() so that the developers manually connect the output tracks to the recorder object.

The receiver objects are kind of special, but they may also be unified with the output track-based API.

whitphx commented 1 year ago

However, with the current API, the players and the recorders are automatically stopped or closed by the webrtc_streamer() component. Removing these APIs also gets rid of this advantage.

-> Keep them and also add the track-based API for verbose usage? --> Only the recorder can automatically be stopped by the "stop"event emitted from the upstream track?