imjoy-team / imjoy-rpc

The RPC library used in ImJoy.
MIT License
23 stars 5 forks source link

Support webrtc connection #547

Closed oeway closed 1 year ago

oeway commented 1 year ago

This PR adds two functions for enabling peer-to-peer communication between clients:

from imjoy_rpc.hypha import connect_to_server, register_rtc_service, get_rtc_service
server = await connect_to_server({"server_url": "https://ai.imjoy.io"})
await register_rtc_service(server, "webrtc-service");

In Python, you can also use the synchronous version:

from imjoy_rpc.hypha.sync import register_rtc_service, get_rtc_service

And in the browser, you can do:

<script src="https://cdn.jsdelivr.net/npm/imjoy-rpc@0.5.30/dist/hypha-rpc-websocket.min.js"></script>
<script>
const server = await hyphaWebsocketClient.connectToServer({"server_url": "https://ai.imjoy.io"})
const pc = await hyphaWebsocketClient.getRTCService(server, "webrtc-service");
</script>

It works by using hypha server as a signaling server, after establishing the connection, the rest goes through webrtc in a peer-to-peer manner. This features is developed to ease the stress for large data transmission via the server, it can also support video streaming between clients. One can use the RPC connection via webrtc and stream video between clients. This is ideally suited for applications such as microscope control:

As an example, we generate a random video stream on the python side, and provide a microscope control service (e.g. move stage and snap image): https://github.com/oeway/webrtc-hypha-demo

Here is how it looks like:

Screenshot 2023-07-09 at 13 02 12
beniroquai commented 1 year ago

Indeed, I love it! Thanks for adding this feature Wei! :)