Spike: multiple screen displays

[gksander]

I think, theoretically, we could have multiple displayMedia streams running at one time and painted to the canvas. I don't know if that'd make sense, but I do think it'd be interested to be able to easily switch between multiple different windows (e.g. jump between IDE and Chrome window).

[AaronPowell96]

Loved finding this repo, although I don't know svelte (yet). Looks like we're painting everything to canvas? We could use RTCPeerConnection and have multiple streams, possibly paint them to the same canvas ontop of eachother to swap between them? Here's a ton of samples I found that might spark some ideas https://webrtc.github.io/samples/

Only briefly looked at this but think it's a great idea, I need to pickup svelte and maybe help out in future!

[gksander]

Yep, everything gets painted to canvas! I ended up implementing the "multiple screens" functionality by just capturing multiple MediaStreams from separate getDisplayMedia calls – and then allowing the user to pick which stream to paint onto the canvas.

The idea of using RTC constructs is super interesting to me! It'd be really cool to be able to have a "guest" on via RTC and paint both feeds to the canvas – which would work very similar to getUserMedia etc, I think!

I'm wrapping up a blog post (internal link) on the technical details of this whole setup if you're interested!