Node.js support

[jeremyckahn]

Support for Trystero running in a Node.js environment could enable some really interesting use cases. @dmotz would you be interested in supporting Node? If so, then I'd be happy to work on a PR to explore this!

[dmotz]

Could you give examples of how you might use Trystero on Node? Is the idea that Node instances would be ephemeral peer clients without the fixed IP a server would have? Maybe on dedicated devices? Just trying to see the use case for the matchmaking functionality.

[jeremyckahn]

Sure! Among other reasons, Trystero is great because it makes it easy to locally-host distributed applications. Right now those applications are limited to the capabilities of a web browser. If Trystero supported Node, those applications could effectively have system-level capabilities as well.

One (admittedly theoretical) use case that comes to mind is a security camera system wherein N IoT-connected cameras send their data to a Node instance to capture the video data. With Trystero, data connections could be P2P and not require an external service to connect (aside from the initial pairing server).

A more practical use case I would personally have is to make a Node CLI-based client for https://github.com/jeremyckahn/chitchatter, a web-based chat app that connects peers with Trystero.

[dmotz]

I welcome explorations on Node support but just want to make sure we can keep the API consistent across browsers and Node and not bloat the bundle size.

There are a few challenges that come to my mind that you're probably aware of. Node doesn't natively support many of the browser APIs Trystero uses -- WebRTC, WebSockets, Web Crypto, and different base64/buffer APIs. These will have to be conditionally polyfilled/substituted to Node libraries without ending up in the browser bundle. Also, since most devs will install Trystero for browser use it would be nice to avoid installing Node dependencies especially since the WebRTC lib requires a substantial native module download/compilation. I'm not sure exactly how to do this and I'd ideally try to avoid splitting the package, but maybe we'd have to.

Having Node support would definitely unlock some novel use cases, but just want to make sure we plan the best approach. Let me know if you have any thoughts.

[jeremyckahn]

I welcome explorations on Node support but just want to make sure we can keep the API consistent across browsers and Node and not bloat the bundle size.

I totally agree! Avoiding browser bundle bloat seems like a perfectly reasonable requirement.

There are a few challenges that come to my mind that you're probably aware of. Node doesn't natively support many of the browser APIs Trystero uses -- WebRTC, WebSockets, Web Crypto, and different base64/buffer APIs. These will have to be conditionally polyfilled/substituted to Node libraries without ending up in the browser bundle.

Ah that's interesting to know. I had thought that conditionally switching from simple-peer-light to simple-peer would get us most of the way there (since the latter seems to support Node out of the box), but it sounds like there's more to it. I'll try to find a good abstraction to manage the environment differences.

Also, since most devs will install Trystero for browser use it would be nice to avoid installing Node dependencies especially since the WebRTC lib requires a substantial native module download/compilation. I'm not sure exactly how to do this and I'd ideally try to avoid splitting the package, but maybe we'd have to.

That makes sense. It sounds like any solution will require some exploration, but generally speaking my approach will be to just get it working in a brute force way and then iterate to get it merge-ready. I won't open a PR until I've got a solution that meets all of our requirements and doesn't introduce any significant bundle size or developer experience regressions.

It seems like there's a bit more to this than I initially anticipated, but it should be achievable. I will explore this as I have time to, but I would also encourage anyone else who is interested in implementing Node support to give it a shot as well. 🙂

[jeremyckahn]

There are a few challenges that come to my mind that you're probably aware of. Node doesn't natively support many of the browser APIs Trystero uses -- WebRTC, WebSockets, Web Crypto, and different base64/buffer APIs.

It appears that Node 19 natively supports WebCrypto. So we should be able to get that for free so long as we require Node 19! 🙌

I'm working through this as I can in https://github.com/jeremyckahn/trystero-node-sandbox. I'd still encourage others to explore making a PR if they're motivated, as my availability to work on this is limited right now.

[jeremyckahn]

I spent some more time exploring and experimenting with this.

There are a few challenges that come to my mind that you're probably aware of. Node doesn't natively support many of the browser APIs Trystero uses -- WebRTC, WebSockets, Web Crypto, and different base64/buffer APIs.

I think that the WebRTC dependency may (effectively) be a blocker for this effort. The only feasible solution I can find is to use https://github.com/node-webrtc/node-webrtc. However, node-webrtc only supports up to Node 15. The greater concern is that the project hasn't seen any updates for over a year, and it no longer appears to be maintained: https://github.com/node-webrtc/node-webrtc/issues/732

While we could proceed with only Node 15 support and conceivably have a working solution, I feel it would be unwise. Being stuck on outdated dependencies would present a considerable maintenance burden and security concern. While Node support would enable some very interesting use cases, it may not be worth the effort at this time. What do you think, @dmotz?

[dmotz]

Ah that's really unfortunate. I hadn't realized the WebRTC extension for Node is unmaintained and it looks like the alternatives are abandoned as well. I think you're right that it's best left on hold until the Node WebRTC situation improves. If you're still looking to experiment with a Trystero-driven CLI you could maybe hack something together with Puppeteer. It would defeat the purpose of being lightweight, but you could at least bring your app to terminals.

I'll close this issue for now, but feel free to reopen if you find another solution.

[jeremyckahn]

Puppeteer is a great idea! I wonder if Electron might also work for this use case, assuming it can be run in a headless way.

Thanks for your guidance so far. I'll keep an eye out for potential ways achieve this in a reasonable way in the future.

[jzombie]

Electron probably would work since it's based on Chromium.

I ran a WebRTC project in a headless Chromium container using a WebSocket connection to control it, and it was very stable. Wasn't getting segmentation faults like I was when trying other libraries.

[jeremyckahn]

According to this comment, node-webrtc may not be the only path forward. It could be worth experimenting with https://github.com/versatica/mediasoup as an alternative WebRTC implementation for Node environments. It purportedly supports modern Node versions and seems actively maintained.

@dmotz do you think it would be worth re-opening this issue? I don't have availability to explore this right now, but I'd like to in the future once I do (others should of course feel free to do so in the meantime).

[butera-simone]

I'm building an app with trystero. As it is now, 2 peers can only communicate when they're both online. Would it be sustainable to build an hybrid app on mobile that keeps a webRTC connection open, to receive messages a-la-telegram? If not, would it be easier after an hypothetical nodeJS integration? (The second question is why I posted it here)

[jzombie]

@butera-simone The "easier" part for Node.js integration is the kicker; it's either unreliable (as in good luck making it work on Node.js directly) or resource intensive (if using a headless browser instance).

For message persistence, it would be far easier to use WebSockets instead (or even HTTP requests), and cache them somewhere server-side, until they can be delivered.

If you're on a mission to do pure WebRTC delivery of the messages, I respect that, but the challenges will be there.

[jzombie]

A long-running WebRTC connection on mobile wouldn't be sustainable.

Push notifications would be far more efficient.

[butera-simone]

As I feared. Thanks for the info.

[dmotz]

@jeremyckahn I haven't tried it, but it might be worth looking at this WebRTC implementation for Node: https://github.com/shinyoshiaki/werift-webrtc

[jeremyckahn]

@jeremyckahn I haven't tried it, but it might be worth looking at this WebRTC implementation for Node: https://github.com/shinyoshiaki/werift-webrtc

This looks great! It looks like werift is close to 1.0 and has a lot of things implemented already. It seems close to being production-ready, but perhaps not quite there yet. That said, it's actively developed and is probably the best option that's currently available.

I'll experiment with this when I have some time! 🙂

[jeremyckahn]

Per @rockey2020 in https://github.com/dmotz/trystero/issues/70#issuecomment-2078544155, we might consider exploring https://github.com/murat-dogan/node-datachannel as a potential solution here. It appears to provide WebRTC polyfills for node.

[dmotz]

node-datachannel looks very promising. There are other APIs needed for Trystero but the gap between the browser APIs and Node/Bun/Deno has been closing.