hashicorp / yamux

Golang connection multiplexing library
Mozilla Public License 2.0
2.22k stars 234 forks source link

Multiple independent listeners on one connection #129

Closed ErikEngerd closed 2 months ago

ErikEngerd commented 3 months ago

I have a use case where I am running a gliderlabs embedded SSH server listening on a yamux.Session (which implements net.Listener). Now I would also like to run another service next to it (sort of a control session). That service would be for instance a simple JSON rest service.

Now, creating multiple sessions is not allowed: yamux.Client() and yamux.Server() should be called only once.

I now have several possible solutions:

  1. creating a separate TCP connection and the control session on top of that
  2. wrapping the communication to SSH in websockets. That way I can demultiplex again based on the URL path
  3. running yamux over yamux. Is that adventurous or has that been done before

ad 1. simplest workaround possible. Will complicate initialization logic since I need two working connections to have fully established interaction. Various error cases to consider where 1 of the 2 connections goes down or is never made. ad 2. is possible, but I am already using websockets for the connection to the server so this would be websockets over yamux over websockets ad 3. fun factor, would be cool if it worked. Likely to expose subtle bugs perhaps? Performance overhead.

Is there any other way to create independent listeners on the same TCP connection using yamux?

schmichael commented 2 months ago

Yamux does not provide a facility to run multiple protocols on the same Session (in your case: an SSH server and a REST service).

If you control both sides of the code each Stream could start with a magic byte to indicate whether it is an SSH stream or a REST stream. This is basically implementing extremely basic service routing based on an initial byte; similar to HTTP URL routers but obviously much simpler.

If you don't control both sides, perhaps you could peek at the initial bytes of a stream and route based on whether they look SSH-ish or HTTP-ish.

I think all of your proposed solutions are valid as well. If you hit bugs with #3 feel free to file them. It's not an explicitly supported use case, but Yamux shouldn't care what protocols you run in it or what transports it runs on. If it does care either way that could be a bug!

I'm closing for now since there's no yamux issue here as far as I can tell. Feel free to reopen or open a new issue if you have another question or issue.

ErikEngerd commented 2 months ago

I like the solution with the initial byte. Basically it is solution no. 4 where I would write a demultiplexing listener.

Meanwhile, I have already implemented another solution. Instead of rest, I am setting up another connection initially over yamux that I keep alive. Then using gob to exchange information instead of rest. After that, I setup the Ssh listener. I don't really need rest, I just need a way to exchange information between client and server while the Ssh server is running. And that eliminates the requirement for a second listener.