Using a single tunnel/gateway/jump host to connect to multiple hosts

[andrewhavens]

I feel like I'm running into a performance issue. I'm trying to connect to multiple hosts (about 50), through a single jump host. I can configure the Net::SSH proxy to use the jump host:

SSHKit::Backend::Netssh.configure do |ssh|
  ssh.ssh_options = {
    proxy: Net::SSH::Proxy::Jump.new("myuser@jumphost"),
  }
end
hosts = ['example1', 'example2']
on(hosts) do |host|
  # ...
end

However, it looks like it results in 50 individual connections through the same jump host. What I'm expecting is something more like Net::SSH::Multi which uses a shared Net::SSH::Gateway:

hosts = ['example1', 'example2']
Net::SSH::Multi.start do |session|
  session.via 'jumphost', 'myuser'

  hosts.each do |host|
    session.use "root@#{host}"
  end
end

Is it possible to configure SSHKit to use a single jump host or a Net::SSH::Multi session?

[leehambley]

With or without SSHKit (i.e in the Ruby config, or on your ~/.ssh/config config) you might be able to use a "control master" to speed things up https://www.cyberciti.biz/faq/linux-unix-reuse-openssh-connection/

Essentially the control master reuses existing connections (in general, not only for jump hosts) and multi-plexes new "channels" on top of them.

[G-Rath]

@leehambley that works a treat but sadly it's apparently not supported in net-ssh (yet) and I don't know how much work it would take to implement (I'd be happy to help, but I've not really got any idea where to start and would need support).

I'm in a similar situation with trying to use a helper for AWS SSM + EC2 Instance Connect: every task results in a new SSM session being created which takes a few seconds (resulting in a standard rails deployment taking ~3-4 minutes instead of 30 seconds) & is harder to audit (since we get 30+ sessions per deployment to a single host, instead of only one or two).

I'm assuming that it's not possible for Capistrano to use the native ssh command instead of net-ssh, due to the implementation structure? (at least as far as I can tell there's no way to have Net::SSH result in using native ssh).

[leehambley]

Hi, I'm sorry to hear that going through this SSM+EC2Instance Connect is so slow. There's no real way around this and Net::SSH is complicated enough that I couldn't advise you where to begin.

There's no real way to swap out net-ssh for real SSH, unfortunately because SSH is a multiplexed protocol (each connnection can have multiple channels, each channel has a "mode", and one of those modes is a pseudoterminal) we need a lot of control on the underlying connection, more than we could get by shelling out to a system service (probably).

My best idea would be for you to configure you jump host once, in a pre-task for capistrano, or by hand, once, and use a local ssh config to use localhost:someport (which is a tunnel to your jump host) as a proxy/gateway, this way the tunneling may be invisible to net-ssh. I've never tried something like this, but maybe it gets you away from some of the library limitations you have found.

Happy to give more pointers if you run into problems.

Lee Hambley http://lee.hambley.name/ +49 (0) 170 298 5667

On Fri, 10 Sept 2021 at 02:09, Gareth Jones @.***> wrote:

@leehambley https://github.com/leehambley that works a treat but sadly it's apparently not supported in net-ssh (yet) https://github.com/net-ssh/net-ssh/issues/443 and I don't know how much work it would take to implement (I'd be happy to help, but I've not really got any idea where to start and would need support).

I'm in a similar situation with trying to use a helper for AWS SSM + EC2 Instance Connect: every task results in a new SSM session being created which takes a few seconds (resulting in a standard rails deployment taking ~3-4 minutes instead of 30 seconds) & is harder to audit (since we get 30+ sessions per deployment to a single host, instead of only one or two).

I'm assuming that it's not possible for Capistrano to use the native ssh command instead of net-ssh, due to the implementation structure? (at least as far as I can tell there's no way to have Net::SSH result in using native ssh).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/capistrano/sshkit/issues/491#issuecomment-916524327, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAEUCGXEO5MPCHXTJZS2ZLUBFECTANCNFSM42RSGOVQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[G-Rath]

I got around to looking back into this and figured out how to resolve this which we've implemented as a gem that we plan to publish soon.

It uses the session-manager-plugin to create an SSH tunnel in a background process for each server, then Capistrano is pointed at the local port for said tunnel; so far it is working really well 🎉

[leehambley]

Thanks for the follow-up @G-Rath -- a little over a year! Such is the wheel of FOSS. Really pleased that you found a solution, your gem looks pretty good, too. I only took a small glance, but first impressions were great.