search

rapier1 / hpn-ssh

HPN-SSH based on OpenSSH
https://psc.edu/hpn-ssh-home
Other
319 stars 43 forks source link

Upstreaming hpn-ssh into OpenSSH #89

Open nh2 opened 3 months ago

nh2 commented 3 months ago

Is there already an issue somewhere that tracks what needs to be done to simply upstream this project into OpenSSH?

If no, let's use this one.

rapier1 commented 3 months ago

The main issue is that the people at OpenSSH have clearly stated that they do not want to incorporate HPN-SSH into OpenSSH. I've been trying to do that since I started working on this 20 years ago. It's just not what they want.

nh2 commented 3 months ago

Is it documented with parts they don't want?

For example in http://www.allanjude.com/bsd/AsiaBSDCon2017_-_SSH_Performance.pdf it says

The default SSH window size was 64 - 128 KB, which worked well for interactive sessions, but was severely limiting for bulk transfer in high bandwidth-delay product situations.

OpenSSH later increased the default SSH window size to 2 MB in 2007.

That sounds like OpenSSH generally is OK with increasing the window size. What are the criteria on which they are OK to raise it? Just faster networks? Because with the current 2 MB one can only get 160 Mbit/s on 100 ms latency.

My question would be if OpenSSH could just disable the window size limitation entirely, just letting TCP do its dynamic window sizing like all other sane TCP programs.

Why does that limitation exist in the first place?

Separately, hpn-ssh has more features than just the window size, so I guess it is conceivable that OpenSSH would accept some of the features but not others?

rapier1 commented 3 months ago

The windowing exists because SSHv2 is actually a multiplexed protocol. It has multiple data channels running over a single TCP connection. So they implemented a sliding window like flow control mechanism to handle traffic on each individual channel. It would be a lot easier if they hadn't done this but I understand why they did it this way. For example, the inbound and outbound side of the connection uses different encryption keys. Splitting the data flow into channels makes this easier to code.

So the objections raised all came up years ago. In general they thought that the patch was too large, wasn't necessary, and so forth. I tried for a few years to get them to work with me but it never happened. That's why I split hpn-ssh off into its own soft fork.

One of the things we hope to work on in the not so distant future is a separate path in the code that would let TCP handle all flow control for bulk data transfers. That might get a little dicey but we think it's doable. We just haven't had the resources to work on it as of yet.