rapier1 / hpn-ssh

HPN-SSH based on OpenSSH
https://psc.edu/hpn-ssh-home
Other
302 stars 41 forks source link

None cipher blocked from use except on command line #78

Closed justinclift closed 2 months ago

justinclift commented 2 months ago

In my homelab I've been doing a lot of testing Proxmox 8.x (based on Debian 12) recently.

However, when Proxmox's automatic replication jobs are transferring VM images from one machine to another it's maxing sshd on the cpu without even getting close to a fraction of the (40GbE) network bandwidth.

Thus, trying out HPN-SSH, as Proxmox doesn't allow for using straight netcat or similar.

After getting HPN-SSH installed and replacing the standard ssh with it (both client and server) the transfer speeds are looking a bit better. ~700MB/s+ rather than ~490MB/s.

That's with the aes256-ctr cipher.

Next step is trying to get the NONE cipher working, as the source machine is literally cabled directly to the target machine. No switch. It's secure enough for the NONE cipher to be a reasonable option. :wink:

It seems like HPN-SSH has other ideas though:

NoneSwitch is found in /root/.ssh/config.
You may only use this configuration option from the command line

After spending a bunch of effort to get everything else working, this is... pretty disappointing. :frowning:

Is there an override for this, so people don't need to go and edit the source to make it work?

rapier1 commented 2 months ago

With the client you have to use -oNoneEnabled=yes and -oNoneSwitch=yes on the command line to use the none cipher. I disable allowing either option being in the config files because I want the user to explicitly make the choice to do this and be aware of it. If typing that into the command line is an issue you could set up an alias if you like.

Additionally, you have to use this with another HPN-SSH server that has NoneEnabled=yes set in the sshd_config or on the command line when you instantiate the server. You can also disable the MAC (message authentication cipher) with -oNoneMacEnabled=yes (or in the sshd_config file).

Does that help or am I missing something?

justinclift commented 2 months ago

Does that help ... ?

Not sure yet. The alias mention might do the trick, but the way Proxmox works is that it launches all of the (client) ssh connections in the background (as root) to transfer things around.

I've already moved the standard ssh client out of the way, and ensured hpn-ssh is used instead:

# cd /usr/bin
# mv ssh ssh-old
# ln -s hpnssh ssh

And I've already disabled the regular ssh server (systemctl disable ssh, etc), plus added those two options to the sshd config file on both ends.

To have the none cipher used between the two hosts, I've then created a host specific configuration in /root/.ssh/config (with appropriate Host name on each server):

Host [redacted]
  NoneSwitch=yes
  NoneEnabled=yes

I want the user to explicitly make the choice to do this and be aware of it

It's a bit hard to imagine how all of the above isn't a case of being aware and making the choice. :wink:

gdevenyi commented 2 months ago

Perhaps you should be upstreaming this request to explicitly control the ssh commands to proxmox.

justinclift commented 2 months ago

@gdevenyi That seems like the wrong spot for it. At least for now. Might be a worthwhile idea later on though. :wink:

hpn-ssh arbitrarily disallowing people to use it when configured in some way the author hadn't considered yet is probably what needs looking at first. :smile:

rapier1 commented 2 months ago

HPN-SSH doesn't arbitrarily disallow people from using the none ciphers. It only allows it under certain conditions in order to preserve awareness and security for a very non-standard operational method for the SSH protocol. That's not going to change. The thing is that the use of none is only technically allowed by the protocol and, even then, the implied use case is for debugging purposes only. See https://datatracker.ietf.org/doc/rfc4251/ section 9.3.1. Requiring the command line invocation of NONE on the client side is, along with maintaining encrypted authentication and disabling it for interactive sessions, from my perspective, the fig leaf that make this acceptable.

What I don't understand is why proxmox, which I have no experience with, would disallow you from passing run time arguments to HPN-SSH or any other implementation of SSH. That's all the -o arguments are after all. I understand that you may to use the host directives for the sake of convenience but I'm not stopping anyone from using NoneSwitch. I'm only limiting the ways in which it can be used in canonical releases.

If you'd like to disable these limitations on your own that's fine but I won't provide support for it. The file you need to modify is readconf.c. Search for the case oNoneSwitch: statement and remove the if/else statements.

As an aside - the default cipher, ChaChaPoly1305, is the slowest available cipher even with parallelization. If you need more speed use aes-ctr or aes-gcm.

justinclift commented 2 months ago

No worries. I'm thinking it might be more effective for my use case to remap the ssh connectivity piece to just pipe directly using netcat or similar between these two hosts.

Shouldn't be too hard (in theory) to replace the /usr/bin/ssh binary with a script that checks the arguments it's being called with, using netcat for stuff between these two hosts, and launching standard ssh for everything else. :smile:

justinclift commented 2 months ago

For anyone else that's interested, this is the relevant piece of code @rapier1 mentions above: https://github.com/rapier1/hpn-ssh/blob/4aa6f04fb329dc04d8212ffc3f310d7966f8b7e9/readconf.c#L1303-L1312

Commenting out lines 1304, and 1307-1312 will probably achieve the result. Haven't tested it though. :smile:

justinclift commented 2 months ago

What I don't understand is why proxmox, which I have no experience with, would disallow you from passing run time arguments to HPN-SSH or any other implementation of SSH.

Proxmox is kind of like an OSS version of VMware vSphere, if you've seen that? ssh is just what it uses for the underlying communication between things.

They're not going out of their way to block providing arguments... it's just the ssh communication pieces are buried down in the layers of abstraction.

Using an alias may in fact work (haven't tested it yet), or even just overriding the ssh command with a script that calls it using the extra "none" options.

rapier1 commented 2 months ago

Ah, I see why this could be a problem for you. So yeah, the easiest thing for you to do is just change that section in readconf.c. You'll be able to use the none switch from your config at that point. In any case, yes you are right about the lines you need to comment out. The diff is

diff --git a/readconf.c b/readconf.c
index 88e24407d..b717c5a01 100644
--- a/readconf.c
+++ b/readconf.c
@@ -1295,21 +1295,9 @@ parse_time:
                intptr = &options->fallback_port;
                goto parse_int;

-       /*
-        * We check to see if the command comes from the command
-        * line or not. If it does then enable it otherwise fail.
-        *  NONE should never be a default configuration.
-        */
        case oNoneSwitch:
-               if (strcmp(filename, "command-line") == 0) {
-                       intptr = &options->none_switch;
-                       goto parse_flag;
-               } else {
-                       error("NoneSwitch is found in %.200s.\nYou may only use this configuration option from the command line", filename);
-                       error("Continuing...");
-                       debug("NoneSwitch directive found in %.200s.", filename);
-                       return 0;
-               }
+               intptr = &options->none_switch;
+               goto parse_flag;

        case oVerifyHostKeyDNS:
                intptr = &options->verify_host_key_dns;

That will works as expected but, please, keep in mind that I won't support this. I feel that the risk/benefit isn't worth it for the majority of users.

justinclift commented 2 months ago

Cool. Thanks heaps @rapier1. :smile:

rapier1 commented 2 months ago

No problem at all. If you have any more questions or run into problem let me know. I won't be able to address everything but I do want to support my users as best as I can.

justinclift commented 2 months ago

Added notes for easily "unconditionally enabling" the none cipher between Proxmox nodes:

https://github.com/rapier1/hpn-ssh/wiki/Installation-on-Proxmox-8.x#blanket-override-of-ssh-to-use-the-none-cipher

I'm yet to benchmark things though, as I've only just rebuilt both nodes today with ssds again.

rapier1 commented 2 months ago

Sweet! Thanks for the documentation!