opnsense / plugins

OPNsense plugin collection
https://opnsense.org/
BSD 2-Clause "Simplified" License
826 stars 614 forks source link

security/acme-client fails to upload cert and key to ESXI server #3862

Open SXN31 opened 5 months ago

SXN31 commented 5 months ago

Important notices Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug

os-acme-client (installed) 4.1 https://afirewall.sd.s31.me/ui/acmeclient/actions

After properly setting up the upload_sftp automation and validating connection settings I ran the automation and encountered the following error message:

Failed uploading file '/tmp/sftp-upload-v71HKV' to 'ca.pem' ; Cause: {"file_not_found":true,"error":"remote fsetstat: No such file or directory"}

However, the ca.pem file WAS uploaded, yet the private key and cert, which are what I actually want uploaded do not get uploaded.

I don't think it's an issue with my configuration as I can see the ca.pem file uploaded and apparently intact, but here's what I'm specifying in the name template fields. I'm fairly certain what I used should work because in other testing I renamed it to ca2.pem and that also got uploaded properly, just not the other files, and all with the same error message.

I would prefer not use the ca.pem or fullchain.pem as they are not needed for my setup, but if I don't specify anything then the default / gets created and I'm trying to keep my system clean.

Tip: to validate your setup was working with the previous version, use opnsense-revert (https://docs.opnsense.org/manual/opnsense_tools.html#opnsense-revert)

To Reproduce Steps to reproduce the behavior:

  1. Go to Services --> ACME Client --> Automations
  2. Create a new "Upload certificate via SFTP" automation
  3. Fill out all the details, specify your user and identity type.
  4. Specify remote path to write to. /etc/vmware/ssl
  5. fill out the names you want to use under the naming section
  6. click show identity and copy the cipher type and base64 key
  7. Paste the key to your authorized_keys file in esxi: vi /etc/ssh/keys-root/authorized-keys
  8. It will looks similar to ecdsa-sha2-nistp521 AAAAE2...
  9. Back in OPNSense attach the automation to a certificate.
  10. Run the automation, or if you haven't done it yet, create the certificate and let the automation run
  11. See error

Expected behavior key.pem and cert.pem copied to the destination host as rui.crt and rui.key respectively.

Screenshots

Relevant log files

ACME Log showing cert path sudo cat /var/log/acmeclient/latest.log

<14>1 2024-03-17T10:21:03-07:00 <HOSTNAME> acme.sh 72574 - [meta sequenceId="154"] [Sun Mar 17 10:21:03 PDT 2024] Installing cert to: /var/etc/acme-client/certs/65f5<REDACTED>.97076326/cert.pem
<14>1 2024-03-17T10:21:03-07:00 <HOSTNAME> acme.sh 73669 - [meta sequenceId="155"] [Sun Mar 17 10:21:03 PDT 2024] Installing CA to: /var/etc/acme-client/certs/65f5<REDACTED>.97076326/chain.pem
<14>1 2024-03-17T10:21:03-07:00 <HOSTNAME> acme.sh 74245 - [meta sequenceId="156"] [Sun Mar 17 10:21:03 PDT 2024] Installing key to: /var/etc/acme-client/keys/65f5<REDACTED>.97076326/private.key
<14>1 2024-03-17T10:21:03-07:00 <HOSTNAME> acme.sh 74937 - [meta sequenceId="157"] [Sun Mar 17 10:21:03 PDT 2024] Installing full chain to: /var/etc/acme-client/certs/65f5<REDACTED>.97076326/fullchain.pem

System log with upload failure sudo cat /var/log/system/latest.log

11>1 2024-03-17T10:21:07-07:00 <HOSTNAME> opnsense 78259 - [meta sequenceId="13"] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed uploading file '/tmp/sftp-upload-v71HKV' to 'ca.pem' ; Cause: {"file_not_found":true,"error":"remote fsetstat: No such file or directory"}
<11>1 2024-03-17T10:21:07-07:00 <HOSTNAME> opnsense 78259 - [meta sequenceId="14"] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Failed on {"source":"/tmp/sftp-upload-v71HKV","target":"ca.pem","mode":"0440","group":false,"delete_source":true}
<11>1 2024-03-17T10:21:07-07:00 <HOSTNAME> opnsense 78259 - [meta sequenceId="15"] /usr/local/opnsense/scripts/OPNsense/AcmeClient/upload_sftp.php: Command execution failed, exit code 2. Last input was: {"host":"10.0.10.10","host-key":"","port":"22","identity-type":"ecdsa","user":"root","remote-path":"/etc/vmware/ssl","chgrp":"","chmod":"","chmod-key":"","cert-name":"rui.crt","key-name":"rui.key","ca-name":"ca.pem","fullchain-name":"fullchain.pem","certificates":"65f5<REDACTED>.97076326","automation-id":"65f5<REDACTED>.74631258"}

Proof files exist at source location:

$ ls keys/65f5<REDACTED>.97076326/
private.key
$ ls certs/65f5<REDACTED>.97076326/
cert.pem        chain.pem       fullchain.pem

Additional context Add any other context about the problem here.

This bug looks similar, although they had tried to change permissions where I did not.

https://github.com/opnsense/plugins/issues/3003

Environment Software version used and hardware type if relevant. e.g.:

OPNsense 24.1.3_1-amd64 FreeBSD 13.2-RELEASE-p10 OpenSSL 3.0.13 Intel(R) Core(TM) i5-5200U CPU @ 2.20GHz (2 cores, 4 threads) os-acme-client (installed) 4.1

tmacphail commented 1 month ago

I've ran into the same issue and tracked down the source of it somewhat. It relates to setting permissions after the file is uploaded (fsetstat). Specifically, for some reason, the sftp-server isn't finding the uploaded file when it comes to preserving modtime:

2024-06-26T19:54:54.528Z sftp-server[1075459]: open "/etc/vmware/ssl/rui.crt" flags WRITE,CREATE,TRUNCATE mode 0600 2024-06-26T19:54:54.529Z sftp-server[1075459]: set "/etc/vmware/ssl/rui.crt" mode 0600 2024-06-26T19:54:54.529Z sftp-server[1075459]: set "/etc/vmware/ssl/rui.crt" modtime 20240625-21:16:07 2024-06-26T19:54:54.529Z sftp-server[1075459]: sent status No such file 2024-06-26T19:54:54.530Z sftp-server[1075459]: close "/etc/vmware/ssl/rui.crt" bytes read 0 written 1802

It is using openssh's sftp-server but I'm not aware of an issue with that. It seems to have created and written the file fine. Even setting mode 0600 worked (or the initial mode of 0600 was successful and the second mode 0600 was a no-op so didn't cause an error). It didn't get to the point of trying to set group ownership so I can't tell if that would work yet.

I manually tested with scp -p and it failed similarly. When testing scp without the -p option everything succeeded:

2024-06-26T20:07:01.586Z sftp-server[1076334]: mkdir name "/etc/vmware/ssl/hostname" mode 0755 2024-06-26T20:07:01.587Z sftp-server[1076334]: open "/etc/vmware/ssl/hostname/fullchain.pem" flags WRITE,CREATE mode 0440 2024-06-26T20:07:01.588Z sftp-server[1076334]: set "/etc/vmware/ssl/hostname/fullchain.pem" size 3974 2024-06-26T20:07:01.588Z sftp-server[1076334]: close "/etc/vmware/ssl/hostname/fullchain.pem" bytes read 0 written 3974 2024-06-26T20:07:01.589Z sftp-server[1076334]: open "/etc/vmware/ssl/hostname/ca.pem" flags WRITE,CREATE mode 0440 2024-06-26T20:07:01.590Z sftp-server[1076334]: set "/etc/vmware/ssl/hostname/ca.pem" size 1827 2024-06-26T20:07:01.590Z sftp-server[1076334]: close "/etc/vmware/ssl/hostname/ca.pem" bytes read 0 written 1827 2024-06-26T20:07:01.590Z sftp-server[1076334]: open "/etc/vmware/ssl/hostname/cert.pem" flags WRITE,CREATE mode 0440 2024-06-26T20:07:01.591Z sftp-server[1076334]: set "/etc/vmware/ssl/hostname/cert.pem" size 2147 2024-06-26T20:07:01.592Z sftp-server[1076334]: close "/etc/vmware/ssl/hostname/cert.pem" bytes read 0 written 2147 2024-06-26T20:07:01.592Z sftp-server[1076334]: open "/etc/vmware/ssl/hostname/key.pem" flags WRITE,CREATE mode 0400 2024-06-26T20:07:01.593Z sftp-server[1076334]: set "/etc/vmware/ssl/hostname/key.pem" size 3243 2024-06-26T20:07:01.593Z sftp-server[1076334]: close "/etc/vmware/ssl/hostname/key.pem" bytes read 0 written 3243 2024-06-26T20:07:01.594Z sftp-server[1076334]: set "/etc/vmware/ssl/hostname" mode 0755

From this we can see it is still setting ownership when it opens the files, so the set mode isn't really necessary. If the plugin doesn't care about setting modtime, it could get possibly away without using the -p option as a workaround. I haven't tested how the group id field of the plugin interacts with this yet.