CZERTAINLY / CZERTAINLY-Keystore-Entity-Provider

MIT License
1 stars 0 forks source link

Incorrect error handling and invalid error reporting when working with remote ssh keystore #39

Open semik opened 9 months ago

semik commented 9 months ago

Describe the bug

When trying to access pkcs12 keystore on a Linux server which missing keytool command CZERTAINLY doesn't report problem correctly.

In case I want just to synchronize keystore (ie. read it into CZERTAINLY) it doesn't report anything. I would expect info that there was problem to read key store.

In case I want to push some certificate into keystore it reports: Failed to push Certificate (400): Failed to push Certificate 1c525054-6f0e-4752-a304-35aeaa06c0bd to Location test-p12. Reason: {"message":""} I expect to get info that there is some problem like bash: keytool: command not found.

I'm testing on version 2.10.0.

To Reproduce Steps to reproduce the behavior:

  1. Prepare testing server without keytool
  2. Define location on that server
  3. Try to push there some certificate
  4. See error

Expected behavior I expect to receive some helpful error.

Additional context There is no useful error message inside keystore-entity-provider POD:

2024-02-01T18:01:43.377Z  INFO 9 --- [nio-8080-exec-1] .p.e.k.s.i.LocationAttributesServiceImpl : Getting the Attributes for Location of Entity test with UUID c75ae346-a83a-482d-a778-c6fe77e38dd3
2024-02-01T18:01:43.378Z  INFO 9 --- [nio-8080-exec-1] .p.e.k.s.i.LocationAttributesServiceImpl : Getting the Attributes for Push of Entity test with UUID c75ae346-a83a-482d-a778-c6fe77e38dd3
2024-02-01T18:01:43.395Z  WARN 9 --- []-nio2-thread-4] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /192.168.1.164:22 presented unverified EC key: SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ
2024-02-01T18:01:43.395Z  WARN 9 --- []-nio2-thread-4] o.a.s.c.k.KnownHostsServerKeyVerifier    : handleKnownHostsFileUpdateFailure(ClientSessionImpl[test@/192.168.1.164:22])[/192.168.1.164:22] failed (FileSystemException) to update key=ecdsa-sha2-nistp256-SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ in /opt/czertainly/.ssh/known_hosts: /opt/czertainly/.ssh/known_hosts: Read-only file system
2024-02-01T18:01:43.396Z  INFO 9 --- []-nio2-thread-4] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-02-01T18:01:44.003Z  WARN 9 --- []-nio2-thread-3] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /192.168.1.164:22 presented unverified EC key: SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ
2024-02-01T18:01:44.004Z  WARN 9 --- []-nio2-thread-3] o.a.s.c.k.KnownHostsServerKeyVerifier    : handleKnownHostsFileUpdateFailure(ClientSessionImpl[test@/192.168.1.164:22])[/192.168.1.164:22] failed (FileSystemException) to update key=ecdsa-sha2-nistp256-SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ in /opt/czertainly/.ssh/known_hosts: /opt/czertainly/.ssh/known_hosts: Read-only file system
2024-02-01T18:01:44.005Z  INFO 9 --- []-nio2-thread-3] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-02-01T18:01:44.229Z  WARN 9 --- []-nio2-thread-3] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /192.168.1.164:22 presented unverified EC key: SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ
2024-02-01T18:01:44.229Z  WARN 9 --- []-nio2-thread-3] o.a.s.c.k.KnownHostsServerKeyVerifier    : handleKnownHostsFileUpdateFailure(ClientSessionImpl[test@/192.168.1.164:22])[/192.168.1.164:22] failed (FileSystemException) to update key=ecdsa-sha2-nistp256-SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ in /opt/czertainly/.ssh/known_hosts: /opt/czertainly/.ssh/known_hosts: Read-only file system
2024-02-01T18:01:44.230Z  INFO 9 --- []-nio2-thread-3] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-02-01T18:01:44.444Z  WARN 9 --- []-nio2-thread-2] o.a.s.c.k.AcceptAllServerKeyVerifier     : Server at /192.168.1.164:22 presented unverified EC key: SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ
2024-02-01T18:01:44.445Z  WARN 9 --- []-nio2-thread-2] o.a.s.c.k.KnownHostsServerKeyVerifier    : handleKnownHostsFileUpdateFailure(ClientSessionImpl[test@/192.168.1.164:22])[/192.168.1.164:22] failed (FileSystemException) to update key=ecdsa-sha2-nistp256-SHA256:sOzXOJfiKR2gzPDxyp/4sWilZ7J+kx0PE4RBIHrNVAQ in /opt/czertainly/.ssh/known_hosts: /opt/czertainly/.ssh/known_hosts: Read-only file system
2024-02-01T18:01:44.445Z  INFO 9 --- []-nio2-thread-2] .s.c.k.e.p.HostBoundPubkeyAuthentication : Server announced support for publickey-hostbound@openssh.com version 0
2024-02-01T18:01:44.663Z  INFO 9 --- [nio-8080-exec-1] c.c.p.e.k.ExceptionHandlingAdvice        : HTTP 400: 

Calling remote commands is not correctly handling errors. Here is transcription of executed commands:

Feb  1 19:01:43 syslog test[53144]: executing scp -t -- /tmp/WETZ7Qecy5k=
Feb  1 19:01:43 syslog test[53144]: executing scp -t -- /tmp/WETZ7Qecy5k=
Feb  1 19:01:44 syslog test[53155]: executing keytool -importcert -keystore /home/test/cert-clt.p12 -storetype PKCS12 -storepass whatever -alias czertainly -file /tmp/WETZ7Qecy5k= -trustcacerts -noprompt
Feb  1 19:01:44 syslog test[53155]: executing keytool -importcert -keystore /home/test/cert-clt.p12 -storetype PKCS12 -storepass whatever -alias czertainly -file /tmp/WETZ7Qecy5k= -trustcacerts -noprompt
Feb  1 19:01:44 syslog test[53164]: executing keytool -list -rfc -keystore /home/test/cert-clt.p12 -storetype PKCS12 -storepass whatever -alias czertainly
Feb  1 19:01:44 syslog test[53164]: executing keytool -list -rfc -keystore /home/test/cert-clt.p12 -storetype PKCS12 -storepass whatever -alias czertainly
Feb  1 19:01:44 syslog test[53173]: executing rm /tmp/WETZ7Qecy5k=
Feb  1 19:01:44 syslog test[53173]: executing rm /tmp/WETZ7Qecy5k=

Note that:

  1. CZERTAINLY tries to executes several keytool commands even after the first one is failing (keytool cmd is not present on target server`), this means that it doesn't handle errors correctly.
  2. Commands are duplicated.

To record commands what are being executed place:

Match User test
        ForceCommand /bin/bash -rc 'logger -p user.notice "executing $SSH_ORIGINAL_COMMAND"; $SSH_ORIGINAL_COMMAND'

test is username of used user, executed commands are logged into /var/log/user.log file on default Debian Bookworm system.

3keyroman commented 9 months ago

I see there is unhandled return codes from the ssh, for example, when the keytool is not found, the code is 127 (command not found). This should be considered as bug and fixed:

https://github.com/3KeyCompany/CZERTAINLY-Keystore-Entity-Provider/blob/develop/src/main/java/com/czertainly/provider/entity/keystore/service/impl/SshServiceImpl.java#L73

When the codes are properly handled, you should not be able to create the Location, because you should get an error response.

I was testing the same as you have described, however, I am not able to see duplicated commands, this is my log:

Feb  2 08:33:21 lab testssh: executing scp -t -- /tmp/LUlzPRHnIbw=
Feb  2 08:33:21 lab testssh: executing keytool -importcert -keystore /home/testssh/test.p12 -storetype PKCS12 -storepass 00000000 -alias czertainly -file /tmp/LUlzPRHnIbw= -trustcacerts -noprompt
Feb  2 08:33:22 lab testssh: executing keytool -list -rfc -keystore /home/testssh/test.p12 -storetype PKCS12 -storepass 00000000 -alias czertainly
Feb  2 08:33:22 lab testssh: executing rm /tmp/LUlzPRHnIbw=

I was trying to run command from CZERTAINLY front end and also directly through the Keystore Entity Provider interface, in both cases, I do not see duplicated commands. Looking into the code, I do not see reason for duplicated commands.

@semik what version of Keystore Entity Provider you are using? I was testing with the deployment using Helm chart version 2.10.0.

semik commented 9 months ago

I'm sorry, I can't reproduce that double execution again. It took some more experiments until I learned how to record commands in first place. So maybe something was running twice? After target system reboot commands are lodged only once.

@semik what version of Keystore Entity Provider you are using? I was testing with the deployment using Helm chart version 2.10.0.

I'm running stable 2.10.0:

    Image:          docker.io/3keycompany/czertainly-keystore-entity-provider:1.4.1
    Image ID:       docker.io/3keycompany/czertainly-keystore-entity-provider@sha256:9d8bc2d76bbb4d448eacdc1c720fd8d1d21cd5fb719414493f4f5628b6d6f503