confidential-containers / simple-kbs

Key Broker Server for SEV(-ES)
Apache License 2.0
13 stars 13 forks source link

'tokio-runtime-worker' panicked at 'called `Result::unwrap()` #55

Closed wainersm closed 1 year ago

wainersm commented 1 year ago
  1. I started a simple-kbs with stack trace logs enabled
  2. After a while I had to change the key secret. I adjusted the database (MySQL) but I didn't stop/start simple-kbs
  3. Now I am getting:
[wmoschet@virtlab1012 ~]$ thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: InvalidByte(0, 226)', src/request.rs:166:39

The agent doesn't get the key:

Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  3m54s                  default-scheduler  Successfully assigned default/coco-custom-nginx to virtlab1012
  Normal   Pulling    2m21s (x4 over 3m53s)  kubelet            Pulling image "wainersm/coco-custom-nginx:encrypted"
  Warning  Failed     2m21s (x4 over 3m52s)  kubelet            Failed to pull image "wainersm/coco-custom-nginx:encrypted": rpc error: code = Internal desc = failed to async handle layer: failed to get decrypt key failed to retrive decrypt key!
  Warning  Failed     2m21s (x4 over 3m52s)  kubelet            Error: ErrImagePull
  Warning  Failed     2m9s (x6 over 3m52s)   kubelet            Error: ImagePullBackOff
  Normal   BackOff    116s (x7 over 3m52s)   kubelet            Back-off pulling image "wainersm/coco-custom-nginx:encrypted"

The new secret:

$ mysql -u ${KBS_DB_USER} -p${KBS_DB_PW} -D ${KBS_DB} -e "SELECT * FROM secrets;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+----+-----------+----------------------------------------------------+-------+
| id | secret_id | secret                                             | polid |
+----+-----------+----------------------------------------------------+-------+
| 10 | key_id1   | ”7JOQd6TASVf9xL6h9AbmBz6Cn/RozUfl/VBD/QPEsCk=”     |  NULL |
+----+-----------+----------------------------------------------------+-------+
wainersm commented 1 year ago

And the keysset:

$ mysql -u ${KBS_DB_USER} -p${KBS_DB_PW} -D ${KBS_DB} -e "SELECT * FROM keysets;"
mysql: [Warning] Using a password on the command line interface can be insecure.
+----+----------+-----------+-------+
| id | keysetid | kskeys    | polid |
+----+----------+-----------+-------+
| 10 | KEYSET-1 | [key_id1] |  NULL |
+----+----------+-----------+-------+

I restarted simple-kbs but still getting the same error.

wainersm commented 1 year ago

Hmmm it was a unfortunate copy/paste of the key which is invalid:

$ echo $ENC_KEY
”7JOQd6TASVf9xL6h9AbmBz6Cn/RozUfl/VBD/QPEsCk=”
$ echo $ENC_KEY | base64 -d
base64: invalid input

But maybe simple-kbs should gentile log an error instead of panic at https://github.com/confidential-containers/simple-kbs/blob/main/src/request.rs#L166

fitzthum commented 1 year ago

Ah yeah. Error logging is a bit weak for simple-kbs in general. Let me see if there is an easy fix to this, though.

dubek commented 1 year ago

And we should take a close look at all the .unwrap()s we have in the code (except the tests).

wainersm commented 1 year ago

And we should take a close look at all the .unwrap()s we have in the code (except the tests).

True. calling unwrap() is not the proper way to handle error because it simply doesn't handle the error :) (Non-Rust expert speaking)

fitzthum commented 1 year ago

Yeah this one snuck in because I implementing into_bytes and I wanted to mirror the method that String provides. This was a mistake since base64 -> string is fundamentally fallible.

We should comb through the other unwraps.