Open leochr opened 2 years ago
@idlewis please add your analysis of truststore handling in Liberty container here. Thank you
I think I can see two issues with the trust store handling in docker-server.sh
The fix for 2 is pretty simple (only add config if docker-server.sh created the truststore)
For 1, we need to decide what to do in the case of an existing trust store.
The docker-server.sh
script shipped with Liberty does not import .crt files that are missing a newline at the end of the file in the /var/run/secrets/kubernetes.io/serviceaccount
dir.
The problem is on this line where we are cat
ing *.crt and attempting to iterate over the certs to import them: https://github.com/OpenLiberty/ci.docker/blob/main/releases/latest/full/helpers/runtime/docker-server.sh#L62
thing1.crt
and thing2.crt
mkdir badcerts
printf "%s" "$(< thing1.crt)" > badcerts/thing1-bad.crt
printf "%s" "$(< thing2.crt)" > badcerts/thing2-bad.crt
This removes the newline at the end of the files:
# diff thing1.crt badcerts/thing1-bad.crt
19c19
< -----END CERTIFICATE-----
---
> -----END CERTIFICATE-----
\ No newline at end of file
Created a configmap with the bad certs:
oc create configmap --from-file cert1.crt=badcerts/thing1-bad.crt --from-file cert2.crt=badcerts/thing2-bad.crt certs
Created this deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: liberty-deployment
namespace: cert-test
spec:
replicas: 1
selector:
matchLabels:
app: liberty-test
template:
metadata:
labels:
app: liberty-test
namespace: liberty-test
spec:
containers:
- name: liberty
image: image-registry.openshift-image-registry.svc:5000/cert-test/ol-test:1.0
ports:
- containerPort: 9080
imagePullPolicy: Always
volumeMounts:
- name: cert-volume
mountPath: /var/run/secrets/kubernetes.io/serviceaccount
env:
- name: SEC_IMPORT_K8S_CERTS
value: 'true'
volumes:
- name: cert-volume
configMap:
name: certs
Check details in the running pod:
# oc rsh liberty-deployment-86df686575-8kc72
sh-4.4$ ls -la /output/resources/security/
total 4
drwxrwx---. 2 default root 21 Feb 15 19:56 .
drwxrwx---. 1 default root 22 Feb 15 19:56 ..
-rw-rwx---. 1 default root 2650 Feb 15 19:56 key.p12
sh-4.4$ ls -la /var/run/secrets/kubernetes.io/serviceaccount/
total 0
drwxrwsrwx. 3 root 1000680000 94 Feb 22 17:21 .
drwxr-xr-x. 3 root root 60 Feb 22 17:21 ..
drwxr-sr-x. 2 root 1000680000 40 Feb 22 17:21 ..2024_02_22_17_21_26.2791951379
lrwxrwxrwx. 1 root 1000680000 16 Feb 22 17:21 cert1.crt -> ..data/cert1.crt
lrwxrwxrwx. 1 root 1000680000 16 Feb 22 17:21 cert2.crt -> ..data/cert2.crt
lrwxrwxrwx. 1 root 1000680000 32 Feb 22 17:21 ..data -> ..2024_02_22_17_21_26.2791951379
sh-4.4$ cat *.crt
-----BEGIN CERTIFICATE-----
...
cert-data-1
...
-----END CERTIFICATE----------BEGIN CERTIFICATE-----
...
cert-data-2
...
-----END CERTIFICATE-----sh-4.4$
Delete & recreate the configmap with the 'good' certs and restart the app pod:
oc delete cm certs
oc create configmap --from-file cert1.crt=thing1.crt --from-file cert2.crt=thing2.crt certs
oc delete po -l 'app=liberty-test'
Check details in the new pod:
# oc rsh liberty-deployment-86df686575-s42f7
sh-4.4$ ls -la /output/resources/security/
total 8
drwxrwx---. 1 default root 23 Feb 22 17:35 .
drwxrwx---. 1 default root 22 Feb 15 19:56 ..
-rw-rwx---. 1 default root 2650 Feb 15 19:56 key.p12
-rw-r--r--. 1 1000680000 root 2054 Feb 22 17:35 trust.p12
sh-4.4$ cat /var/run/secrets/kubernetes.io/serviceaccount/*.crt
-----BEGIN CERTIFICATE-----
...
cert-data-1
...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...
cert-data-2
...
-----END CERTIFICATE-----
sh-4.4$
There are various ways to inject keystore and truststore for the application. In some cases, some conflicts could prevent the application from working as expected. Re-evaluate the generation and handling of keystore and truststore and improve as possible.
Open Liberty runtime also supports providing certificates as env var to establish trust. Switch to this mechanism if possible: https://openliberty.io/docs/latest/reference/feature/transportSecurity-1.0.html#_provide_certificates_from_an_environment_variable_or_a_file