Use Nodeport service for kbs deployment

[kartikjoshi21]

Refer to comment here https://github.com/confidential-containers/cloud-api-adaptor/pull/1735#issuecomment-2059367274

Create nodeport service and expose kbs deployment to this service.

[surajssd]

Create nodeport service and expose kbs deployment to this service.

Using the service IP (nodeport or not) will not work. It is that exposing the service as NodePort and then using the node ip and nodeport service node-port.

I usually would do this to connect:

kubectl -n coco-tenant patch svc kbs --type='json' -p '[{"op":"replace","path":"/spec/type","value":"NodePort"}]'

# Get the port
kubectl -n coco-tenant get svc kbs

export KBS_IP=$(kubectl get nodes -o jsonpath='{.items[0].status.addresses[0].address}'):$(kubectl get svc kbs -n coco-tenant -o jsonpath='{.spec.ports[0].nodePort}')
export AA_KBC_PARAMS="cc_kbc::http://${KBS_IP}"

kubectl -n confidential-containers-system patch configmap peer-pods-cm --patch "{\"data\": {\"AA_KBC_PARAMS\": \"${AA_KBC_PARAMS}\"}}"

[surajssd]

I have updated the KBS k8s config to be able to run the KBS service as nodeport type: https://github.com/confidential-containers/trustee/pull/371/

[mythi]

kubectl -n confidential-containers-system patch configmap peer-pods-cm --patch "{\"data\": {\"AA_KBC_PARAMS\": \"${AA_KBC_PARAMS}\"}}"

patching an existing ConfigMap requires a pod restart. I was struggling with the KBS config since AA_KBC_PARAMS was not mentioned in our docs. Would be good to have that mentioned early on so that patching can be avoided.

On the CAA ConfigMap and how it's populated, would it make more sense to have everything populated via a separate file:

configMapGenerator:
- name: peer-pods-cm
  namespace: confidential-containers-system
  envs:
  - custom-envs