In our own private infrastructure environment, we often need to use our own self-signed TLS certificates to serve our sites over HTTPS.
Step CA can help you generate TLS certificates for your sites using the ACME protocol, and automate the TLS certificate renewal process as well.
In this post we will walk though the process of deploying a PostgreSQL cluster on Kubernetes, then deploying a Step CA Intermediate Certificate Authority that will use the PostgreSQL cluster as the database.
Here is how the final architecture will look like:
## Preparation
To start the entire deployment, let's create the `stepca` Kubernetes namespace where everything will be deployed to:
```bash
kubectl create ns stepca
```
Sign a leaf TLS cert for the domain where you will be hosting your Step CA. We can inject the TLS secret as follows:
```bash
kubectl create secret tls stepca-tls -n stepca --key private.key --cert public.crt
```
Download the `step` binary [here](https://github.com/smallstep/cli/releases) and place the binary in `/usr/bin`
Then generate an intermediate certificate signing request:
```bash
step certificate create "Intermediate CA Name" intermediate.csr intermediate_ca_key --csr
```
Transfer the certificate signing request to your existing root CA and get it signed. You should have the `root_ca.crt` from your existing root CA, `intermediate_ca.crt` from signing the certificate signing request, and `intermediate_ca.key` that was created when you generated the intermediate certificate signing request from the previous step.
## Deploying a PostgreSQL Cluster
There are many available PostgreSQL Operators that can help you lifecycle a highly available PostgreSQL cluster. In this example, we will be using CloudNativePG. I like CloudNativePG because the operator creates the database instances using `Pods` instead of `StatefulSets`, which avoids all the limitations that comes with `StatefulSets`.
To install the CloudNativePG Operator, run the following command:
```bash
kubectl apply -f \ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.22/releases/cnpg-1.22.0.yaml
```
You can verify the Operator is installed with:
```bash
kubectl get deployment -n cnpg-system cnpg-controller-manager
```
Here is the YAML deployment file to create the Postgres Cluster. In this example we will be storing our postgres backups to our own MinIO instance hosted at `minio.domain.org`
```YAML
# Inject MinIO Credentials as Secret
apiVersion: v1
kind: Secret
metadata:
name: minio
namespace: stepca
type: Opaque
stringData:
ACCESS_KEY_ID: minio # S3 username
ACCESS_SECRET_KEY: minio123 # S3 password
---
# Step CA Deployment YAML
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: stepca-postgres
namespace: stepca
spec:
imageName: ghcr.io/cloudnative-pg/postgresql:15.0
instances: 3
primaryUpdateStrategy: unsupervised # Rolling update process to be automated and managed by Kubernetes
monitoring:
enablePodMonitor: true # Expose prometheus metrics
storage:
storageClass: vsan-default-storage-policy # Define the storage class you use in your Kubernetes cluster
size: 1Gi
backup:
# Configure to use S3 to store backup resources
barmanObjectStore:
destinationPath: s3://stepca/ # S3 bucket location
endpointURL: https://minio.domain.org # S3 endpoint
s3Credentials:
accessKeyId:
name: minio
key: ACCESS_KEY_ID
secretAccessKey:
name: minio
key: ACCESS_SECRET_KEY
wal:
compression: gzip
encryption: AES256
retentionPolicy: "7d"
```
You should see the following Kubernetes resources once your PostgreSQL cluster is created:
Services:
- stepca-postgres-r
- applications to connect to any of the instances for read-only workloads
- stepca-postgres-ro
- applications to connect to any of the hot standby, non-primary replicas for read-only workloads
- stepca-postgres-rw
- applications to connect to the primary instance for read-write workloads
Secrets:
- stepca-postgres-app
- database credentials for the default user called `app`, corresponds to the user owning the database
- stepca-postgres-ca
- self-signed CA generated and used to support TLS within the postgres cluster
- stepca-postgres-replication
- streaming replication client certificate generated by the client CA
- stepca-postgres-server
- server TLS certificate signed by the server CA
- stepca-postgres-superuser
- superuser credentials to be used only for administrative purposes, corresponds to the `postgres` user
- stepca-postgres-token
- kubernetes service account created for the database operator
Monitoring:
- When enablePodMonitor is set to `true`, CloudNativePG will automatically expose prometheus metrics relating to CloudNativePG clusters, and create a `PodMonitor` resource for your prometheus to scrape the endpoint
- The pre-requisite is that you must have prometheus already installed
### PostgreSQL Backups
For on-demand backups, apply the following `YAML`:
```
apiVersion: postgresql.cnpg.io/v1
kind: Backup
metadata:
name: backup-example
spec:
cluster:
name: stepca-postgres
```
To schedule backups, apply the following `YAML`:
```YAML
apiVersion: postgresql.cnpg.io/v1
kind: ScheduledBackup
metadata:
name: backup-daily-midnight
namespace: stepca
spec:
schedule: "0 0 16 * * *" # 0000 SGT in UTC time
backupOwnerReference: self
cluster:
name: stepca-postgres
```
To restore backup from S3 object store, apply the following `YAML`:
```YAML
apiVersion: postgresql.cnpg.io/v1
kind: Cluster
metadata:
name: stepca-postgres
namespace: stepca
spec:
imageName: ghcr.io/cloudnative-pg/postgresql:15.0
instances: 3
primaryUpdateStrategy: unsupervised # Rolling update process to be automated and managed by Kubernetes
monitoring:
enablePodMonitor: true # Expose prometheus metrics
storage:
storageClass: vsan-default-storage-policy # Define the storage class you use in your Kubernetes cluster
size: 1Gi
backup:
# Configure to use S3 to store backup resources
barmanObjectStore:
destinationPath: s3://stepca/ # S3 bucket location
endpointURL: https://minio.domain.org # S3 endpoint
s3Credentials:
accessKeyId:
name: minio
key: ACCESS_KEY_ID
secretAccessKey:
name: minio
key: ACCESS_SECRET_KEY
wal:
compression: gzip
encryption: AES256
retentionPolicy: "7d"
bootstrap:
recovery:
source: stepca-postgres
recoveryTarget:
# Use backupID and targetImmediate to backup to that instant and stop immediately, or use targetTime to do point-in-time recovery where the database will run through the WAL to the timestamp specified after restoring from the nearest base backup
backupID: 20240102T160000
targetTime: "2024-01-02T09:00:25"
externalClusters:
- name: stepca-postgres # Name has to be same as the previous cluster since barman will be searching for backups based on the name
barmanObjectStore:
destinationPath: s3://stepca/
endpointURL: https://minio.domain.org # S3 endpoint
s3Credentials:
accessKeyId:
name: minio
key: ACCESS_KEY_ID
secretAccessKey:
name: minio
key: ACCESS_SECRET_KEY
wal:
maxParallel: 8 # Take advantage of the parallel WAL restore feature to dedicate up to 8 concurrent jobs to fetch required WAL files from the archive
```
## Deploying Step CA
Step CA includes a helm chart to deploy on Kubernetes, but there is a disclaimer that they can only support 1 replica instance. Therefore we will not be using their helm chart, and instead we take the following deployment path:
1. Run a Step CA on Docker
2. Retrieve the configuration template to use in our actual deployment
3. Deploy our Step CA with our specified configuration injected as a Kubernetes secret
To run Step CA on Docker, we can use the following `docker-compose.yml` file:
```YAML
version: '3.3'
services:
ca:
image: smallstep/step-ca:0.24.1
networks:
- default
ports:
- "9000:9000"
environment:
- DOCKER_STEPCA_INIT_NAME=${DOCKER_STEPCA_INIT_NAME} # Name of your CA - this will be the issuer of your CA certificates
- DOCKER_STEPCA_INIT_DNS_NAMES=${DOCKER_STEPCA_INIT_DNS_NAMES} # Hostname(s) or IPs that the CA will accept requests on
- DOCKER_STEPCA_INIT_PROVISIONER_NAME=${DOCKER_STEPCA_INIT_PROVISIONER_NAME} # Label for the initial admin (JWK) provisioner. Default: "admin"
- DOCKER_STEPCA_INIT_PASSWORD=${DOCKER_STEPCA_INIT_PASSWORD} # Password for the encrypted CA keys and the default CA provisioner
volumes:
- ./data/home/step:/home/step
restart: always
networks:
default:
ipam:
driver: default
config:
- subnet: "172.97.0.0/16"
volumes:
step_home:
```
From here, we can retrieve the configuration files from `/home/step/`, then edit the template to use the PostgreSQL cluster that we deployed earlier. A reference of the configuration options can be found [here](https://smallstep.com/docs/step-ca/configuration/#basic-configuration-options)
A customised `config/ca.json` is as follows:
```JSON
{
"root": "/home/step/certs/root_ca.crt",
"federatedRoots": null,
"crt": "/home/step/certs/intermediate_ca.crt",
"key": "/home/step/certs/intermediate_ca.key",
"address": "9000",
"insecureAddress": "true",
"dnsNames": [
"localhost",
"ca.domain.org"
],
"logger": {
"format": "text"
},
"db":{
"type": "postgresql",
"dataSource": "postgresql://app:password@stepca-postgres-rw.stepca.svc.cluster.local:5432",
"database": "app"
},
"authority": {
"provisioners": [
{
"type": "JWK",
"name": "admin",
"key": {
"use": "sig",
"kty": "EC",
"kid": "YYNxZ0rq0WsT2MlqLCWvgme3jszkmt99KjoGEJJwAKs",
"crv": "P-256",
"alg": "ES256",
"x": "LsI8nHBflc-mrCbRqhl8d3hSl5sYuSM1AbXBmRfznyg",
"y": "F99LoOvi7z-ZkumsgoHIhodP8q9brXe4bhF3szK-c_w"
},
"encryptedKey": "eyJhbGciOiJQQkVTMi1IUzI1NitBMTI4S1ciLCJjdHkiOiJqd2sranNvbiIsImVuYyI6IkEyNTZHQ00iLCJwMmMiOjEwMDAwMCwicDJzIjoiVERQS2dzcEItTUR4ZDJxTGo0VlpwdyJ9.2_j0cZgTm2eFkZ-hrtr1hBIvLxN0w3TZhbX0Jrrq7vBMaywhgFcGTA.mCasZCbZJ-JT7vjA.bW052WDKSf_ueEXq1dyxLq0n3qXWRO-LXr7OzBLdUKWKSBGQrzqS5KJWqdUCPoMIHTqpwYvm-iD6uFlcxKBYxnsAG_hoq_V3icvvwNQQSd_q7Thxr2_KtPIDJWNuX1t5qXp11hkgb-8d5HO93CmN7xNDG89pzSUepT6RYXOZ483mP5fre9qzkfnrjx3oPROCnf3SnIVUvqk7fwfXuniNsg3NrNqncHYUQNReiq3e9I1R60w0ZQTvIReY7-zfiq7iPgVqmu5I7XGgFK4iBv0L7UOEora65b4hRWeLxg5t7OCfUqrS9yxAk8FdjFb9sEfjopWViPRepB0dYPH8dVI.fb6-7XWqp0j6CR9Li0NI-Q",
"claims": {
"enableSSHCA": false,
"disableRenewal": false,
"allowRenewalAfterExpiry": false
},
"options": {
"x509": {},
"ssh": {}
}
},
{
"type": "ACME",
"name": "acme",
"forceCN": true,
"claims": {
"maxTLSCertDuration": "2160h0m0s",
"defaultTLSCertDuration": "2160h0m0s",
"policy": {
"x509": {
"allow": ["*.domain.org"]
}
}
},
"options": {
"x509": {
"templateFile": "templates/certs/x509/leaf.tpl"
}
}
}
],
"template": {},
"backdate": "1m0s"
},
"tls": {
"cipherSuites": [
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
],
"minVersion": 1.2,
"maxVersion": 1.3,
"renegotiation": false
},
"commonName": "Step Online CA"
}
```
This `ca.json` references a custom leaf cert template `leaf.tpl` to set Subject Alternative Name (SAN) in the provisioned TLS certificate. Check [here](https://smallstep.com/docs/step-ca/templates/) on how to configure your own Step CA templates.
The custom `leaf.tpl` as follows:
```
{
"subject": {{ toJson .subject }},
{{- if .Insecure.User.dnsName }}
"dnsNames": {{ toJSON .Insecure.User.dnsName }},
{{- else }}
"sans": {{ toJson .SANs }},
{{- end }}
{{-if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
"keyUsage": ["keyEncipherment", "digitalSignature"],
{{- else }}
"keyUsage": ["digitalSignature"],
{{- end }}
"extKeyUsage": ["serverAuth", "clientAuth]
}
```
We can then create the entire deployment YAML file including the injecting of configuration as secrets, and the referencing of the secrets from the deployment resource.
Our example deployment assumes an NGINX ingress controller is already deployed in the cluster.
The `deployment.yaml` is as follows:
```YAML
apiVersion: v1
kind: Secret
metadata:
name: stepca-config
namespace: stepca
type: Opaque
stringData:
intermediate_ca_crt: |
root_ca.crt: |
ca.json: |
{
"root": "/home/step/certs/root_ca.crt",
"federatedRoots": null,
"crt": "/home/step/certs/intermediate_ca.crt",
"key": "/home/step/certs/intermediate_ca.key",
"address": "9000",
"insecureAddress": "true",
"dnsNames": [
"localhost",
"ca.domain.org"
],
"logger": {
"format": "text"
},
"db":{
"type": "postgresql",
"dataSource": "postgresql://app:password@stepca-postgres-rw.stepca.svc.cluster.local:5432",
"database": "app"
},
"authority": {
"provisioners": [
{
"type": "JWK",
"name": "admin",
"key": {
"use": "sig",
"kty": "EC",
"kid": "YYNxZ0rq0WsT2MlqLCWvgme3jszkmt99KjoGEJJwAKs",
"crv": "P-256",
"alg": "ES256",
"x": "LsI8nHBflc-mrCbRqhl8d3hSl5sYuSM1AbXBmRfznyg",
"y": "F99LoOvi7z-ZkumsgoHIhodP8q9brXe4bhF3szK-c_w"
},
"encryptedKey": "eyJhbGciOiJQQkVTMi1IUzI1NitBMTI4S1ciLCJjdHkiOiJqd2sranNvbiIsImVuYyI6IkEyNTZHQ00iLCJwMmMiOjEwMDAwMCwicDJzIjoiVERQS2dzcEItTUR4ZDJxTGo0VlpwdyJ9.2_j0cZgTm2eFkZ-hrtr1hBIvLxN0w3TZhbX0Jrrq7vBMaywhgFcGTA.mCasZCbZJ-JT7vjA.bW052WDKSf_ueEXq1dyxLq0n3qXWRO-LXr7OzBLdUKWKSBGQrzqS5KJWqdUCPoMIHTqpwYvm-iD6uFlcxKBYxnsAG_hoq_V3icvvwNQQSd_q7Thxr2_KtPIDJWNuX1t5qXp11hkgb-8d5HO93CmN7xNDG89pzSUepT6RYXOZ483mP5fre9qzkfnrjx3oPROCnf3SnIVUvqk7fwfXuniNsg3NrNqncHYUQNReiq3e9I1R60w0ZQTvIReY7-zfiq7iPgVqmu5I7XGgFK4iBv0L7UOEora65b4hRWeLxg5t7OCfUqrS9yxAk8FdjFb9sEfjopWViPRepB0dYPH8dVI.fb6-7XWqp0j6CR9Li0NI-Q",
"claims": {
"enableSSHCA": false,
"disableRenewal": false,
"allowRenewalAfterExpiry": false
},
"options": {
"x509": {},
"ssh": {}
}
},
{
"type": "ACME",
"name": "acme",
"forceCN": true,
"claims": {
"maxTLSCertDuration": "2160h0m0s",
"defaultTLSCertDuration": "2160h0m0s",
"policy": {
"x509": {
"allow": ["*.domain.org"]
}
}
},
"options": {
"x509": {
"templateFile": "templates/certs/x509/leaf.tpl"
}
}
}
],
"template": {},
"backdate": "1m0s"
},
"tls": {
"cipherSuites": [
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256"
],
"minVersion": 1.2,
"maxVersion": 1.3,
"renegotiation": false
},
"commonName": "Step Online CA"
}
defaults.json: |
{
"ca-url": "https://localhost:9000",
"ca-config": "/home/step/config/ca.json",
"fingerprint": "93cff06dc36251fb0c4985d0b5ed7265a368cd70697fba90355c93cc4aabff0d",
"root": "/home/step/certs/root_ca.crt"
}
intermediate_ca_key: |
password: |
leaf.tpl: |
{
"subject": {{ toJson .subject }},
{{- if .Insecure.User.dnsName }}
"dnsNames": {{ toJSON .Insecure.User.dnsName }},
{{- else }}
"sans": {{ toJson .SANs }},
{{- end }}
{{-if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }}
"keyUsage": ["keyEncipherment", "digitalSignature"],
{{- else }}
"keyUsage": ["digitalSignature"],
{{- end }}
"extKeyUsage": ["serverAuth", "clientAuth]
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: stepca
namespace: stepca
spec:
replicas: 1
selector:
matchLabels:
app: stepca
template:
metadata:
labels:
app: stepca
spec:
containers:
- name: stepca
image: smallstep/step-ca:0.23.0
ports:
- containerPort: 9000
env:
- name: DOCKER_STEPCA_INIT_NAME
value: stepca
- name: DOCKER_STEPCA_INIT_DNS_NAMES
value: localhost, ca.domain.org
- name: DOCKER_STEPCA_INIT_PROVISIONER_NAME
value: admin
- name: DOCKER_STEP_CA_INIT_PASSWORD
valueFrom:
secretKeyRef:
name: stepca-config
key: password
volumeMounts:
- mountPath: /home/step
name: stepca-config
readOnly: false
volumes:
- name: stepca-config
secret:
secretName: stepca-config
items:
- key: intermediate_ca.crt
path: certs/intermediate_ca.crt
- key: root_ca.crt
path: certs/root_ca.crt
- key: ca.json
path: config/ca.json
- key: defaults.json
path: config/defaults.json
- key: intermediate_ca_key
path: secrets/intermediate_ca_key
- key: password
path: secrets/password
- key: leaf.tpl
path: templates/certs/x509/leaf.tpl
defaultMode: 0755
---
apiVersion: v1
kind: Service
metadata:
name: stepca
namespace: stepca
labels:
app: stepca
spec:
type: ClusterIP
ports:
- port: 9000
selector:
app: stepca
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: stepca-ingress
namespace: stepca
annotations:
kubernetes.io/ingress.class: 'nginx'
nginx.ingress.kubernetes.io/backend-protocol: 'HTTPS'
spec:
tls:
- hosts:
- ca.domain.org
secretName: stepca-tls
rules:
- host: ca.domain.org
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: stepca
port:
number: 9000
---
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: stepca
namespace: stepca
labels:
app: stepca
resource: horizontalpodautoscaler
spec:
maxReplicas: 6
minReplicas: 1
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: stepca
targetCPUUtilizationPercentage: 80
```
You check the health of your Intermediate Certificate Authority:
```bash
curl https://ca.domain.org/health
# {"status":"ok"}
curl https://ca.domain.org/acme/acme/directory
# {"newNonce":"https://ca.domain.org/acme/acme/new-nonce","newAccount":"https://ca.domain.org/acme/acme/new-account","newOrder":"https://ca.domain.org/acme/acme/new-order","revokeCert":"https://ca.domain.org/acme/acme/revoke-cert","keyChange":"https://ca.domain.org/acme/acme/key-change"}
```
In our own private infrastructure environment, we often need to use our own self-signed TLS certificates to serve our sites over HTTPS.
Step CA can help you generate TLS certificates for your sites using the ACME protocol, and automate the TLS certificate renewal process as well.
In this post we will walk though the process of deploying a PostgreSQL cluster on Kubernetes, then deploying a Step CA Intermediate Certificate Authority that will use the PostgreSQL cluster as the database.
Here is how the final architecture will look like:
## Preparation To start the entire deployment, let's create the `stepca` Kubernetes namespace where everything will be deployed to: ```bash kubectl create ns stepca ``` Sign a leaf TLS cert for the domain where you will be hosting your Step CA. We can inject the TLS secret as follows: ```bash kubectl create secret tls stepca-tls -n stepca --key private.key --cert public.crt ``` Download the `step` binary [here](https://github.com/smallstep/cli/releases) and place the binary in `/usr/bin` Then generate an intermediate certificate signing request: ```bash step certificate create "Intermediate CA Name" intermediate.csr intermediate_ca_key --csr ``` Transfer the certificate signing request to your existing root CA and get it signed. You should have the `root_ca.crt` from your existing root CA, `intermediate_ca.crt` from signing the certificate signing request, and `intermediate_ca.key` that was created when you generated the intermediate certificate signing request from the previous step. ## Deploying a PostgreSQL Cluster There are many available PostgreSQL Operators that can help you lifecycle a highly available PostgreSQL cluster. In this example, we will be using CloudNativePG. I like CloudNativePG because the operator creates the database instances using `Pods` instead of `StatefulSets`, which avoids all the limitations that comes with `StatefulSets`. To install the CloudNativePG Operator, run the following command: ```bash kubectl apply -f \ https://raw.githubusercontent.com/cloudnative-pg/cloudnative-pg/release-1.22/releases/cnpg-1.22.0.yaml ``` You can verify the Operator is installed with: ```bash kubectl get deployment -n cnpg-system cnpg-controller-manager ``` Here is the YAML deployment file to create the Postgres Cluster. In this example we will be storing our postgres backups to our own MinIO instance hosted at `minio.domain.org` ```YAML # Inject MinIO Credentials as Secret apiVersion: v1 kind: Secret metadata: name: minio namespace: stepca type: Opaque stringData: ACCESS_KEY_ID: minio # S3 username ACCESS_SECRET_KEY: minio123 # S3 password --- # Step CA Deployment YAML apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: stepca-postgres namespace: stepca spec: imageName: ghcr.io/cloudnative-pg/postgresql:15.0 instances: 3 primaryUpdateStrategy: unsupervised # Rolling update process to be automated and managed by Kubernetes monitoring: enablePodMonitor: true # Expose prometheus metrics storage: storageClass: vsan-default-storage-policy # Define the storage class you use in your Kubernetes cluster size: 1Gi backup: # Configure to use S3 to store backup resources barmanObjectStore: destinationPath: s3://stepca/ # S3 bucket location endpointURL: https://minio.domain.org # S3 endpoint s3Credentials: accessKeyId: name: minio key: ACCESS_KEY_ID secretAccessKey: name: minio key: ACCESS_SECRET_KEY wal: compression: gzip encryption: AES256 retentionPolicy: "7d" ``` You should see the following Kubernetes resources once your PostgreSQL cluster is created: Services: - stepca-postgres-r - applications to connect to any of the instances for read-only workloads - stepca-postgres-ro - applications to connect to any of the hot standby, non-primary replicas for read-only workloads - stepca-postgres-rw - applications to connect to the primary instance for read-write workloads Secrets: - stepca-postgres-app - database credentials for the default user called `app`, corresponds to the user owning the database - stepca-postgres-ca - self-signed CA generated and used to support TLS within the postgres cluster - stepca-postgres-replication - streaming replication client certificate generated by the client CA - stepca-postgres-server - server TLS certificate signed by the server CA - stepca-postgres-superuser - superuser credentials to be used only for administrative purposes, corresponds to the `postgres` user - stepca-postgres-token - kubernetes service account created for the database operator Monitoring: - When enablePodMonitor is set to `true`, CloudNativePG will automatically expose prometheus metrics relating to CloudNativePG clusters, and create a `PodMonitor` resource for your prometheus to scrape the endpoint - The pre-requisite is that you must have prometheus already installed ### PostgreSQL Backups For on-demand backups, apply the following `YAML`: ``` apiVersion: postgresql.cnpg.io/v1 kind: Backup metadata: name: backup-example spec: cluster: name: stepca-postgres ``` To schedule backups, apply the following `YAML`: ```YAML apiVersion: postgresql.cnpg.io/v1 kind: ScheduledBackup metadata: name: backup-daily-midnight namespace: stepca spec: schedule: "0 0 16 * * *" # 0000 SGT in UTC time backupOwnerReference: self cluster: name: stepca-postgres ``` To restore backup from S3 object store, apply the following `YAML`: ```YAML apiVersion: postgresql.cnpg.io/v1 kind: Cluster metadata: name: stepca-postgres namespace: stepca spec: imageName: ghcr.io/cloudnative-pg/postgresql:15.0 instances: 3 primaryUpdateStrategy: unsupervised # Rolling update process to be automated and managed by Kubernetes monitoring: enablePodMonitor: true # Expose prometheus metrics storage: storageClass: vsan-default-storage-policy # Define the storage class you use in your Kubernetes cluster size: 1Gi backup: # Configure to use S3 to store backup resources barmanObjectStore: destinationPath: s3://stepca/ # S3 bucket location endpointURL: https://minio.domain.org # S3 endpoint s3Credentials: accessKeyId: name: minio key: ACCESS_KEY_ID secretAccessKey: name: minio key: ACCESS_SECRET_KEY wal: compression: gzip encryption: AES256 retentionPolicy: "7d" bootstrap: recovery: source: stepca-postgres recoveryTarget: # Use backupID and targetImmediate to backup to that instant and stop immediately, or use targetTime to do point-in-time recovery where the database will run through the WAL to the timestamp specified after restoring from the nearest base backup backupID: 20240102T160000 targetTime: "2024-01-02T09:00:25" externalClusters: - name: stepca-postgres # Name has to be same as the previous cluster since barman will be searching for backups based on the name barmanObjectStore: destinationPath: s3://stepca/ endpointURL: https://minio.domain.org # S3 endpoint s3Credentials: accessKeyId: name: minio key: ACCESS_KEY_ID secretAccessKey: name: minio key: ACCESS_SECRET_KEY wal: maxParallel: 8 # Take advantage of the parallel WAL restore feature to dedicate up to 8 concurrent jobs to fetch required WAL files from the archive ``` ## Deploying Step CA Step CA includes a helm chart to deploy on Kubernetes, but there is a disclaimer that they can only support 1 replica instance. Therefore we will not be using their helm chart, and instead we take the following deployment path: 1. Run a Step CA on Docker 2. Retrieve the configuration template to use in our actual deployment 3. Deploy our Step CA with our specified configuration injected as a Kubernetes secret To run Step CA on Docker, we can use the following `docker-compose.yml` file: ```YAML version: '3.3' services: ca: image: smallstep/step-ca:0.24.1 networks: - default ports: - "9000:9000" environment: - DOCKER_STEPCA_INIT_NAME=${DOCKER_STEPCA_INIT_NAME} # Name of your CA - this will be the issuer of your CA certificates - DOCKER_STEPCA_INIT_DNS_NAMES=${DOCKER_STEPCA_INIT_DNS_NAMES} # Hostname(s) or IPs that the CA will accept requests on - DOCKER_STEPCA_INIT_PROVISIONER_NAME=${DOCKER_STEPCA_INIT_PROVISIONER_NAME} # Label for the initial admin (JWK) provisioner. Default: "admin" - DOCKER_STEPCA_INIT_PASSWORD=${DOCKER_STEPCA_INIT_PASSWORD} # Password for the encrypted CA keys and the default CA provisioner volumes: - ./data/home/step:/home/step restart: always networks: default: ipam: driver: default config: - subnet: "172.97.0.0/16" volumes: step_home: ``` From here, we can retrieve the configuration files from `/home/step/`, then edit the template to use the PostgreSQL cluster that we deployed earlier. A reference of the configuration options can be found [here](https://smallstep.com/docs/step-ca/configuration/#basic-configuration-options) A customised `config/ca.json` is as follows: ```JSON { "root": "/home/step/certs/root_ca.crt", "federatedRoots": null, "crt": "/home/step/certs/intermediate_ca.crt", "key": "/home/step/certs/intermediate_ca.key", "address": "9000", "insecureAddress": "true", "dnsNames": [ "localhost", "ca.domain.org" ], "logger": { "format": "text" }, "db":{ "type": "postgresql", "dataSource": "postgresql://app:password@stepca-postgres-rw.stepca.svc.cluster.local:5432", "database": "app" }, "authority": { "provisioners": [ { "type": "JWK", "name": "admin", "key": { "use": "sig", "kty": "EC", "kid": "YYNxZ0rq0WsT2MlqLCWvgme3jszkmt99KjoGEJJwAKs", "crv": "P-256", "alg": "ES256", "x": "LsI8nHBflc-mrCbRqhl8d3hSl5sYuSM1AbXBmRfznyg", "y": "F99LoOvi7z-ZkumsgoHIhodP8q9brXe4bhF3szK-c_w" }, "encryptedKey": "eyJhbGciOiJQQkVTMi1IUzI1NitBMTI4S1ciLCJjdHkiOiJqd2sranNvbiIsImVuYyI6IkEyNTZHQ00iLCJwMmMiOjEwMDAwMCwicDJzIjoiVERQS2dzcEItTUR4ZDJxTGo0VlpwdyJ9.2_j0cZgTm2eFkZ-hrtr1hBIvLxN0w3TZhbX0Jrrq7vBMaywhgFcGTA.mCasZCbZJ-JT7vjA.bW052WDKSf_ueEXq1dyxLq0n3qXWRO-LXr7OzBLdUKWKSBGQrzqS5KJWqdUCPoMIHTqpwYvm-iD6uFlcxKBYxnsAG_hoq_V3icvvwNQQSd_q7Thxr2_KtPIDJWNuX1t5qXp11hkgb-8d5HO93CmN7xNDG89pzSUepT6RYXOZ483mP5fre9qzkfnrjx3oPROCnf3SnIVUvqk7fwfXuniNsg3NrNqncHYUQNReiq3e9I1R60w0ZQTvIReY7-zfiq7iPgVqmu5I7XGgFK4iBv0L7UOEora65b4hRWeLxg5t7OCfUqrS9yxAk8FdjFb9sEfjopWViPRepB0dYPH8dVI.fb6-7XWqp0j6CR9Li0NI-Q", "claims": { "enableSSHCA": false, "disableRenewal": false, "allowRenewalAfterExpiry": false }, "options": { "x509": {}, "ssh": {} } }, { "type": "ACME", "name": "acme", "forceCN": true, "claims": { "maxTLSCertDuration": "2160h0m0s", "defaultTLSCertDuration": "2160h0m0s", "policy": { "x509": { "allow": ["*.domain.org"] } } }, "options": { "x509": { "templateFile": "templates/certs/x509/leaf.tpl" } } } ], "template": {}, "backdate": "1m0s" }, "tls": { "cipherSuites": [ "TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256", "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256" ], "minVersion": 1.2, "maxVersion": 1.3, "renegotiation": false }, "commonName": "Step Online CA" } ``` This `ca.json` references a custom leaf cert template `leaf.tpl` to set Subject Alternative Name (SAN) in the provisioned TLS certificate. Check [here](https://smallstep.com/docs/step-ca/templates/) on how to configure your own Step CA templates. The custom `leaf.tpl` as follows: ``` { "subject": {{ toJson .subject }}, {{- if .Insecure.User.dnsName }} "dnsNames": {{ toJSON .Insecure.User.dnsName }}, {{- else }} "sans": {{ toJson .SANs }}, {{- end }} {{-if typeIs "*rsa.PublicKey" .Insecure.CR.PublicKey }} "keyUsage": ["keyEncipherment", "digitalSignature"], {{- else }} "keyUsage": ["digitalSignature"], {{- end }} "extKeyUsage": ["serverAuth", "clientAuth] } ``` We can then create the entire deployment YAML file including the injecting of configuration as secrets, and the referencing of the secrets from the deployment resource. Our example deployment assumes an NGINX ingress controller is already deployed in the cluster. The `deployment.yaml` is as follows: ```YAML apiVersion: v1 kind: Secret metadata: name: stepca-config namespace: stepca type: Opaque stringData: intermediate_ca_crt: |