HELM Chart for Puppet Server

Introduction

This chart bootstraps Puppet Server and its components on a Kubernetes cluster using the Helm package manager.

Prerequisites

Code Repos

• You must specify your Puppet Control Repo using puppetserver.puppeturl variable in the values.yaml file or include --set puppetserver.puppeturl=<your_public_repo> in the command line of helm install. You can specify your separate Hieradata Repo as well using the hiera.hieradataurl variable.

• You can also use private repos. Just remember to specify your credentials using r10k.code.viaSsh.credentials.ssh.value or r10k.code.viaHttps.credentials.netrc.value. You can set similar credentials for your Hieradata Repo.

Load-Balancing Puppet Server

In case a Load Balancer (LB) must sit in front of Puppet Server - please keep in mind that having a Network LB (operating at OSI Layer 4) is preferable.

NGINX Ingress Controller Configuration

The Ingress resource is disabled by default, but if it is enabled then ssl-passthrough must be used so that puppet agents will get the expected server certificate when connecting to the service. This feature must be enabled on the Ingress resource itself, but also must be enabled via command line argument to the NGINX Ingress Controller. More information on that can be found here.

NOTE: Ingress URLs must be passed in the Values.puppetserver.masters.fqdns.alternateServerNames. Also - in the Values.puppetserver.compilers.fqdns.alternateServerNames (if Puppet Compilers and their Ingress resources are deployed).

Migrating from Bare-Metal Puppet Infrastructure

Auto-Signing Certificate Requests

In general, the easiest way to switch the Puppet Agents from using one Puppet master to another is by enabling the auto-signing of CSRs. By default, that has been pre-enabled in the Puppet Server Docker container. It can be disabled in the Values file by passing an extra environment variable: AUTOSIGN=false (in .Values.puppetserver.masters.extraEnv).

You will also need to remove the existing certificates in /etc/puppetlabs/puppet/ssl on each Puppet agent.

Using Pre-Generated Puppet Master Certificates

If you prefer not to auto-sign or manually sign the Puppet Agents' CSRs - you can use the same Puppet master and PuppetDB certificates which you used in your bare-metal setup. Please archive into two separate files and place your certificates in the init/puppet-certs/puppetserver and init/puppet-certs/puppetdb directories and enable their usage in the Values file (.Values.puppetserver.preGeneratedCertsJob.enabled).

NOTE: For more information please check - README.md. For more general knowledge on the matter you can also read the article - https://puppet.com/docs/puppet/5.5/ssl_regenerate_certificates.html.

Using Single CA

If you prefer, you can use a single externally issued CA - https://puppet.com/docs/puppet/7/config_ssl_external_ca.html. Enable it with .Values.singleCA.enable, add the crl.pem url with .Values.singleCA.crl.url.

Generate puppet & puppetdb secret (must be name puppet.pem & puppetdb.pem):

kubectl create secret generic puppet-certificate --from-file=puppet.pem --from-file=puppet.key --from-file=ca.pem
kubectl create secret generic puppetdb-certificate --from-file=puppetdb.pem --from-file=puppetdb.key --from-file=ca.pem

finally set .Values.singleCA.certificates.existingSecret.puppetserver and .Values.singleCA.certificates.existingSecret.puppetdb.

Additionnaly, if you use a public certificate authority, you can't use private SAN name, so you have to override puppetdb name with .Values.singleCA.puppetdb.overrideHostname (with the full name ie: puppetdb.my.domain)

If you prefer, you can use crl update as cronjob instead of sidecar, it reduce resources utilization because only 1 pod is running. :warning: it may not work on multi zone cluster. that why it's not enable by default

Horizontal Scaling

To achieve better availability and higher throughput of Puppet Infrastructure, you'll need to scale out Puppet Masters and/or Puppet Compilers.

Multiple Puppet Masters

To achieve better availability of Puppet Infrastructure, you can scale out Puppet Server Masters using .Values.puppetserver.masters.multiMasters. These Servers are known as masters, and are responsible for the creation and signing of your Puppet Agents' certificates. They are also responsible for receiving catalog requests from agents and synchronize the results with each other.

Multiple Puppet Compilers

To achieve better throughput of Puppet Infrastructure, you can enable and scale out Puppet Server Compilers using .Values.puppetserver.compilers. These Servers are known as compile masters, and are simply additional load-balanced Puppet Servers that receive catalog requests from agents and synchronize the results with each other.

Multiple PostgreSQL Read Replicas

For now it's not available anymore, since bitnami cleanned their old release. for multiple Postgresql we have to use postgresql-ha. Read replica return an error on puppetdb: ERROR [p.p.c.services] Will retry database connection after temporary failure: java.sql.SQLTransientConnectionException: PDBMigrationsPool: default - Connection is not available, request timed out after 3002ms.

Deploy R10K as deployment

:warning: may not work in multi zone environment. that why it's not enable by default

You can configure r10k to run as deployment instead of sidecar to avoid r10k container multiplication (and avoid r10k run conflict) to share r10k data between all componant (master or compiler), change the following values:

# values.yaml

# change this only if you use compilers
puppetserver:
    compilers:
        kind: Deployment

r10k:
  asSidecar: false

Deploy Puppetserver deployment (master & compilers) as non root

:warning: for now only puppetserver can run as non root, it's not available for the puppetdb It will run a pre-install job to configure all repository & permissions for masters & compilers

Benefits:

• running puppetserver with limited permissions
• improve puppetserver deployment (because certificate are not regenerated each time)

You can enable it using:

global.runAsNonRoot: true

Chart Components

• Creates three deployments: Puppet Server Master/s, and PuppetDB.
• Creates three statefulsets (optional): Puppet Server Compiler/s, PostgreSQL Master, and PostgreSQL Read Replicas.
• Creates seven services that expose: Puppet Server Masters, Puppet Server Compilers (optional), PuppetDB, PostgreSQL, and Puppetboard (optional).
• Creates secrets to hold credentials for PuppetDB, PosgreSQL, and r10k.

Installing the Chart

Add Puppet Server Helm Repository

Before installing Puppet Server Helm chart, you need to add the Puppet Server Helm repository to your Helm client as below.

helm repo add puppet https://puppetlabs.github.io/puppetserver-helm-chart

Install the Chart

To install the chart with the release name puppetserver.

helm install --namespace puppetserver --name puppetserver puppet/puppetserver --set puppetserver.puppeturl='https://github.com/$SOMEUSER/control-repo.git'

Note - If you do not specify a name, helm will select a name for you.

Installed Components

You can use kubectl get to view all of the installed components.

$ kubectl get --namespace puppetserver all
NAME                                                   READY   STATUS    RESTARTS   AGE
pod/puppetserver-postgresql-master-0                   1/1     Running   0          9m25s
pod/puppetserver-postgresql-slave-0                    1/1     Running   0          9m25s
pod/puppetserver-puppetdb-fc4978b6d-z9bsx              2/2     Running   0          9m25s
pod/puppetserver-puppetserver-compiler-0               2/2     Running   0          9m25s
pod/puppetserver-puppetserver-master-7b59cb946-9bwv6   2/2     Running   0          9m25s

NAME                                       TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                    AGE
service/agents-to-puppet                   ClusterIP   10.96.236.202   <none>        8140/TCP                   9m25s
service/puppet                             ClusterIP   10.96.51.23     <none>        8140/TCP                   9m25s
service/puppet-compilers                   ClusterIP   10.96.185.43    <none>        8140/TCP                   9m25s
service/puppet-compilers-headless          ClusterIP   None            <none>        443/TCP                    9m25s
service/puppetdb                           ClusterIP   10.96.160.91    <none>        8080/TCP,8081/TCP,80/TCP   9m25s
service/puppetserver-postgresql            ClusterIP   10.96.48.166    <none>        5432/TCP                   9m25s
service/puppetserver-postgresql-headless   ClusterIP   None            <none>        5432/TCP                   9m25s
service/puppetserver-postgresql-read       ClusterIP   10.96.219.155   <none>        5432/TCP                   9m25s

NAME                                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/puppetserver-puppetdb              1/1     1            1           9m25s
deployment.apps/puppetserver-puppetserver-master   1/1     1            1           9m25s

NAME                                                         DESIRED   CURRENT   READY   AGE
replicaset.apps/puppetserver-puppetdb-fc4978b6d              1         1         1       9m25s
replicaset.apps/puppetserver-puppetserver-master-7b59cb946   1         1         1       9m25s

NAME                                                  READY   AGE
statefulset.apps/puppetserver-postgresql-master       1/1     9m25s
statefulset.apps/puppetserver-postgresql-slave        1/1     9m25s
statefulset.apps/puppetserver-puppetserver-compiler   1/1     9m25s

NAME                                                                    REFERENCE                                         TARGETS                        MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/puppetserver-compilers-autoscaler   StatefulSet/puppetserver-puppetserver-compilers   33%/75%, 47%/75%               1         3         0          9m25s

Upgrading

from 7.x to 8.0

customPersistentVolumeClaim was removed instead check puppetdb.persistence.existingClaim or puppetserver.persistence.*.existingClaim for similar functionnality.

Added support for setting the size and Storage Class of individual Persistent Volume Claims fullfilling most uses of customPersistentVolumeClaim check puppetdb.persistence.size puppetdb.persistence.storageClass, puppetserver.persistence.*.storageClass and puppetserver.persistence.*.size for more information.

Configuration

The following table lists the configurable parameters of the Puppetserver chart and their default values.

Specify each parameter using the --set key=value[,key=value] argument to helm install. For example,

helm install --namespace puppetserver --name puppetserver puppet/puppetserver --set puppetserver.puppeturl='https://github.com/$SOMEUSER/puppet.git',hiera.hieradataurl='https://github.com/$SOMEUSER/hieradata.git'

Alternatively, a YAML file that specifies the values for the above parameters can be provided while installing the chart. For example,

helm install --namespace puppetserver --name puppetserver puppet/puppetserver -f values.yaml

Tip: You can use the default values.yaml

Testing Helm Chart (in progress)

https://github.com/quintush/helm-unittest/

It would be great to test all ressources to avoid regression in the future

run test:

helm unittest . -3

Testing the Deployed Chart Resources

kubectl port-forward -n puppetserver svc/agents-to-puppet 8140:8140 &
kubectl port-forward -n puppetserver svc/puppet-compilers 8141:8140 &

TIME_NOW="$(date +"%Y%m%dT%H%M")"
cp "/etc/hosts"{,.backup_"$TIME_NOW"}
echo '127.0.0.1 puppet agents-to-puppet puppet-compilers' >> /etc/hosts
# if Ingress is used, e.g.
# INGRESS_IP="$(kubectl -n puppetserver get ingress | grep '\-masters' | tr -s " " | cut -d' ' -f4 | cut -d',' -f1)"
# echo "$INGRESS_IP" puppet.local.masters puppet.local.compilers >> /etc/hosts

docker run -dit --network host --name goofy_xtigyro --entrypoint /bin/bash puppet/puppet-agent
docker exec -it goofy_xtigyro bash
puppet agent -t --server puppet --masterport 8140 --waitforcert 15 --summarize --certname ubuntu-goofy_xtigyro
puppet agent -t --server puppet-compilers --ca_server agents-to-puppet --masterport 8141 --ca_port 8140 --summarize --certname ubuntu-goofy_xtigyro
# if Ingress is used, e.g.
# puppet agent -t --server puppet.local.compilers --ca_server puppet.local.masters --masterport 443 --ca_port 443 --summarize --certname ubuntu-goofy_xtigyro
puppet agent -t --server puppet-compilers --masterport 8141 --summarize --certname ubuntu-goofy_xtigyro
exit
docker rm -f goofy_xtigyro

docker run -dit --network host --name buggy_xtigyro --entrypoint /bin/bash puppet/puppet-agent
docker exec -it buggy_xtigyro bash
puppet agent -t --server puppet-compilers --ca_server agents-to-puppet --masterport 8141 --ca_port 8140 --summarize --certname ubuntu-buggy_xtigyro
puppet agent -t --server puppet-compilers --masterport 8141 --summarize --certname ubuntu-buggy_xtigyro
# if Ingress is used, e.g.
# puppet agent -t --server puppet.local.compilers --ca_server puppet.local.masters --masterport 443 --ca_port 443 --summarize --certname ubuntu-buggy_xtigyro
puppet agent -t --server puppet --masterport 8140 --waitforcert 15 --summarize --certname ubuntu-buggy_xtigyro
exit
docker rm -f buggy_xtigyro

yes | mv "/etc/hosts.backup_"$TIME_NOW"" "/etc/hosts"
unset TIME_NOW

jobs | grep 'port-forward' | grep 'puppetserver'
# [1]+  Running                 kubectl port-forward -n puppetserver svc/puppet 8140:8140 &
kill %[job_numbers_above]

Credits

• Miroslav Hadzhiev, Lead Author and Owner
• David Phayanouvong, Maintainer
• Pupperware Team, Co-Author and Owner
• Morgan Rhodes, Maintainer
• Sean Conley, Maintainer
• Raphaël Pinson, Maintainer
• Reinier Schoof, Maintainer
• Niels Højen, Maintainer
• Scott Cressi, Co-Author
• Linas Daneliukas, Maintainer
• Anthony Somerset, Maintainer
• Kai Sisterhenn, Contributor
• chwehrli, Contributor
• Hryhorii Didenko, Contributor
• John Stewart, Contributor
• Erlon Pinheiro, Contributor
• Manasseh MMadu, Contributor
• Aidan, Contributor
• Aurélien Le Clainche, Contributor
• Simon Fuhrer, Contributor
• Kevin Harrington, Contributor
• Grégoire Menuel, Contributor
• Jean-François Roche, Contributor
• Alexander Kiryushin, Contributor
• Ben Feld, Contributor
• Julien Godin, Contributor
• Diego Abelenda, Contributor
• Johann Agnarsson, Contributor