expose metrics of the operator with tls (https) certificates

[kaushiksrinivas]

Building operator using operator-sdk framework. Do not see much information about exposing metrics with tls enabled. Is it supported to expose metrics with tls (configurable tls certificates) and if yes, how to configure certificates for the metrics port with tls on the operator pod ?

Is there any documentation pages and configuration links/samples required to achieve this ?

[kaushiksrinivas]

@camilamacedo86 Can you provide any inputs here ?

[camilamacedo86]

Hi @kaushiksrinivas,

The metrics are exported by default in the entrypoint metrics.

How does it work?

See that in the manager ( main.go ) when we init the Operator we expose that: https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/main.go#L70

Then, if you follow up on the docs we recommend using it with the Prometheus Operator, see: https://book.kubebuilder.io/reference/metrics.html

Therefore, if you look at the default scaffolds you will find:

• The ServiceMonitor: https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/config/prometheus/monitor.yaml#L10-L20 where we are in my understanding skipping the tls
• Then, we have this cluster role to allow expose the metrics: https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/config/rbac/auth_proxy_client_clusterrole.yaml

Note that all projects are scaffolded by default with a side-car container which is a proxy to protect the manager. See: https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/config/default/manager_auth_proxy_patch.yaml#L11-L39

What/where do you probably need to address the changes?

To work with tls certificates you need to pass this option for the kube-rbac-proxy. You can check this project and how it works here: https://github.com/brancz/kube-rbac-proxy.

• See the tlsConfig options: https://github.com/brancz/kube-rbac-proxy/blob/master/main.go#L68-L74

Then, it seems that you will need to customize the following scaffolds:

• (The ServiceMonitor to pass the tlsConfig): https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/config/prometheus/monitor.yaml#L10-L20
• (The proxy to pass it is as arg) https://github.com/operator-framework/operator-sdk/blob/master/testdata/go/v3/memcached-operator/config/default/manager_auth_proxy_patch.yaml#L11-L39
• Also, you probably need to pass it to the manager and change the main.go (https://github.com/brancz/kube-rbac-proxy/blob/master/main.go#L68-L74). It seems to be with the parmeter defined here: https://github.com/kubernetes-sigs/controller-runtime/blob/196828e54e4210497438671b2b449522c004db5c/pkg/manager/manager.go#L237-L242. But it is required check better in the controller-runtime the options. Also, it seems like C+R has some tests checking it which can help out to know how to config.

CAVEAT: If you are using webhooks, then you need to get the controller-runtime latest release see (v0.12.2 ): https://github.com/kubernetes-sigs/controller-runtime/releases/tag/v0.12.2

About SDK docs and instructions to achieve the goal:

However, we do have not a doc that describes how to do it. Also, would be great if you could contribute with SDK and others after you do the changes for this config by:

• a) at least adding here the changes made in the default scaffold with an example to achieve the goal
• b) OR better yet, if you could add a doc for that in the section: https://sdk.operatorframework.io/docs/building-operators/golang/advanced-topics/

I hope that can help you out.

[varshaprasad96]

In case more info is needed from controller-runtime's end on configuring custom tls certificates for metrics: https://github.com/kubernetes-sigs/controller-runtime/pull/993. It can passed from here: https://github.com/kubernetes-sigs/controller-runtime/blob/196828e54e4210497438671b2b449522c004db5c/pkg/manager/manager.go#L237-L242

[varshaprasad96]

@kaushiksrinivas please let us know if we have answered the question. I'm closing this issue for now, please feel free to reopen.