Open Montana opened 4 years ago
Hi @Montana I just met the same challenge when trying to patch the Prometheus with Stackdriver sidecar.
Then I found out that to patch prometheus managed by Prometheus-Operator, you need to update your Prometheus
object(CRD). There is a field call containers
for you to inject sidecars.
Once you update the Prometheus
yaml. The prometheus operator will update the statefulset accordingly, inject the sidecar for you.
apiVersion: monitoring.coreos.com/v1
kind: Prometheus
...
spec:
...
containers:
- args:
-
image: gcr.io/stackdriver-prometheus/stackdriver-prometheus-sidecar:0.8.0
imagePullPolicy: Always
name: sidecar
ports:
- containerPort: 9091
name: sidecar
volumeMounts:
- mountPath: /prometheus
name: prometheus-k8s-db
If you are using Helm chart, there should be a value for that. If you are using kube-promtheus you can update using jsonnet.
Hey @kehao95,
Thank you so much! This did the trick.
@kehao95 @Montana I have tried using this too. However, I am just getting the metrics from the prometheus namespace.
Hey Folks Montana Mendy here,
I have setup Prometheus + Grafana on our GKE cluster using the stable/prometheus-operator helm chart. Now I want to export some metrics to Stackdriver because I have installed custom metrics Stackdriver adapter. I'm already using some Pub/Sub metrics from Stackdriver for autoscaling few deployments. Now I also want to use some Prometheus metrics (mainly nginx request rate) in the autoscaling of other deployments.
So, my first question: Can I use Prometheus adapter in parallel with Stackdriver adapter for autoscaling in the same cluster?
If not, I will need to install Stackdriver Prometheus Sidecar for exporting the Prometheus metrics to Stackdriver and then use them for autoscaling via Stackdriver adapter.
From the instructions here, it looks like I need to install Stackdriver sidecar on same pod on which Prometheus is running. I gave it a try. When I run the
patch.sh
script, I got the message back:statefulset.apps/prometheus-prom-operator-prometheus-o-prometheus patched
but when I inspected thestatefulset
again, it didn't have the Stackdriver sidecar container in it. Since thisstatefulset
is created by a Helm chart, I probably can't modify it directly. Is there a recommended way of doing this in Helm?I thought I could in theory update the deployment by updating the helm template or something like
kuebctl get statefulset prometheus-prom-operator-prometheus-o-prometheus -o yaml --export file.yaml
, add the required fields thenkubectl apply -f file.yaml
.I've found the volume that is mounted on this container on same path. In my case, it was
prometheus-prom-operator-prometheus-o-prometheus-db
so I mounted the same volume on my Stackdriver sidecar container as well. Still no dice, I also ran:helm list
(for obvious reasons) then ran:helm upgrade -f prom-config.yaml prom-operator stable/prometheus-operator
.Thanks, Montana