search

concourse / hush-house

Concourse k8s-based environment
https://hush-house.pivotal.io
29 stars 23 forks source link

Have log-based metric as code (terraform) #104

Open cirocosta opened 4 years ago

cirocosta commented 4 years ago

Hey,

The dashboards that display error metrics (web and worker) are based on the log messages that get ingested and processed by Stackdriver:

Screen Shot 2020-01-09 at 9 38 09 AM

https://github.com/concourse/hush-house/blob/a3f7662a65cc32ea0de47e31851e577564455a4a/deployments/with-creds/metrics/dashboards/concourse/concourse.json#L620-L622

While that has served us well, having that not described as code makes it very easy to break, and hard to keep track of.

It turns out that we can leverage Terraform to describe such metrics:

https://www.terraform.io/docs/providers/google/r/logging_metric.html

The goal here is to port those that we already have:

Screen Shot 2020-01-09 at 9 36 33 AM

Thanks!

gowrisankar22 commented 4 years ago

@cirocosta basically, we also design the same dashboard. Somehow I am missing the information w.r.t creating the log metrics in the GCP stack driver. Can you help me out if you have terraform code for this, where I can find this or if it was done manually, can give short document so that i can try out ? https://metrics-hush-house.concourse-ci.org/d/1mUBdWWWk/stackdriver?orgId=1&refresh=1m

gowrisankar22 commented 4 years ago

I was able to achieve this. BTW screenshot mentioned above helped me to achieve my goal thanks ..