search

GoogleCloudPlatform / cloud-ops-sandbox

Cloud Operations Sandbox is an open source collection of tools that helps practitioners to learn O11y and R9y practices from Google and apply them using Cloud Operations suite of tools.
Apache License 2.0
406 stars 148 forks source link

profiler does not work #898

Closed rick-c-goog closed 2 years ago

rick-c-goog commented 2 years ago

Based on instructions, The following Hipster Shop microservices are configured to capture profiling data:

Checkout service Currency service Frontend Payment service Product-catalog service Shipping service

But none of service show up in Monitoring /profiler.

The code is in place, e.g. Golang code for Checkout Service func main() { ctx := context.Background() go initOpenCensusStats() initTraceProvider() go initProfiling("checkoutservice", "1.0.0")

func initProfiling(service, version string) { // TODO(ahmetb) this method is duplicated in other microservices using Go // since they are not sharing packages. for i := 1; i <= 3; i++ { if err := profiler.Start(profiler.Config{ Service: service, ServiceVersion: version, // ProjectID must be set if not running on GCP. // ProjectID: "my-project", }); err != nil { log.Warnf("failed to start profiler: %+v", err) } else { log.Info("started stackdriver profiler") return } d := time.Second 10 time.Duration(i) log.Infof("sleeping %v to retry initializing stackdriver profiler", d) time.Sleep(d) } log.Warn("could not initialize stackdriver profiler after retrying, giving up") }

rick-c-goog commented 2 years ago

It turned out to be a permission issue with the default Service account with GKE node vms should have cloud profiler agent permission.

losalex commented 2 years ago

Thank you @rick-c-goog for filing an issue and really sorry for late response. Can you please confirm that your issue is resolved?

daniel-sanche commented 2 years ago

This should be resolved in https://github.com/GoogleCloudPlatform/cloud-ops-sandbox/pull/931