pixie-io / pixie

Instant Kubernetes-Native Application Observability
https://px.dev
Apache License 2.0
5.63k stars 433 forks source link

Unhealthy cluster / possible PEM health check issue. #184

Closed htroisi closed 3 years ago

htroisi commented 3 years ago

@jdotjdot installed Pixie and is seeing the following issues:

image

@nserrino took a quick look at the logs: "It looks like the PEMs are taking more than 5 seconds to send their results to the Kelvin, which shouldn't be happening. Looks like kelvin is correctly sending health check results back, but query broker is interpreting them as invalid for some reason."

App information (please complete the following information): self-hosted on AWS using kops. v1.18.8 we use spotinst for control plane / scheduling

zasgar commented 3 years ago

Stale issue.