canonical / mysql-operator

Machine charm for MySQL following the operator framework
https://charmhub.io/mysql
Apache License 2.0
6 stars 8 forks source link

[DPE-4367] Add support for tracing through tempo-k8s #456

Closed shayancanonical closed 1 month ago

shayancanonical commented 1 month ago

Issue

We would like to be able to instrument our charm and generate traces to discover inefficiencies in charm code execution. The observability team has developed a charm for tempo that we would like to integrate with.

Solution

Integrate with the tempo charm

Considerations

1. We added 2 new relation interfaces: one for tracing to be related with tempo-k8s and one tracing-certificates to be related with self-signed-certificates (or another certificates operator). The tracing-certificates relation is used to retrieve (with the help of the obsevability_libs CertHandler) the server ca cert for tempo-k8s when it has HTTPS enabled We added a new relation interface for tracing 2. The tempo operator does not yet support rotation of certs, so we will need to address support for rotating certs when tempo adds this functionality 3. The CertHandler (by observability_libs) uses v3 of tls_certificates charm lib. Our charm uses v2. We should upgrade to our charm to use v3 in a follow up PR

  1. We need to use edge version of the COS-lite bundle until their next stable release (as the COS team has some critical unreleased changes)
  2. We are sending traces to tempo-k8s using HTTP as there are a few issues that need be addressed before we can send traces using HTTPS

image