canonical / tempo-k8s-operator

This charmed operator automates the operational procedures of running Grafana Tempo, an open-source tracing backend.
https://charmhub.io/tempo-k8s
Apache License 2.0
5 stars 3 forks source link

Add traefik_route to serve tracing endpoints through ingress if it exists #94

Closed mmkay closed 2 months ago

mmkay commented 2 months ago

Issue

Tempo receiver endpoints were not accessible over traefik as they are bound to different ports. Fixes #82

Solution

Use traefik_route with changes introduced in traefik-k8s-operator#325 to define ports traefik needs to open. Also fix creating URLs for tracing consumers that use external_url - scheme and port were missing from the created url.

Context

This change is needed for cross-model traces to work. They cannot access internal endpoints of a different k8s cluster . No traces were also coming from machine models.

Testing Instructions

Run the following bundle and verify endpoints (example, port 3200, /metrics, port 4318 (expected prometheus metrics), /v1/traces (expected HTTP code 405)) are accessible on traefik's external IP.

bundle: kubernetes
applications:
  tempo:
    charm: local:tempo-k8s-1
    scale: 1
    constraints: arch=amd64
    storage:
      data: kubernetes,1,1024M
  traefik-k8s:
    charm: traefik-k8s
    channel: latest/edge
    revision: 184
    base: ubuntu@20.04/stable
    resources:
      traefik-image: 159
    scale: 1
    constraints: arch=amd64
    storage:
      configurations: kubernetes,1,1024M
relations:
- - tempo:tracing
  - traefik-k8s:tracing
- - traefik-k8s:traefik-route
  - tempo:ingress

Release Notes

mmkay commented 2 months ago

@PietroPasotti: regarding this part:

Secondly, wondering if we should be renaming the 'ingress' endpoint to traefik-route to follow the convention of having endpoints and interfaces similarly named. Also, if some day we choose to provide this kind of multi-route, hybrid ingress via traefik and upgrade the ingress relation accordingly, we can swap them out without the ingress name being taken by something that isn't quite ingress.

I don't mind changing the interface name but I was basing the current approach on that Grafana also uses ingress in the same manner:

https://github.com/canonical/grafana-k8s-operator/blob/main/charmcraft.yaml#L54

As it was the only o11y charm that uses traefik_route that I know of, I chose the same pattern as it is fairly transparent to the charm users - they want to relate to Traefik and they're (mostly) used that the protocol they would use is named "ingress". I can however change that if you think having ingress free for future use is useful :)