rgrandl
commented
1 year ago I think we should think a little bit more on how to do this.
It makes sense to disable both prometheus and jaeger for testing (if part of a continuous integration thing). In that case, we should have a single flag that disable observability.
Otherwise, we need ways for the user to easily plug their custom prometheus and jaeger service, which should probably be done through configuration. Otherwise, if they disable jaeger/prometheus through flags, there is no easy way (or no option at all) for them to provide an alternative.
Prometheus:
- the Prometheus service will pull the metrics from the /metrics path from all pods that satisfy a regex (more specific the app name);
- if someone wants to use their own Prometheus, then they can probably modify the .yaml file and look at the definition of the config map to understand how to set their own Prometheus to scrape the app pods.
- if we disable generating Prometheus yaml file at all, the user will not be able to see that, and will have no idea how to change it.
Jaeger:
- this is push based; the babysitters will export traces to the Jaeger service
- even if we don't generate yaml info for Jaeger, there is no way for the babysitter right now to understand where to send the traces; maybe we will need a config option with the name of the Jaeger endpoint; maybe we can do something similar for Prometheus then too
mwhittaker
18