[Tracking] Integration scenarios

mowies commented 8 months ago

Goal

Integrate well with other tools so that users can take the most advantage of the synergies of those tools together with Keptn.

Details

This epic should be used to do research on possible integration opportunities with the tools below, to connect with the maintainer teams to present what Keptn can do and to get feedback from them on integration options. This epic should also be used for any user guides and other documentation that comes out as a result of the research. Additionally, this epic should also include any needed implementation work to integrate better with other tools.

The tools that have the best fit are:

KEDA: scale workloads with KEDA using the Keptn metrics server
Flux: not sure how to integrate this
Flagger: use Keptn metrics for release analysis, have o11y into progressive delivery
Thanos: could be a quick win to have a metrics provider since it's basically prometheus
Cortex: could be a quick win to have a metrics provider since it's basically prometheus
Backstage: show keptnapps with the versions of workloads directly on backstage pages, maybe even which version of workloads is on which gitops stage
Kyverno: the only possible scenario is the chainsaw tool since Kyverno is already well-equipped with o11y. otherwise, maybe link traces between keptn and kyverno
Crossplane: o11y for infra releases
ArgoCD: tracing for whatever happens during deployments
Argo Rollouts: integrate Keptn analyses into Argo experiments
OpenSLO: introduce this to have a common language for analysis definitions
AWS Cloudwatch: metrics provider
Google Cloud Metrics: metrics provider
Azure Monitor: metrics provider
OpenFeature: change FFs as promotion step, run analyses after FF changes
KubeJanitor: can be used for easy clean up of old jobs/tasks of Keptn
Devaten
k8sGPT

Definition of Done

have research results for all above tools, that show, if and how an integration with Keptn makes sense
have documentation for all tools where an integration with Keptn makes sense
have special handling code where necessary for integrations to work

### Tasks
- [ ] #2899
- [ ] #3096 
- [ ] #2900
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2903
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2901
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2262
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/2963
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3002
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3025
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3026
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3027
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3050
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3051
- [ ] https://github.com/keptn/lifecycle-toolkit/issues/3052

sudiptob2 commented 8 months ago

Thanos: could be a quick win to have a metrics provider since it's basically prometheus

I tested the Keptn Prometheus provider with Thanos, and it looks like it's working. I was considering to create a Thanos provider, but it seems that the Prometheus provider can be used to query Thanos 🤔

I followed Keptn + HPA example and used Thanos server as the TargetServer for metrics instead of the Prometheus server. I might be mistaken as I don't have much experience with this.

Here is my KeptnMetric manifests I used

apiVersion: metrics.keptn.sh/v1alpha3
kind: KeptnMetricsProvider
metadata:
  name: thanos-provider
  namespace: podtato-kubectl
spec:
  type: prometheus
  targetServer: "http://thanos-query.thanos.svc.cluster.local:9090"

---
apiVersion: metrics.keptn.sh/v1alpha3
kind: KeptnMetric
metadata:
  name: cpu-throttling
  namespace: podtato-kubectl
spec:
  provider:
    name: thanos-provider
  query: 'sum(kube_pod_container_resource_limits{resource="cpu"})'
  fetchIntervalSeconds: 10
  range:
    interval: "5m"
    step: "1m"
    aggregation: "avg"

As you can see in the screenshot, we get metric output.

rakshitgondwal commented 8 months ago

@sudiptob2 That should be the expected behavior because Thanos implements the Prometheus HTTP API to query data via PromQL. I think we can have a new KeptnMetricsProvider type for Thanos in this case and reuse the functionality of the Prometheus provider.

sudiptob2 commented 8 months ago

@rakshitgondwal make sense. I was also thinking about that, like adding a case here for Thanos which just return PrometheusProvider. Perhaps, it might also be possible to solve without any code change, by simply mentioning to use prometheus type for thanos in the documentation.

mowies commented 8 months ago

thanks for looking into this! i think we can find a nice solution that prevents lots of code duplication, but let's see. for now, the epic is still work in progress. i'll write some sub-tickets for it soon and then we can refine and work them

heckelmann commented 8 months ago

I know that @thschue was integrating Keptn with K8sGPT, maybe we should put it on the List as well.

keptn / lifecycle-toolkit