DataDog / datadog-operator

Kubernetes Operator for Datadog Resources
Apache License 2.0
302 stars 104 forks source link

Adopting pre-existing SLOs #1248

Open philippeVV opened 4 months ago

philippeVV commented 4 months ago

The Datadog Operator should implement resource adoption for the SLO CRD.

Use case

We have existing SLOs in our Datadog account that we would like to import into the operator. Deleting and recreating these SLOs would erase their historical data and break all associated dashboards, monitors, and references by ID. In our scenario, we frequently need to adopt resources due to cluster A/B testing on major changes.

Why it doesn’t work

The operator currently doesn’t support importing SLOs. It will always try to create them when the CR is first created.

Expected behavior

When an SLO CR is created, we expect the operator to import the SLO from Datadog if an SLO with the same name already exists. If no such SLO exists, the operator should create a new one.

philippeVV commented 4 months ago

We are willing to contribute if there's interest in the feature and we get support selecting the right design.

philippeVV commented 4 months ago

Crossplane uses annotation to import pre-existing resources.

We could also use the SLO's name or SLO's name and tags to make a search query to Datadog. The human operator would be responsible for managing SLO unicity.