Introduce ability to track settings that are currently not maintained by Terraform

[xoronet]

It would be very beneficial to have data sources (for every existing resource type) that provide the current number of Dynatrace objects in an environment. This would allow to easily detect if there are objects in the Dynatrace environment that are not managed by Terraform yet. The difference to the current data sources is that they do not return a single specific object but all objects or at least a count value.

So far I use the generic http provider and trigger a call to the Dynatrace REST API. From the response body, I count the entries in the values list. See the example below:

data "http" "request_attributes" {
  url             = "${var.dynatrace_env_url}/api/config/v1/service/requestAttributes"
  request_headers = {
      Authorization : "Api-Token ${var.dynatrace_api_token}"
  }
}

output "count" {
  value = length(jsondecode(data.http.request_attributes.response_body)["values"])
}

With that, I can check whether new request attributes have been added to Dynatrace environment. The downside is that I have to inject url and token as variables.

It would be much easier to have a dedicated data source dynatrace_request_attributes:

data "dynatrace_request_attributes" "request_attributes" {
}

output "count" {
  value = data.dynatrace_request_attributes.request_attributes.count
}

An alternative would be a single generic data source that allows to call any REST endpoint and provides the response in a generic "response_body" attribute. This would already simplify the setup because I do not need to use the http provider and inject a token manually as variable.

data "dynatrace_api_call" "request_attributes" {
    uri = "/api/config/v1/service/requestAttributes"
}

output "count" {
  value = length(jsondecode(data.dynatrace_api_call.request_attributes.response_body)["values"])
}

[Dynatrace-Reinhard-Pilz]

Hello @xoronet ,

Is your main objective here to ensure that nobody is creating settings behind your back (i.e. outside of terraform)? If so, I could give you an outlook on something we're currently working on. The working title of that resource is called dynatrace_golden_state.

resource "dynatrace_golden_state" "golden_state" {
  request_attributes = [
    dynatrace_request_attribute.att1.id,
    dynatrace_request_attribute.att2.id,
    ...
  ],
  management_zones = [
     ...
  ],
  alerting_profiles = [
     ...
  ],
  ...
}

What it does is, that it deletes all the settings that aren't explicitly specified in there.

The data source you're suggesting (or a variant of it) wouldn't be that difficult to implement. But in the end it would just produce a number - in other words, it wouldn't solve an actual problem.

Is the resource example mentioned above something that's related to what you're trying to achieve?

[xoronet]

Hi @Dynatrace-Reinhard-Pilz,

yes indeed, the main objective for me is to recognize if settings and other objects in the Dynatrace environment are created outside of Terraform. For settings and objects that are already covered by Terraform, I can detect configuration drift by simply executing terraform plan again. Settings and objects that are entirely unknown to the Terraform state cannot be detected that way. The outlook is pretty interesting! Do you have a rough timeline to share more details?

Having a data source publishing "just" a number how many objects exist would be a simple and cheap indicator that something is not covered in Terraform and I can continue investigation. But if your "golden_state" solves the problem in a more professional way, I would definitely be interested.

Thanks

[Dynatrace-Reinhard-Pilz]

Current plan for a first version is the release on August 2nd. That version may not yet cover all possible resources the provider supports as a whole, but it should be enough for users to provide early responses about how to improve.

I'm going to rename this ticket to be a bit more generic. We will use it to collect feedback for the golden-state resource. But we'll keep the door still open for the data source you've suggested

[xoronet]

@Dynatrace-Reinhard-Pilz any updates on the timeline you can share?

[Dynatrace-Reinhard-Pilz]

Hi @xoronet

Apologies for the delay here. Coincidentally I was tempted to include a first version into todays release - and decided against it, because I'll be out of office until the end of August. We're talking about a pretty delicate functionality and I don't want my colleague to pay the price in case users run into issues.

The version I'm working locally with covers about 1/3 of the resources. Reason is that we need to onboard every resource here one by one and re-check whether there don't exist any environment wide default settings we would get rid of unintentionally.

What I can offer is to provide a custom build for you, in case you're looking for a solution very urgent.

[xoronet]

Hi @Dynatrace-Reinhard-Pilz, no worries and thanks for the update! A custom build is not needed yet.