[Bug]: Error: Provider produced inconsistent final plan - grafana_contact_point

[evertonspader-tomtom]

Terraform Version

v1.8.0

Terraform Grafana Provider Version

v2.18.0

Grafana Version

No response

Affected Resource(s)

• grafana_contact_point

Terraform Configuration Files

resource "grafana_contact_point" "team_pagerduty" {
  name = "PagerDuty critical ${var.environment}-tf"

  pagerduty {
    integration_key = data.azurerm_key_vault_secret.autostream_pagerduty_integration_key.value
    settings = {
      uploadImage = "true"
      autoResolve = "true"
    }
  }
}

Expected Behavior

No changes should be suggested or errors be seen after first successful apply and provisioning of resources

Actual Behavior

The resource is already provisioned but running a terraform apply yields in errors pointing to bugs in the provider. Running the apply command again succeeds with no changes.

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for grafana_contact_point.team_pagerduty to include new values learned so far during apply, provider "registry.terraform.io/grafana/grafana" produced an invalid new
│ value for .pagerduty: planned set element cty.ObjectVal(map[string]cty.Value{"class":cty.StringVal(""), "client":cty.StringVal(""), "client_url":cty.StringVal(""), "component":cty.StringVal(""),
│ "details":cty.NullVal(cty.Map(cty.String)), "disable_resolve_message":cty.False, "group":cty.StringVal(""), "integration_key":cty.UnknownVal(cty.String),
│ "settings":cty.MapVal(map[string]cty.Value{"autoResolve":cty.StringVal("true"), "uploadImage":cty.StringVal("true")}), "severity":cty.StringVal(""), "source":cty.StringVal(""),
│ "summary":cty.StringVal(""), "uid":cty.UnknownVal(cty.String)}) does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.
╵

Steps to Reproduce

1. terraform apply - the errors will be seen
2. terraform apply - no changes will be perceived

Important Factoids

No response

References

No response

[julienduchesne]

Is this on Grafana Cloud and is this still happening? I've tried to reproduce it on Grafana 10 and 11 (Cloud) and I'm unable to do so

[julienduchesne]

Closing because I'm unable to reproduce, and this may be resolved with the new major version. I'd recommend you try that one. Feel free to re-open if this is still happening!

[lutzwillek-tomtom]

Yes, Grafana Cloud, at the moment Grafana v11.3.0-75420 (d1dc595692). The issue still appears after bumping the grafana/grafana provider from 2.18.0 to 2.19.4

We will try out the new major version and report back.

[JelleSmet-TomTom]

We are now using version 3.7.0

Here's an attempt to give more information ...

There are 2 styles of errors we get, which presumable have a shared cause:

│ Error: Provider produced inconsistent final plan
│ 
│ When expanding the plan for module.sre_service_dev["Scalyr"].grafana_contact_point.slack[0] to include new values learned so far during apply, provider
│ "registry.terraform.io/grafana/grafana" produced an invalid new value for .slack: planned set element cty.ObjectVal(map[string]cty.Value{"disable_resolve_message":cty.False,
│ "endpoint_url":cty.StringVal(""), "icon_emoji":cty.StringVal(""), "icon_url":cty.StringVal(""), "mention_channel":cty.StringVal(""), "mention_groups":cty.StringVal(""),
│ "mention_users":cty.StringVal(""), "recipient":cty.StringVal(""), "settings":cty.NullVal(cty.Map(cty.String)), "text":cty.StringVal(""), "title":cty.StringVal(""),
│ "token":cty.StringVal(""), "uid":cty.UnknownVal(cty.String), "url":cty.UnknownVal(cty.String), "username":cty.StringVal("")}) does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

and

│ Error: [PUT /v1/provisioning/contact-points/{UID}][400] putContactpointBadRequest {"message":"invalid object specification: failed to validate integration \"AutoStream email notification prod-tf\" (UID ET25S4L7kz) of type \"email\": could not find addresses in settings"}
│ 
│   with grafana_contact_point.autostream_email,
│   on autostream_monitoring.tf line 33, in resource "grafana_contact_point" "autostream_email":
│   33: resource "grafana_contact_point" "autostream_email" {
│ 
╵

The Terraform configuration surrounding the 2nd error is the following: (emails obfuscated)

resource "grafana_contact_point" "autostream_email" {
  name = "AutoStream email notification ${var.environment}-tf"
  email {
    addresses = ["xxxxxxxxxxxx@xxxxxxxx.xxx"]
  }
}

The provider is capable of successfully destroying and adding the resource again:

terraform destroy -target grafana_contact_point.autostream_email
terraform aplly -target grafana_contact_point.autostream_email

All looks good at this point.

If we apply a 2nd time the same resource we get following error:

╷
│ Error: [PUT /v1/provisioning/contact-points/{UID}][400] putContactpointBadRequest {"message":"invalid object specification: failed to validate integration \"AutoStream email notification prod-tf\" (UID ET25S4L7kz) of type \"email\": could not find addresses in settings"}
│ 
│   with grafana_contact_point.autostream_email,
│   on autostream_monitoring.tf line 33, in resource "grafana_contact_point" "autostream_email":
│   33: resource "grafana_contact_point" "autostream_email" {
│ 
╵

If we look at the state file I can see following configuration: (both addresses and private values have been obfuscated)

{
      "mode": "managed",
      "type": "grafana_contact_point",
      "name": "autostream_email",
      "provider": "provider[\"registry.terraform.io/grafana/grafana\"]",
      "instances": [
        {
          "schema_version": 0,
          "attributes": {
            "alertmanager": [],
            "dingding": [],
            "disable_provenance": false,
            "discord": [],
            "email": [
              {
                "addresses": [
                  "xxxxxxxxxxxxx@xxxxxxxxx.xxx"
                ],
                "disable_resolve_message": false,
                "message": "",
                "settings": {},
                "single_email": false,
                "subject": "",
                "uid": "adxomefqqca2oa"
              }
            ],
            "googlechat": [],
            "id": "0:AutoStream email notification prod-tf",
            "kafka": [],
            "line": [],
            "name": "AutoStream email notification prod-tf",
            "oncall": [],
            "opsgenie": [],
            "org_id": "0",
            "pagerduty": [],
            "pushover": [],
            "sensugo": [],
            "slack": [],
            "sns": [],
            "teams": [],
            "telegram": [],
            "threema": [],
            "victorops": [],
            "webex": [],
            "webhook": [],
            "wecom": []
          },
          "sensitive_attributes": [],
          "private": "xxxxx"
        }
      ]
    }

This is all I am able to collect of the current state.