`powerplatform_data_loss_prevention_policy` produces errors, while successfully provisioning DLP policies

[A-Tanner]

Describe the bug

powerplatform_data_loss_prevention_policy can fail due to unexpected state. In the sample code, this happens both if a new DLP policy is being provisioned, or an existing DLP policy is being modified. New policies are successfully created, with all specified connectors properly categorized.

Sample Terraform Code

Slightly modified version of the example for DLP policy resource

data "powerplatform_connectors" "dad_joke_sms_policy" {}

locals {

  business_connectors = toset([
    {
      action_rules                 = []
      default_action_rule_behavior = ""
      endpoint_rules               = []
      id                           = "/providers/Microsoft.PowerApps/apis/shared_approvals"
    },
    {
      action_rules                 = []
      default_action_rule_behavior = "Allow"
      endpoint_rules               = []
      id                           = "/providers/Microsoft.PowerApps/apis/shared_d7sms"
    },
    {
      id                           = "/providers/Microsoft.PowerApps/apis/shared_dadjokes"
      default_action_rule_behavior = "Allow"
      action_rules                 = []
      endpoint_rules               = []
    },
    {
      id                           = "/providers/Microsoft.PowerApps/apis/shared_dadjokesioip"
      default_action_rule_behavior = "Block"
      action_rules                 = []
      endpoint_rules               = []
    }
  ])

  non_business_connectors = toset([
    {
      id                           = "/providers/Microsoft.PowerApps/apis/shared_daffyip"
      default_action_rule_behavior = "Allow"
      action_rules                 = []
      endpoint_rules               = []
    }
  ])

  blocked_connectors = toset([])
}

resource "powerplatform_data_loss_prevention_policy" "my_policy" {
  display_name                      = "dad_joke_sms_policy"
  default_connectors_classification = "Blocked"
  environment_type                  = "OnlyEnvironments"
  environments                      = var.environments

  business_connectors     = local.business_connectors
  non_business_connectors = local.non_business_connectors
  blocked_connectors      = local.blocked_connectors

  custom_connectors_patterns = toset([
    {
      order            = 1
      host_url_pattern = "https://*.contoso.com"
      data_group       = "Blocked"
    },
    {
      order            = 2
      host_url_pattern = "*"
      data_group       = "Ignore"
    }
  ])
}

Expected behavior

As mentioned, the connectors are successfully created despite the error messages (see following section). I would expect that, if there are errors generated that the action fails to create these resources, or if the creation is successful then warning messages are supplied instead, especially if these warning messages can provide clarity on what the root cause of the issue is.

Additional context

Here are the errors caused by a sample run of the above terraform. This was ran as part of a main that also spin up a few environments, environment groups, etc. in a long standing power platform tenant.

These errors were generated from a run with no existing tfstate or DLP policies in the tenant.

It appears that after apply the DLP automatically gets a few (23-24) connectors added to the policy, but additionally some connectors are described as "not correlating with any element in actual." Unsure what the "Actual" is in question, as these connectors and policies are fully present in the DLP policy after the application step errors out.

╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.data_loss_prevention_policies.powerplatform_data_loss_prevention_policy.my_policy,
│ provider "provider[\"registry.terraform.io/microsoft/power-platform\"]"
│ produced an unexpected new value: .non_business_connectors: planned set
│ element
│ cty.ObjectVal(map[string]cty.Value{"action_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"action_id":cty.String,
│ "behavior":cty.String})),
│ "default_action_rule_behavior":cty.StringVal("Allow"),
│ "endpoint_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"behavior":cty.String,
│ "endpoint":cty.String, "order":cty.Number})),
│ "id":cty.StringVal("/providers/Microsoft.PowerApps/apis/shared_daffyip")})
│ does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.data_loss_prevention_policies.powerplatform_data_loss_prevention_policy.my_policy,
│ provider "provider[\"registry.terraform.io/microsoft/power-platform\"]"
│ produced an unexpected new value: .non_business_connectors: length changed
│ from 1 to 24.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.data_loss_prevention_policies.powerplatform_data_loss_prevention_policy.my_policy,
│ provider "provider[\"registry.terraform.io/microsoft/power-platform\"]"
│ produced an unexpected new value: .business_connectors: planned set element
│ cty.ObjectVal(map[string]cty.Value{"action_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"action_id":cty.String,
│ "behavior":cty.String})),
│ "default_action_rule_behavior":cty.StringVal("Allow"),
│ "endpoint_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"behavior":cty.String,
│ "endpoint":cty.String, "order":cty.Number})),
│ "id":cty.StringVal("/providers/Microsoft.PowerApps/apis/shared_d7sms")})
│ does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.data_loss_prevention_policies.powerplatform_data_loss_prevention_policy.my_policy,
│ provider "provider[\"registry.terraform.io/microsoft/power-platform\"]"
│ produced an unexpected new value: .business_connectors: planned set element
│ cty.ObjectVal(map[string]cty.Value{"action_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"action_id":cty.String,
│ "behavior":cty.String})),
│ "default_action_rule_behavior":cty.StringVal("Allow"),
│ "endpoint_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"behavior":cty.String,
│ "endpoint":cty.String, "order":cty.Number})),
│ "id":cty.StringVal("/providers/Microsoft.PowerApps/apis/shared_dadjokes")})
│ does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵
╷
│ Error: Provider produced inconsistent result after apply
│ 
│ When applying changes to
│ module.data_loss_prevention_policies.powerplatform_data_loss_prevention_policy.my_policy,
│ provider "provider[\"registry.terraform.io/microsoft/power-platform\"]"
│ produced an unexpected new value: .business_connectors: planned set element
│ cty.ObjectVal(map[string]cty.Value{"action_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"action_id":cty.String,
│ "behavior":cty.String})),
│ "default_action_rule_behavior":cty.StringVal("Block"),
│ "endpoint_rules":cty.ListValEmpty(cty.Object(map[string]cty.Type{"behavior":cty.String,
│ "endpoint":cty.String, "order":cty.Number})),
│ "id":cty.StringVal("/providers/Microsoft.PowerApps/apis/shared_dadjokesioip")})
│ does not correlate with any element in actual.
│ 
│ This is a bug in the provider, which should be reported in the provider's
│ own issue tracker.
╵

Contribution

Do you plan to raise a PR to address this issue? No, not at this time

[mawasile]

hi @A-Tanner

here is a working example of what you are trying to accomplish:

data "powerplatform_connectors" "all_connectors" {}

locals {

  business_connectors = toset([
    {
      action_rules                 = []
      default_action_rule_behavior = ""
      endpoint_rules               = []
      id                           = "/providers/Microsoft.PowerApps/apis/shared_approvals"
    },
    {
      action_rules                 = []
      default_action_rule_behavior = ""
      endpoint_rules               = []
      id                           = "/providers/Microsoft.PowerApps/apis/shared_d7sms"
    },
    {
      id                           = "/providers/Microsoft.PowerApps/apis/shared_dadjokes"
      default_action_rule_behavior = ""
      action_rules                 = []
      endpoint_rules               = []
    },
    {
      id                           = "/providers/Microsoft.PowerApps/apis/shared_dadjokesioip"
      default_action_rule_behavior = ""
      action_rules                 = []
      endpoint_rules               = []
    }
  ])

  non_business_connectors = toset([for conn
    in data.powerplatform_connectors.all_connectors.connectors :
    {
      id                           = conn.id
      default_action_rule_behavior = ""
      action_rules                 = [],
      endpoint_rules               = []
    }
    if conn.unblockable == true && !contains([for bus_conn in local.business_connectors : bus_conn.id], conn.id)
  ])

  blocked_connectors = toset([for conn
    in data.powerplatform_connectors.all_connectors.connectors :
    {
      id                           = conn.id
      default_action_rule_behavior = ""
      action_rules                 = [],
      endpoint_rules               = []
    }
  if conn.unblockable == false && !contains([for bus_conn in local.business_connectors : bus_conn.id], conn.id)])
}

resource "powerplatform_data_loss_prevention_policy" "my_policy" {
  display_name                      = "Block All Policy"
  default_connectors_classification = "Blocked"
  environment_type               = "OnlyEnvironments"
  environments                      = var.environments

  business_connectors     = local.business_connectors
  non_business_connectors = local.non_business_connectors
  blocked_connectors      = local.blocked_connectors

  custom_connectors_patterns = toset([
    {
      order            = 1
      host_url_pattern = "https://*.contoso.com"
      data_group       = "Blocked"
    },
    {
      order            = 2
      host_url_pattern = "*"
      data_group       = "Ignore"
    }
  ])
}

Two things to mention

1. You always have to put all the connectors into your policy like in my examples using powerplatform_connectors datasource, otherwise terraform reading the your policy from the backend, will return exception like this one: .non_business_connectors: length changed │ from 1 to 24.
2. if you don't have any action_rules then the default_action_rule_behavior should be empty. I will look at it as this shold be explicitly throw a validation exception during terraform plan