azurerm_mssql_database_vulnerability_assessment_rule_baseline - Root resource was present, but now absent. - Provider produced inconsistent result after apply

[artyq]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform (and AzureRM Provider) Version

terraform v0.14.10 azurerm v2.72.0

Affected Resource(s)

• azurerm_mssql_database_vulnerability_assessment_rule_baseline

Terraform Configuration Files

  server_vulnerability_assessment_id = module.mssql.sqlserver[0].assessment_id
  database_name                      = module.mssql.db[0].sql_db.name
  rule_id                            = "VA1258"
  baseline_name                      = "default"
  baseline_result {
    result = [
      var.dbowner
    ]
  }
  depends_on = [
    module.mssql,
    null_resource.configureazsql
  ]
}

Debug Output

I am unable to provide a link with the debug output as this is from an internal gitlab.

Error: Provider produced inconsistent result after apply
When applying changes to
module.mssql_example.azurerm_mssql_database_vulnerability_assessment_rule_baseline.app_va1258_baseline,
provider "registry.terraform.io/hashicorp/azurerm" produced an unexpected new
value: Root resource was present, but now absent.
This is a bug in the provider, which should be reported in the provider's own
issue tracker.

Expected Behaviour

Acknowledge that the resource is there and exists.

Actual Behaviour

Resource is created, but Terraform can no longer see it.

Steps to Reproduce

1. terraform apply

Important Factoids

There is several other reports with the exact same output but for different resources. It doesn't matter if I again run plan and apply, the same error is presented.

References

• 0000

[artyq]

Issue https://github.com/hashicorp/terraform-provider-azurerm/issues/11488

Appears to be similar to my issue.

[aristosvo]

I'll try to reproduce it and diagnose the proble, also based on duplicate issue #11488. I'll stick with this one as it is more recent.

Validation setup will be something along the lines of:

variable "prefix" {
  default = "vulnbaseline"
}
resource "azurerm_resource_group" "test" {
  name     = "test_vuln_assesm_baseline"
  location = "westeurope"
}

resource "azurerm_storage_account" "test" {
  name                  = "sa${var.prefix}"
  resource_group_name           = azurerm_resource_group.test.name
}

resource "azurerm_storage_account" "test" {
  name                     = "sa${var.prefix}"
  resource_group_name      = azurerm_resource_group.test.name
  location                 = azurerm_resource_group.test.location
  account_tier             = "Standard"
  account_replication_type = "GRS"
}
resource "azurerm_storage_container" "test" {
  name                  = "sc${var.prefix}"
  storage_account_name  = azurerm_storage_account.test.name
  container_access_type = "private"
}

resource "azurerm_mssql_server" "main" {
  depends_on                    = [azurerm_storage_container.main]
  name                          = "sqlsrvr${var.prefix}"
  resource_group_name           = azurerm_resource_group.test.name
  location                      = azurerm_resource_group.test.location
  version                       = "12.0"
  administrator_login           = "mradministratortest"
  administrator_login_password  = "!@#%%$#asdfsd"
  connection_policy             = var.connection_policy
  minimum_tls_version           = "1.2"
  public_network_access_enabled = true

  azuread_administrator {
    login_username = var.sql_server_azure_ad_admin_name
    object_id      = var.sql_server_azure_ad_object_id
    tenant_id      = var.sql_server_azure_ad_tenant_id
  }
  identity {
    type = "SystemAssigned"
  }
}
resource "azurerm_role_assignment" "main" {
  depends_on           = [azurerm_mssql_server.main]
  scope                = azurerm_storage_account.test.id
  role_definition_name = "Storage Blob Data Contributor"
  principal_id         = azurerm_mssql_server.main.identity.0.principal_id
}
resource "azurerm_mssql_server_extended_auditing_policy" "main" {
  depends_on                              = [azurerm_role_assignment.main]

  server_id                               = azurerm_mssql_server.main.id
  storage_endpoint                        = "https://${var.storage_name}.blob.core.windows.net/"
  retention_in_days                       = var.extended_auditing_policy_retention_in_days
  log_monitoring_enabled                  = true
}

resource "azurerm_mssql_server_security_alert_policy" "main" {
  resource_group_name = var.resource_group_name
  server_name         = azurerm_mssql_server.main.name
  state               = "Enabled"
}
resource "azurerm_mssql_firewall_rule" "main" {
  name             = "FirewallRuleForAzureServices"
  server_id        = azurerm_mssql_server.main.id
  start_ip_address = "0.0.0.0"
  end_ip_address   = "0.0.0.0"
}
resource "azurerm_mssql_server_vulnerability_assessment" "main" {
  depends_on                      = [azurerm_mssql_server_extended_auditing_policy.main]
  server_security_alert_policy_id = azurerm_mssql_server_security_alert_policy.main.id
  storage_container_path          = "https://${var.storage_name}.blob.core.windows.net/${var.vulnerability_assessment_scans_storage_container_name}/"
  storage_account_access_key      = azurerm_mssql_server_extended_auditing_policy.main.storage_account_access_key
  recurring_scans {
    enabled                   = true
    email_subscription_admins = [admin@test.nl]
    emails                    = [test@test.nl]
  }
}
resource "azurerm_mssql_database_vulnerability_assessment_rule_baseline" "va2130" {
  depends_on                         = [azurerm_mssql_server_vulnerability_assessment.main]
  server_vulnerability_assessment_id = azurerm_mssql_server_vulnerability_assessment.main.id
  database_name                      = "master"
  rule_id                            = "VA2130"
  baseline_name                      = "master"
  baseline_result {
    result = [
      var.sql_server_admin_name
    ]
  }
  baseline_result {
    result = [
      var.sql_server_azure_ad_admin_name
    ]
  }
}
resource "azurerm_mssql_database_vulnerability_assessment_rule_baseline" "va2065" {
  depends_on                         = [azurerm_mssql_database_vulnerability_assessment_rule_baseline.va2130]
  server_vulnerability_assessment_id = azurerm_mssql_server_vulnerability_assessment.main.id
  database_name                      = "master"
  rule_id                            = "VA2065"
  baseline_name                      = "master"
  baseline_result {
    result = [
      "FirewallRuleForAzureServices",
      "0.0.0.0",
      "0.0.0.0"
    ]
  }
}

[artyq]

Is there any progress on this?

It usually takes 3 + attempts for Terraform to notice it and not error during apply.

I confirmed that the issue is still present in 2.93.1

[catt231]

I'm also experiencing on >=2.80.0 seems to be VA2065 it has an issue with

resource "azurerm_mssql_server_security_alert_policy" "alert-policy" {
  resource_group_name = azurerm_mssql_server.server.resource_group_name
  server_name         = azurerm_mssql_server.server.name
  state               = var.mssql-alert-policy.enabled
  retention_days      = var.mssql-alert-policy.retain-for
  email_addresses     = var.mssql-alert-policy.notify
}

resource "azurerm_mssql_server_vulnerability_assessment" "server-vulnerability-assessment" {
  server_security_alert_policy_id = azurerm_mssql_server_security_alert_policy.alert-policy.id
  storage_container_path          = "${data.azurerm_storage_account.audit-storage-account.primary_blob_endpoint}${data.azurerm_storage_container.audit-storage-container.name}/"

  recurring_scans {
    enabled                   = true
    email_subscription_admins = false
    emails                    = var.mssql-alert-policy.notify
  }
}

resource "azurerm_mssql_database_vulnerability_assessment_rule_baseline" "firewall-rules-baseline" {
  server_vulnerability_assessment_id = azurerm_mssql_server_vulnerability_assessment.server-vulnerability-assessment.id
  database_name = "master"
  rule_id       = "VA2065"
  baseline_name = "master"

  dynamic "baseline_result" {
    for_each = local.firewall-rules

    content {
      result = [
        baseline_result.key,
        baseline_result.value["start-ip-address"],
        baseline_result.value["end-ip-address"]
      ]
    }
  }
}

[alxndr13]

@catt231 can confirm, we expierence this issue also with version 2.95.0 of the provider. For us it also fails with va2109 and others.

[jamesw4]

Also hitting issues with this, in that sometimes when creating baselines I receive the error "provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced │ an unexpected new value: Root resource was present, but now absent". Seen on multiple projects we have, some on 2.59 of the provider, others on 2.78.

In my experiences for the resources that this occurs for they are not created. If I re-run it tries to create again and normally succeeds.

While this particular error seems refined to initial creation of a baseline, updates also seem to be problematic. E.g. lets say I add a user to the baseline for VA1258 - sometimes TF reports it is done, but if I check manually I see it has not been done. If I re-run pipeline, it warns me that a change was made outside of TF (removing the user from the baseline). It seems it failed to update the first time around, but TF thinks it updated OK.

[jamesw4]

I'm starting to wonder if the root of this really is the provider and not Azure. I've enabled trace logging and reproduced the issue. What I'm seeing is.

• Put request to Azure API for the baseline resource (including my correct payload).
• HTTP 200 response to the put.
• A get request for the same baseline resource.
• A HTTP 404 response to the get.

Cross checking the correlation id from the trace log to the activity log in azure I see that the action for this baseline is reported as "started" and then also a separate event for "succeeded".

[jamesw4]

Going to raise ticket with MS. In my scenario we are setting 6 baselines, randomly but quite frequently upon create one or two would get the error mention in the first post.

As a test I added dependencies in TF just to test if them being done in sequence vs parallel had any impact. It seems in my case it does, have not been able to get it to fail when they are created in sequence.

Then went and tested with the az PowerShell module. Same findings, doing the 6 of them in parallel resulted sometimes in one or two of them not being created despite PowerShell reporting all were created ok.

[krohm]

@jamesw4 , any update on this?

[jamesw4]

My issues didn’t seem specific to TF. In the end I worked around this using dependencies so they created in sequence rather than in parallel. I think MS has maybe sorted since as when I tried to reproduce the issue weeks later again with PowerShell I couldn’t reproduce the issue anymore. Thanks.

[jsmith-speedeon]

I'm hitting this in April of 2023. It seems that applying multiple baselines to the same data warehouse sku (in my case) causes the above error:

Error: Provider produced inconsistent result after apply When applying changes to azurerm_mssql_database_vulnerability_assessment_rule_baseline.sqlserver-va1248-baseline, provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced an unexpected new value: Root resource was present, but now absent.

If I apply multiple times, it eventually applies all the baselines. But there is definitely some kind of race condition it would seem.

[jamesw4]

Since changed jobs, hit by this issue again. Used same work around of chaining dependencies between each baseline resource so they are not created in parallel.