Open artyq opened 3 years ago
Issue https://github.com/hashicorp/terraform-provider-azurerm/issues/11488
Appears to be similar to my issue.
I'll try to reproduce it and diagnose the proble, also based on duplicate issue #11488. I'll stick with this one as it is more recent.
Validation setup will be something along the lines of:
variable "prefix" {
default = "vulnbaseline"
}
resource "azurerm_resource_group" "test" {
name = "test_vuln_assesm_baseline"
location = "westeurope"
}
resource "azurerm_storage_account" "test" {
name = "sa${var.prefix}"
resource_group_name = azurerm_resource_group.test.name
}
resource "azurerm_storage_account" "test" {
name = "sa${var.prefix}"
resource_group_name = azurerm_resource_group.test.name
location = azurerm_resource_group.test.location
account_tier = "Standard"
account_replication_type = "GRS"
}
resource "azurerm_storage_container" "test" {
name = "sc${var.prefix}"
storage_account_name = azurerm_storage_account.test.name
container_access_type = "private"
}
resource "azurerm_mssql_server" "main" {
depends_on = [azurerm_storage_container.main]
name = "sqlsrvr${var.prefix}"
resource_group_name = azurerm_resource_group.test.name
location = azurerm_resource_group.test.location
version = "12.0"
administrator_login = "mradministratortest"
administrator_login_password = "!@#%%$#asdfsd"
connection_policy = var.connection_policy
minimum_tls_version = "1.2"
public_network_access_enabled = true
azuread_administrator {
login_username = var.sql_server_azure_ad_admin_name
object_id = var.sql_server_azure_ad_object_id
tenant_id = var.sql_server_azure_ad_tenant_id
}
identity {
type = "SystemAssigned"
}
}
resource "azurerm_role_assignment" "main" {
depends_on = [azurerm_mssql_server.main]
scope = azurerm_storage_account.test.id
role_definition_name = "Storage Blob Data Contributor"
principal_id = azurerm_mssql_server.main.identity.0.principal_id
}
resource "azurerm_mssql_server_extended_auditing_policy" "main" {
depends_on = [azurerm_role_assignment.main]
server_id = azurerm_mssql_server.main.id
storage_endpoint = "https://${var.storage_name}.blob.core.windows.net/"
retention_in_days = var.extended_auditing_policy_retention_in_days
log_monitoring_enabled = true
}
resource "azurerm_mssql_server_security_alert_policy" "main" {
resource_group_name = var.resource_group_name
server_name = azurerm_mssql_server.main.name
state = "Enabled"
}
resource "azurerm_mssql_firewall_rule" "main" {
name = "FirewallRuleForAzureServices"
server_id = azurerm_mssql_server.main.id
start_ip_address = "0.0.0.0"
end_ip_address = "0.0.0.0"
}
resource "azurerm_mssql_server_vulnerability_assessment" "main" {
depends_on = [azurerm_mssql_server_extended_auditing_policy.main]
server_security_alert_policy_id = azurerm_mssql_server_security_alert_policy.main.id
storage_container_path = "https://${var.storage_name}.blob.core.windows.net/${var.vulnerability_assessment_scans_storage_container_name}/"
storage_account_access_key = azurerm_mssql_server_extended_auditing_policy.main.storage_account_access_key
recurring_scans {
enabled = true
email_subscription_admins = [admin@test.nl]
emails = [test@test.nl]
}
}
resource "azurerm_mssql_database_vulnerability_assessment_rule_baseline" "va2130" {
depends_on = [azurerm_mssql_server_vulnerability_assessment.main]
server_vulnerability_assessment_id = azurerm_mssql_server_vulnerability_assessment.main.id
database_name = "master"
rule_id = "VA2130"
baseline_name = "master"
baseline_result {
result = [
var.sql_server_admin_name
]
}
baseline_result {
result = [
var.sql_server_azure_ad_admin_name
]
}
}
resource "azurerm_mssql_database_vulnerability_assessment_rule_baseline" "va2065" {
depends_on = [azurerm_mssql_database_vulnerability_assessment_rule_baseline.va2130]
server_vulnerability_assessment_id = azurerm_mssql_server_vulnerability_assessment.main.id
database_name = "master"
rule_id = "VA2065"
baseline_name = "master"
baseline_result {
result = [
"FirewallRuleForAzureServices",
"0.0.0.0",
"0.0.0.0"
]
}
}
Is there any progress on this?
It usually takes 3 + attempts for Terraform to notice it and not error during apply.
I confirmed that the issue is still present in 2.93.1
I'm also experiencing on >=2.80.0 seems to be VA2065 it has an issue with
resource "azurerm_mssql_server_security_alert_policy" "alert-policy" {
resource_group_name = azurerm_mssql_server.server.resource_group_name
server_name = azurerm_mssql_server.server.name
state = var.mssql-alert-policy.enabled
retention_days = var.mssql-alert-policy.retain-for
email_addresses = var.mssql-alert-policy.notify
}
resource "azurerm_mssql_server_vulnerability_assessment" "server-vulnerability-assessment" {
server_security_alert_policy_id = azurerm_mssql_server_security_alert_policy.alert-policy.id
storage_container_path = "${data.azurerm_storage_account.audit-storage-account.primary_blob_endpoint}${data.azurerm_storage_container.audit-storage-container.name}/"
recurring_scans {
enabled = true
email_subscription_admins = false
emails = var.mssql-alert-policy.notify
}
}
resource "azurerm_mssql_database_vulnerability_assessment_rule_baseline" "firewall-rules-baseline" {
server_vulnerability_assessment_id = azurerm_mssql_server_vulnerability_assessment.server-vulnerability-assessment.id
database_name = "master"
rule_id = "VA2065"
baseline_name = "master"
dynamic "baseline_result" {
for_each = local.firewall-rules
content {
result = [
baseline_result.key,
baseline_result.value["start-ip-address"],
baseline_result.value["end-ip-address"]
]
}
}
}
@catt231 can confirm, we expierence this issue also with version 2.95.0 of the provider. For us it also fails with va2109 and others.
Also hitting issues with this, in that sometimes when creating baselines I receive the error "provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced │ an unexpected new value: Root resource was present, but now absent". Seen on multiple projects we have, some on 2.59 of the provider, others on 2.78.
In my experiences for the resources that this occurs for they are not created. If I re-run it tries to create again and normally succeeds.
While this particular error seems refined to initial creation of a baseline, updates also seem to be problematic. E.g. lets say I add a user to the baseline for VA1258 - sometimes TF reports it is done, but if I check manually I see it has not been done. If I re-run pipeline, it warns me that a change was made outside of TF (removing the user from the baseline). It seems it failed to update the first time around, but TF thinks it updated OK.
I'm starting to wonder if the root of this really is the provider and not Azure. I've enabled trace logging and reproduced the issue. What I'm seeing is.
Cross checking the correlation id from the trace log to the activity log in azure I see that the action for this baseline is reported as "started" and then also a separate event for "succeeded".
Going to raise ticket with MS. In my scenario we are setting 6 baselines, randomly but quite frequently upon create one or two would get the error mention in the first post.
As a test I added dependencies in TF just to test if them being done in sequence vs parallel had any impact. It seems in my case it does, have not been able to get it to fail when they are created in sequence.
Then went and tested with the az PowerShell module. Same findings, doing the 6 of them in parallel resulted sometimes in one or two of them not being created despite PowerShell reporting all were created ok.
@jamesw4 , any update on this?
My issues didn’t seem specific to TF. In the end I worked around this using dependencies so they created in sequence rather than in parallel. I think MS has maybe sorted since as when I tried to reproduce the issue weeks later again with PowerShell I couldn’t reproduce the issue anymore. Thanks.
I'm hitting this in April of 2023. It seems that applying multiple baselines to the same data warehouse sku (in my case) causes the above error:
Error: Provider produced inconsistent result after apply When applying changes to azurerm_mssql_database_vulnerability_assessment_rule_baseline.sqlserver-va1248-baseline, provider "provider[\"registry.terraform.io/hashicorp/azurerm\"]" produced an unexpected new value: Root resource was present, but now absent.
If I apply multiple times, it eventually applies all the baselines. But there is definitely some kind of race condition it would seem.
Since changed jobs, hit by this issue again. Used same work around of chaining dependencies between each baseline resource so they are not created in parallel.
Community Note
Terraform (and AzureRM Provider) Version
terraform v0.14.10 azurerm v2.72.0
Affected Resource(s)
azurerm_mssql_database_vulnerability_assessment_rule_baseline
Terraform Configuration Files
Debug Output
I am unable to provide a link with the debug output as this is from an internal gitlab.
Expected Behaviour
Acknowledge that the resource is there and exists.
Actual Behaviour
Resource is created, but Terraform can no longer see it.
Steps to Reproduce
terraform apply
Important Factoids
There is several other reports with the exact same output but for different resources. It doesn't matter if I again run plan and apply, the same error is presented.
References
0000