Simultaneous activation of conflicting service configurations in Azure Table Services

[JayChanggithub]

Describe the bug

As mentioned in the documentation, Azure Classic (built-in) cannot be enabled simultaneously with the cloud service. Earlier, we adopted the Terraform Dynatrace provider for this purpose. However, we're currently encountering the mentioned error. I'd appreciate some assistance to clarify the issue. And if we'd like to choose only cloud service instead of classic(built in) while conflict. How to update below code block? Thanks.

dynatrace provider revision: 1.42.0

code block

resource "dynatrace_azure_credentials"  "azure_credential" {
    active                       = true
    monitor_only_tagged_entities = true
    app_id                       = var.app_id
    auto_tagging                 = true
    directory_id                 = var.directory_id
    key                          = var.svc_secret
    label                        = var.k8s_cluster
    monitor_only_tag_pairs {
       name                      = var.k8s_cluster
       value                     = "true"
    }
}

data "dynatrace_azure_supported_services" "supported_services" {
    except                       = [
        "cloud:azure:sql:servers",
        "cloud:azure:sql:servers:elasticpools:vcore",
        "cloud:azure:sql:servers:elasticpools:dtu",
        "cloud:azure:sql:servers:databases:vcore",
        "cloud:azure:sql:servers:databases:dtu",
        "cloud:azure:storage:storageaccounts:file",
        "cloud:azure:storage:storageaccounts:queue",
        "cloud:azure:storage:storageaccounts:table",
        "cloud:azure:storage:storageaccounts",
        "cloud:azure:storage:storageaccounts:blob",
        "cloud:azure:cache:redis",
        "cloud:azure:apimanagement:service",
        "cloud:azure:network:loadbalancers:gateway",
        "cloud:azure:network:loadbalancers:basic",
        "cloud:azure:network:loadbalancers:standard",
        "cloud:azure:devices:iothubs",
        "cloud:azure:documentdb:databaseaccounts:mongo",
        "cloud:azure:documentdb:databaseaccounts:global",
        "cloud:azure:network:applicationgateways"
    ]
}

resource "dynatrace_azure_service" "azure_services" {
    for_each                     = data.dynatrace_azure_supported_services.supported_services.services
    credentials_id            = dynatrace_azure_credentials.azure_credential.id
    use_recommended_metrics      = true
    name                         = each.key
}

error log

2024-08-26 12:07:33.542 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:documentdb:databaseaccounts:global"]: Creating... |  
-- | -- | --
  |   | 2024-08-26 12:07:33.585 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:loadbalancers:standard"]: Creating... |  
  |   | 2024-08-26 12:07:33.628 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:apimanagement:service"]: Creating... |  
  |   | 2024-08-26 12:07:33.662 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:databases:vcore"]: Creating... |  
  |   | 2024-08-26 12:07:33.706 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:blob"]: Creating... |  
  |   | 2024-08-26 12:07:34.301 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:blob"]: Creation complete after 0s [id=AZURE_CREDENTIALS-22676861347BE1F8#cloud:azure:storage:storageaccounts:blob] |  
  |   | 2024-08-26 12:07:34.305 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:table"]: Creating... |  
  |   | 2024-08-26 12:07:35.137 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:table"]: Creation complete after 1s [id=AZURE_CREDENTIALS-22676861347BE1F8#cloud:azure:storage:storageaccounts:table] |  
  |   | 2024-08-26 12:07:35.141 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers"]: Creating... |  
  |   | 2024-08-26 12:07:35.179 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:file"]: Creating... |  
  |   | 2024-08-26 12:07:35.675 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:file"]: Creation complete after 1s [id=AZURE_CREDENTIALS-22676861347BE1F8#cloud:azure:storage:storageaccounts:file] |  
  |   | 2024-08-26 12:07:35.678 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:loadbalancers:basic"]: Creating... |  
  |   | 2024-08-26 12:07:35.781 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:elasticpools:dtu"]: Creating... |  
  |   | 2024-08-26 12:07:35.827 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:queue"]: Creating... |  
  |   | 2024-08-26 12:07:36.394 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:storage:storageaccounts:queue"]: Creation complete after 0s [id=AZURE_CREDENTIALS-22676861347BE1F8#cloud:azure:storage:storageaccounts:queue] |  
  |   | 2024-08-26 12:07:36.397 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:databases:dtu"]: Creating... |  
  |   | 2024-08-26 12:07:36.430 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:applicationgateways"]: Creating... |  
  |   | 2024-08-26 12:07:36.476 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:documentdb:databaseaccounts:mongo"]: Creating... |  
  |   | 2024-08-26 12:07:36.522 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:devices:iothubs"]: Creating... |  
  |   | 2024-08-26 12:07:36.571 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:elasticpools:vcore"]: Creating... |  
  |   | 2024-08-26 12:07:36.607 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:loadbalancers:gateway"]: Creating... |  
  |   | 2024-08-26 12:07:36.643 | module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:cache:redis"]: Creating... |  
  |   | 2024-08-26 12:07:36.923 | ╷ |  
  |   | 2024-08-26 12:07:36.923 | │ Error: Invalid services configuration: you can't have cloud:azure:sql:servers:databases:dtu and AZURE_SQL services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.923 | │ |  
  |   | 2024-08-26 12:07:36.923 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:databases:dtu"], |  
  |   | 2024-08-26 12:07:36.923 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.923 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.923 | │ |  
  |   | 2024-08-26 12:07:36.923 | ╵ |  
  |   | 2024-08-26 12:07:36.923 | ╷ |  
  |   | 2024-08-26 12:07:36.923 | │ Error: Invalid services configuration: you can't have AZURE_LOAD_BALANCER and cloud:azure:network:loadbalancers:basic services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.923 | │ |  
  |   | 2024-08-26 12:07:36.923 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:loadbalancers:basic"], |  
  |   | 2024-08-26 12:07:36.923 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.923 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.923 | │ |  
  |   | 2024-08-26 12:07:36.923 | ╵ |  
  |   | 2024-08-26 12:07:36.924 | ╷ |  
  |   | 2024-08-26 12:07:36.924 | │ Error: Invalid services configuration: you can't have cloud:azure:devices:iothubs and AZURE_IOT_HUB services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:devices:iothubs"], |  
  |   | 2024-08-26 12:07:36.924 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.924 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | ╵ |  
  |   | 2024-08-26 12:07:36.924 | ╷ |  
  |   | 2024-08-26 12:07:36.924 | │ Error: Invalid services configuration: you can't have AZURE_SQL and cloud:azure:sql:servers services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers"], |  
  |   | 2024-08-26 12:07:36.924 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.924 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | ╵ |  
  |   | 2024-08-26 12:07:36.924 | ╷ |  
  |   | 2024-08-26 12:07:36.924 | │ Error: Invalid services configuration: you can't have AZURE_COSMOS_DB and cloud:azure:documentdb:databaseaccounts:global services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:documentdb:databaseaccounts:global"], |  
  |   | 2024-08-26 12:07:36.924 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.924 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | ╵ |  
  |   | 2024-08-26 12:07:36.924 | ╷ |  
  |   | 2024-08-26 12:07:36.924 | │ Error: Invalid services configuration: you can't have AZURE_REDIS_CACHE and cloud:azure:cache:redis services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:cache:redis"], |  
  |   | 2024-08-26 12:07:36.924 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.924 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.924 | │ |  
  |   | 2024-08-26 12:07:36.924 | ╵ |  
  |   | 2024-08-26 12:07:36.925 | ╷ |  
  |   | 2024-08-26 12:07:36.925 | │ Error: Invalid services configuration: you can't have cloud:azure:network:loadbalancers:gateway and AZURE_LOAD_BALANCER services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.925 | │ |  
  |   | 2024-08-26 12:07:36.925 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:loadbalancers:gateway"], |  
  |   | 2024-08-26 12:07:36.925 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.925 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.925 | │ |  
  |   | 2024-08-26 12:07:36.925 | ╵ |  
  |   | 2024-08-26 12:07:36.925 | ╷ |  
  |   | 2024-08-26 12:07:36.925 | │ Error: Invalid services configuration: you can't have AZURE_SQL and cloud:azure:sql:servers:elasticpools:vcore services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.925 | │ |  
  |   | 2024-08-26 12:07:36.925 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:elasticpools:vcore"], |  
  |   | 2024-08-26 12:07:36.925 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.925 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.925 | │ |  
  |   | 2024-08-26 12:07:36.925 | ╵ |  
  |   | 2024-08-26 12:07:36.925 | ╷ |  
  |   | 2024-08-26 12:07:36.925 | │ Error: Invalid services configuration: you can't have cloud:azure:network:applicationgateways and AZURE_APPLICATION_GATEWAY services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.925 | │ |  
  |   | 2024-08-26 12:07:36.925 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:applicationgateways"], |  
  |   | 2024-08-26 12:07:36.925 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.925 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.925 | │ |  
  |   | 2024-08-26 12:07:36.925 | ╵ |  
  |   | 2024-08-26 12:07:36.926 | ╷ |  
  |   | 2024-08-26 12:07:36.926 | │ Error: Invalid services configuration: you can't have cloud:azure:network:loadbalancers:standard and AZURE_LOAD_BALANCER services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.926 | │ |  
  |   | 2024-08-26 12:07:36.926 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:network:loadbalancers:standard"], |  
  |   | 2024-08-26 12:07:36.926 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.926 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.926 | │ |  
  |   | 2024-08-26 12:07:36.926 | ╵ |  
  |   | 2024-08-26 12:07:36.926 | ╷ |  
  |   | 2024-08-26 12:07:36.926 | │ Error: Invalid services configuration: you can't have cloud:azure:sql:servers:elasticpools:dtu and AZURE_SQL services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.926 | │ |  
  |   | 2024-08-26 12:07:36.926 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:elasticpools:dtu"], |  
  |   | 2024-08-26 12:07:36.926 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.926 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.926 | │ |  
  |   | 2024-08-26 12:07:36.926 | ╵ |  
  |   | 2024-08-26 12:07:36.927 | ╷ |  
  |   | 2024-08-26 12:07:36.927 | │ Error: Invalid services configuration: you can't have cloud:azure:apimanagement:service and AZURE_API_MANAGEMENT_SERVICE services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.927 | │ |  
  |   | 2024-08-26 12:07:36.927 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:apimanagement:service"], |  
  |   | 2024-08-26 12:07:36.927 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.927 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.927 | │ |  
  |   | 2024-08-26 12:07:36.927 | ╵ |  
  |   | 2024-08-26 12:07:36.927 | ╷ |  
  |   | 2024-08-26 12:07:36.927 | │ Error: Invalid services configuration: you can't have AZURE_COSMOS_DB and cloud:azure:documentdb:databaseaccounts:mongo services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.927 | │ |  
  |   | 2024-08-26 12:07:36.927 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:documentdb:databaseaccounts:mongo"], |  
  |   | 2024-08-26 12:07:36.927 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.927 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.927 | │ |  
  |   | 2024-08-26 12:07:36.927 | ╵ |  
  |   | 2024-08-26 12:07:36.928 | ╷ |  
  |   | 2024-08-26 12:07:36.928 | │ Error: Invalid services configuration: you can't have cloud:azure:sql:servers:databases:vcore and AZURE_SQL services turned on simultaneously |  
  |   | 2024-08-26 12:07:36.928 | │ |  
  |   | 2024-08-26 12:07:36.928 | │   with module.dynatrace_azure.dynatrace_azure_service.azure_services["cloud:azure:sql:servers:databases:vcore"], |  
  |   | 2024-08-26 12:07:36.928 | │   on ../modules/dynatrace_azure_credentials/dynatrace_azure_credentials.tf line 21, in resource "dynatrace_azure_service" "azure_services": |  
  |   | 2024-08-26 12:07:36.928 | │   21: resource "dynatrace_azure_service" "azure_services" { |  
  |   | 2024-08-26 12:07:36.928 | │ |  
  |   | 2024-08-26 12:07:36.928 | ╵

[kishikawa12]

Hi @JayChanggithub,

Just to clarify, previously you were able to configure both classic and cloud services at the same time via Terraform (API) despite that shouldn't have been possible based off of the documentation? And the behavior now is that you are unable to do so due to the errors?

If that's the case, I don't believe it was intentional that they allowed this functionality via the API (as a result, also in Terraform). The error messages aren't being produced by the Terraform code, but is being returned by our API.

I would recommend to migrate to cloud services as that would allow additional configuration of monitored services.

[JayChanggithub]

Hi @kishikawa12

Thank you for the clarification. I would like to request some best practice examples of Terraform code for these resources. If we decide to migrate to cloud services, should we reuse the previous approach? If so, do you have any recommendations?

The following code is our latest changed.

resource "dynatrace_azure_credentials"  "azure_credential" {
    active                       = true
    monitor_only_tagged_entities = true
    app_id                       = var.app_id
    auto_tagging                 = true
    directory_id                 = var.directory_id
    key                          = var.svc_secret
    label                        = var.k8s_cluster
    monitor_only_tag_pairs {
       name                      = var.k8s_cluster
       value                     = "true"
    }
}

data "dynatrace_azure_supported_services" "supported_services" {
    except                       = [
        "AZURE_STORAGE_ACCOUNT",
        "AZURE_SQL",
        "AZURE_LOAD_BALANCER",
        "AZURE_IOT_HUB",
        "AZURE_COSMOS_DB",
        "AZURE_REDIS_CACHE",
        "AZURE_APPLICATION_GATEWAY",
        "AZURE_API_MANAGEMENT_SERVICE"
    ]
}

resource "dynatrace_azure_service" "azure_services" {
    for_each                     = data.dynatrace_azure_supported_services.supported_services.services
    credentials_id               = dynatrace_azure_credentials.azure_credential.id
    use_recommended_metrics      = true
    name                         = each.key
}

[kishikawa12]

Hi @JayChanggithub,

I double checked with R&D on Azure supported services since I believe the documentation may not have been entirely clear.

The most important thing to note is that for full coverage, you would need to utilize a combination of built-in AND cloud services since there are services that don't exist in the new cloud services.

If you take a look at the right side of this table, these are the eight supported services that have a corresponding new cloud service. In other words, disabling these eight services in your Terraform configuration will migrate you to the new cloud services AND leave the built-in services required to give you full coverage of all supported services.

The most recent Terraform code you provided looks good, I've also tested this in my environment and confirmed that it works as expected.

[JayChanggithub]

@kishikawa12
Thank you for the clarification. I have been using the code I provided across several deployment environments, and the screenshot shows one of those environments.

[kishikawa12]

@JayChanggithub Looks good to me, was there something that didn't look correct?

Thanks!

[kishikawa12]

@JayChanggithub I'll close this ticket for now, please feel free to reopen if you run into anymore issues. Thanks!