terraform-azurerm-avm-res-kusto-cluster

Manages a Kusto (also known as Azure Data Explorer) Cluster and databases

[!IMPORTANT] As the overall AVM framework is not GA (generally available) yet - the CI framework and test automation is not fully functional and implemented across all supported languages yet - breaking changes are expected, and additional customer feedback is yet to be gathered and incorporated. Hence, modules MUST NOT be published at version 1.0.0 or higher at this time.

All module MUST be published as a pre-release version (e.g., 0.1.0, 0.1.1, 0.2.0, etc.) until the AVM framework becomes GA.

However, it is important to note that this DOES NOT mean that the modules cannot be consumed and utilized. They CAN be leveraged in all types of environments (dev, test, prod etc.). Consumers can treat them just like any other IaC module and raise issues or feature requests against them as they learn from the usage of the module. Consumers should also read the release notes for each version, if considering updating to a more recent version of a module to see if there are any considerations or breaking changes etc.

Requirements

The following requirements are needed by this module:

• terraform (>= 1.7.0)

• azurerm (>= 3.103.0)

• random (>= 3.5.0)

Providers

The following providers are used by this module:

• azurerm (>= 3.103.0)

• random (>= 3.5.0)

Resources

The following resources are used by this module:

• azurerm_kusto_cluster.this (resource)
• azurerm_management_lock.this (resource)
• azurerm_monitor_diagnostic_setting.this (resource)
• azurerm_private_endpoint.this (resource)
• azurerm_private_endpoint_application_security_group_association.this (resource)
• azurerm_resource_group_template_deployment.telemetry (resource)
• azurerm_role_assignment.this (resource)
• random_id.telem (resource)

Required Inputs

The following input variables are required:

location

Description: Azure region where the resource should be deployed. If null, the location will be inferred from the resource group location.

Type: string

name

Description: The name of the this resource.

Type: string

resource_group_name

Description: The resource group where the resources will be deployed.

Type: string

sku

Description: A sku block supports the following:

name - (Required) The name of the SKU.

Possible values are:

• Dev(No SLA)_Standard_D11_v2,
• Dev(No SLA)_Standard_E2a_v4,
• Standard_D14_v2,
• Standard_D11_v2,
• Standard_D16d_v5,
• Standard_D13_v2,
• Standard_D12_v2,
• Standard_DS14_v2+4TB_PS,
• Standard_DS14_v2+3TB_PS,
• Standard_DS13_v2+1TB_PS,
• Standard_DS13_v2+2TB_PS,
• Standard_D32d_v5,
• Standard_D32d_v4,
• Standard_EC8ads_v5,
• Standard_EC8as_v5+1TB_PS,
• Standard_EC8as_v5+2TB_PS,
• Standard_EC16ads_v5,
• Standard_EC16as_v5+4TB_PS,
• Standard_EC16as_v5+3TB_PS,
• Standard_E80ids_v4,
• Standard_E8a_v4,
• Standard_E8ads_v5,
• Standard_E8as_v5+1TB_PS,
• Standard_E8as_v5+2TB_PS,
• Standard_E8as_v4+1TB_PS,
• Standard_E8as_v4+2TB_PS,
• Standard_E8d_v5,
• Standard_E8d_v4,
• Standard_E8s_v5+1TB_PS,
• Standard_E8s_v5+2TB_PS,
• Standard_E8s_v4+1TB_PS,
• Standard_E8s_v4+2TB_PS,
• Standard_E4a_v4,
• Standard_E4ads_v5,
• Standard_E4d_v5,
• Standard_E4d_v4,
• Standard_E16a_v4,
• Standard_E16ads_v5,
• Standard_E16as_v5+4TB_PS,
• Standard_E16as_v5+3TB_PS,
• Standard_E16as_v4+4TB_PS,
• Standard_E16as_v4+3TB_PS,
• Standard_E16d_v5,
• Standard_E16d_v4,
• Standard_E16s_v5+4TB_PS,
• Standard_E16s_v5+3TB_PS,
• Standard_E16s_v4+4TB_PS,
• Standard_E16s_v4+3TB_PS,
• Standard_E64i_v3,
• Standard_E2a_v4,
• Standard_E2ads_v5,
• Standard_E2d_v5,
• Standard_E2d_v4,
• Standard_L8as_v3,
• Standard_L8s,
• Standard_L8s_v3,
• Standard_L8s_v2,
• Standard_L4s,
• Standard_L16as_v3,
• Standard_L16s,
• Standard_L16s_v3,
• Standard_L16s_v2,
• Standard_L32as_v3
• Standard_L32s_v3
  capacity - (Optional) Specifies the node count for the cluster. Boundaries depend on the SKU name.
  NOTE:
  If no optimized_auto_scale block is defined, then the capacity is required. ~> NOTE: If an optimized_auto_scale block is defined and no capacity is set, then the capacity is initially set to the value of minimum_instances.

Type:

object({
    name     = string
    capacity = number
  })

Optional Inputs

The following input variables are optional (have default values):

allowed_fqdns

Description: (Optional) List of allowed FQDNs(Fully Qualified Domain Name) for egress from Cluster.

Type: set(string)

Default: null

allowed_ip_ranges

Description: (Optional) The list of ips in the format of CIDR allowed to connect to the cluster.

Type: set(string)

Default: null

auto_stop_enabled

Description: (Optional) Specifies if the cluster could be automatically stopped (due to lack of data or no activity for many days).

Defaults to true.

Type: bool

Default: true

customer_managed_key

Description: Customer managed keys that should be associated with the resource.

Type:

object({
    key_vault_resource_id = string
    key_name              = string
    key_version           = optional(string, null)
    user_assigned_identity = optional(object({
      resource_id = string
    }), null)
  })

Default: null

databases

Description: (Optional) A map of kusto database objects:

• name - (Required) The name of the Kusto Database to create. Changing this forces a new resource to be created.
• location - (Optional) The location where the Kusto Database should be created. If not provided, will default to the location of the kusto cluster. Changing this forces a new resource to be created.
• resource_group_name - (Optional) Specifies the Resource Group where the Kusto Database should exist. If not provided, will default to the location of the kusto cluster. Changing this forces a new resource to be created.
• cluster_name - (Optional) Specifies the name of the Kusto Cluster this database will be added to. If not provided, will default to the location of the kusto cluster. Changing this forces a new resource to be created.
• hot_cache_period - (Optional) The time the data that should be kept in cache for fast queries as ISO 8601 timespan. Default is unlimited. For more information see: ISO 8601 Timespan.
• soft_delete_period - (Optional) The time the data should be kept before it stops being accessible to queries as ISO 8601 timespan. Default is unlimited. For more information see: ISO 8601 Timespan

Type:

map(object({
    name                = string
    location            = optional(string, null)
    resource_group_name = optional(string, null)
    cluster_name        = optional(string, null)
    hot_cache_period    = optional(string)
    soft_delete_period  = optional(string)
  }))

Default: {}

diagnostic_settings

Description: A map of diagnostic settings to create on the Key Vault. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.

• event_hub_authorization_rule_resource_id - (Optional) The resource ID of the event hub authorization rule to send logs and metrics to.
• event_hub_name - (Optional) The name of the event hub. If none is specified, the default event hub will be selected.
• log_analytics_destination_type - (Optional) The destination type for the diagnostic setting. Possible values are Dedicated and AzureDiagnostics. Defaults to Dedicated.
• log_categories - (Optional) A set of log categories to send to the log analytics workspace. Defaults to [].
• log_groups - (Optional) A set of log groups to send to the log analytics workspace. Defaults to ["allLogs"].
• marketplace_partner_resource_id - (Optional) The full ARM resource ID of the Marketplace resource to which you would like to send Diagnostic LogsLogs.
• metric_categories - (Optional) A set of metric categories to send to the log analytics workspace. Defaults to ["AllMetrics"].
• name - (Optional) The name of the diagnostic setting. One will be generated if not set, however this will not be unique if you want to create multiple diagnostic setting resources.
• storage_account_resource_id - (Optional) The resource ID of the storage account to send logs and metrics to.
• workspace_resource_id - (Optional) The resource ID of the log analytics workspace to send logs and metrics to.

Type:

map(object({
    name                                     = optional(string, null)
    log_categories                           = optional(set(string), [])
    log_groups                               = optional(set(string), ["allLogs"])
    metric_categories                        = optional(set(string), ["AllMetrics"])
    log_analytics_destination_type           = optional(string, "Dedicated")
    workspace_resource_id                    = optional(string, null)
    storage_account_resource_id              = optional(string, null)
    event_hub_authorization_rule_resource_id = optional(string, null)
    event_hub_name                           = optional(string, null)
    marketplace_partner_resource_id          = optional(string, null)
  }))

Default: {}

disk_encryption_enabled

Description: (Optional) Specifies if the cluster's disks are encrypted.

Type: bool

Default: true

double_encryption_enabled

Description: (Optional) Is the cluster's double encryption enabled? Changing this forces a new resource to be created.

Type: bool

Default: null

enable_telemetry

Description: This variable controls whether or not telemetry is enabled for the module.
For more information see https://aka.ms/avm/telemetryinfo.
If it is set to false, then no telemetry will be collected.

Type: bool

Default: true

kusto_cluster_principal_assignments

Description: A map that manages a Kusto Cluster Principal Assignment.

• name (Required) The name of the Kusto cluster principal assignment. Changing this forces a new resource to be created.
• principal_id (Required) The object id of the principal. Changing this forces a new resource to be created.
• principal_type (Required) The type of the principal. Valid values include App, Group, User. Changing this forces a new resource to be created.
• role (Required) The cluster role assigned to the principal. Valid values include AllDatabasesAdmin and AllDatabasesViewer. Changing this forces a new resource to be created.
• tenant_id (Required) The tenant id in which the principal resides. Changing this forces a new resource to be created.

Type:

map(object({
    name           = string
    principal_id   = string
    principal_type = string
    role           = string
    tenant_id      = string
  }))

Default: {}

kusto_database_principal_assignment

Description: A map that manages a Kusto (also known as Azure Data Explorer) Database Principal Assignment.

• database_name (Required) The name of the database in which to create the resource. Changing this forces a new resource to be created.
• name (Required) The name of the Kusto cluster principal assignment. Changing this forces a new resource to be created.
• principal_id (Required) The object id of the principal. Changing this forces a new resource to be created.
• principal_type (Required) The type of the principal. Valid values include App, Group, User. Changing this forces a new resource to be created.
• role (Required) The cluster role assigned to the principal. Valid values include AllDatabasesAdmin and AllDatabasesViewer. Changing this forces a new resource to be created.
• tenant_id (Required) The tenant id in which the principal resides. Changing this forces a new resource to be created.

Type:

map(object({
    database_name  = string
    name           = string
    principal_id   = string
    principal_type = string
    role           = string
    tenant_id      = string
  }))

Default: {}

language_extensions

Description: (Optional) An list of language_extensions to enable.

Valid values are: PYTHON, PYTHON_3.10.8 and R.

PYTHON is used to specify Python 3.6.5 image and PYTHON_3.10.8 is used to specify Python 3.10.8 image.
Note that PYTHON_3.10.8 is only available in skus which support nested virtualization.

NOTE:
In v4.0.0 and later version of the AzureRM Provider,
language_extensions will be changed to a list of language_extension block.
In each block, name and image are required.
name is the name of the language extension, possible values are PYTHON, R.
image is the image of the language extension, possible values are Python3_6_5, Python3_10_8 and R.

Type: set(string)

Default: null

lock

Description: The lock level to apply. Default is None. Possible values are None, CanNotDelete, and ReadOnly.

Type:

object({
    name = optional(string, null)
    kind = string
  })

Default: null

managed_identities

Description: Managed identities to be created for the resource.

Type:

object({
    system_assigned            = optional(bool, false)
    user_assigned_resource_ids = optional(set(string), [])
  })

Default: {}

optimized_auto_scale

Description: A optimized_auto_scale block supports the following:

minimum_instances - (Required) The minimum number of allowed instances. Must between 0 and 1000.

maximum_instances - (Required) The maximum number of allowed instances. Must between 0 and 1000.

Type:

object({
    maximum_instances = number
    minimum_instances = number
  })

Default: null

outbound_network_access_restricted

Description: (Optional) Whether to restrict outbound network access.
Value is optional but if passed in, must be true or false.

Default is false.

Type: bool

Default: false

private_endpoints

Description: A map of private endpoints to create on this resource. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.

• name - (Optional) The name of the private endpoint. One will be generated if not set.
• role_assignments - (Optional) A map of role assignments to create on the private endpoint. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time. See var.role_assignments for more information.
• lock - (Optional) The lock level to apply to the private endpoint. Default is None. Possible values are None, CanNotDelete, and ReadOnly.
• tags - (Optional) A mapping of tags to assign to the private endpoint.
• subnet_resource_id - The resource ID of the subnet to deploy the private endpoint in.
• private_dns_zone_group_name - (Optional) The name of the private DNS zone group. One will be generated if not set.
• private_dns_zone_resource_ids - (Optional) A set of resource IDs of private DNS zones to associate with the private endpoint. If not set, no zone groups will be created and the private endpoint will not be associated with any private DNS zones. DNS records must be managed external to this module.
• application_security_group_resource_ids - (Optional) A map of resource IDs of application security groups to associate with the private endpoint. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.
• private_service_connection_name - (Optional) The name of the private service connection. One will be generated if not set.
• network_interface_name - (Optional) The name of the network interface. One will be generated if not set.
• location - (Optional) The Azure location where the resources will be deployed. Defaults to the location of the resource group.
• resource_group_name - (Optional) The resource group where the resources will be deployed. Defaults to the resource group of this resource.
• ip_configurations - (Optional) A map of IP configurations to create on the private endpoint. If not specified the platform will create one. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.
  • name - The name of the IP configuration.
  • private_ip_address - The private IP address of the IP configuration.

Type:

map(object({
    name = optional(string, null)
    role_assignments = optional(map(object({
      role_definition_id_or_name             = string
      principal_id                           = string
      description                            = optional(string, null)
      skip_service_principal_aad_check       = optional(bool, false)
      condition                              = optional(string, null)
      condition_version                      = optional(string, null)
      delegated_managed_identity_resource_id = optional(string, null)
      principal_type                         = optional(string, null)
    })), {})
    lock = optional(object({
      kind = string
      name = optional(string, null)
    }), null)
    tags                                    = optional(map(string), null)
    subnet_resource_id                      = string
    private_dns_zone_group_name             = optional(string, "default")
    private_dns_zone_resource_ids           = optional(set(string), [])
    application_security_group_associations = optional(map(string), {})
    private_service_connection_name         = optional(string, null)
    network_interface_name                  = optional(string, null)
    location                                = optional(string, null)
    resource_group_name                     = optional(string, null)
    ip_configurations = optional(map(object({
      name               = string
      private_ip_address = string
    })), {})
  }))

Default: {}

public_ip_type

Description: (Optional) Indicates what public IP type to create - IPv4 (default), or DualStack (both IPv4 and IPv6). Defaults to IPv4.

Type: string

Default: "IPv4"

public_network_access_enabled

Description: (Optional) Is the public network access enabled? Defaults to true.

Type: bool

Default: false

purge_enabled

Description: (Optional) Specifies if the purge operations are enabled.

Type: bool

Default: null

role_assignments

Description: A map of role assignments to create on this resource. The map key is deliberately arbitrary to avoid issues where map keys maybe unknown at plan time.

• role_definition_id_or_name - The ID or name of the role definition to assign to the principal.
• principal_id - The ID of the principal to assign the role to.
• description - The description of the role assignment.
• skip_service_principal_aad_check - If set to true, skips the Azure Active Directory check for the service principal in the tenant. Defaults to false.
• condition - The condition which will be used to scope the role assignment.
• condition_version - The version of the condition syntax. Valid values are '2.0'.
• principal_type - (Optional) The type of the principal_id. Possible values are User, Group and ServicePrincipal. It is necessary to explicitly set this attribute when creating role assignments if the principal creating the assignment is constrained by ABAC rules that filters on the PrincipalType attribute.

Note: only set skip_service_principal_aad_check to true if you are assigning a role to a service principal.

Type:

map(object({
    role_definition_id_or_name             = string
    principal_id                           = string
    description                            = optional(string, null)
    skip_service_principal_aad_check       = optional(bool, false)
    condition                              = optional(string, null)
    condition_version                      = optional(string, null)
    delegated_managed_identity_resource_id = optional(string, null)
    principal_type                         = optional(string, null)
  }))

Default: {}

streaming_ingestion_enabled

Description: (Optional) Specifies if the streaming ingest is enabled.

Type: bool

Default: null

tags

Description: Map of tags to assign to the resource.

Type: map(string)

Default: null

trusted_external_tenants

Description: (Optional) Specifies a list of tenant IDs that are trusted by the cluster.
New or updated Kusto Cluster will only allow your own tenant by default.

Use trusted_external_tenants = ["*"] to explicitly allow all other tenants,
trusted_external_tenants = [] for only your tenant or
trusted_external_tenants = ["", ""] to allow specific other tenants.

Type: set(string)

Default: []

virtual_network_configuration

Description: (Optional) A virtual_network_configuration block as defined below.
Changing this forces a new resource to be created.

A virtual_network_configuration block supports the following:

subnet_id - (Required) The subnet resource id.

engine_public_ip_id - (Required) Engine service's public IP address resource id.

data_management_public_ip_id - (Required) Data management's service public IP address resource id.

Type:

object({
    data_management_public_ip_id = string
    engine_public_ip_id          = string
    subnet_id                    = string
  })

Default: null

zones

Description: (Optional) Specifies a list of Availability Zones in which this Kusto Cluster should be located. Changing this forces a new Kusto Cluster to be created.

Type: set(string)

Default: null

Outputs

The following outputs are exported:

data_ingestion_uri

Description: The Kusto Cluster URI to be used for data ingestion.

id

Description: The Kusto Cluster ID.

identity

Description: An identity block exports the following:

principal_id - The Principal ID associated with this System Assigned Managed Service Identity.

tenant_id - The Tenant ID associated with this System Assigned Managed Service Identity.

private_endpoints

Description: A map of private endpoints. The map key is the supplied input to var.private_endpoints. The map value is the entire azurerm_private_endpoint resource.

resource

Description: This is the full output for the resource.

resource_id

Description: The Kusto Cluster resource ID.

uri

Description: The FQDN of the Azure Kusto Cluster.

Modules

The following Modules are called:

kusto_cluster_principal_assignment

Source: ./modules/azurerm_kusto_cluster_principal_assignment

Version:

kusto_database

Source: ./modules/azurerm_kusto_database

Version:

kusto_database_principal_assignment

Source: ./modules/azurerm_kusto_database_principal_assignment

Version:

Data Collection

The software may collect information about you and your use of the software and send it to Microsoft. Microsoft may use this information to provide services and improve our products and services. You may turn off the telemetry as described in the repository. There are also some features in the software that may enable you and Microsoft to collect data from users of your applications. If you use these features, you must comply with applicable law, including providing appropriate notices to users of your applications together with a copy of Microsoft’s privacy statement. Our privacy statement is located at https://go.microsoft.com/fwlink/?LinkID=824704. You can learn more about data collection and use in the help documentation and our privacy statement. Your use of the software operates as your consent to these practices.