Open katbyte opened 2 years ago
Hi @katbyte! Only comments on the schema or also on the proposal in general? Schema looks fine in my opinion, except for the Resoruce_group_name
-> resource_group_name
πππ½
location
seems valuable, certainly if it is possible to override for specific resources (failover databases etc)resource_group_name
has less value for the configurations I'm familiar with, as it is often dynamic (not sure if that is supported?) for different environments or is spread among multiple resource groups. Certainly wouldn't hurt :)tags
would be a huge win, even if it would mean general strict rules were applied on the first implementation. Would be nice if it is possible to easily extend the base tags with specific tags here and there.It's worth calling out that tags get really complicated fwiw - since (as I'm sure you know from the upstream issues) whilst tags look consistent in Azure, different resources have different limitations (some are lower-case only keys/values, others are an array rather than a dictionary etc) - which'll likely end up guiding the direction we take here.
@aristosvo to your question about a dynamic resource group name, that wouldn't be supported in the Provider block (since Core doesn't support dynamic interpolation in the provider block at this time) - but all of these would likely be overridable on a per-resource level if we take this approach.
Hi all,
I'd like to share something about the implementation that we may need to introduce some computed properties in each resource to support default value in provider block.
The reason is, take tags
as an example, if we have
provider azurerm {
tags = {
"Key": "1"
}
}
resource "azurerm_xxx" "test" {
// no tags
}
azurerm_xxx.test
will use default tags from provider block and set back to its state, to prevent plan diff, we need to change property tags
from optional
to optional + computed
. After that, if we run plan command again, we don't know where the tags
in state came from, it can be default value from provider, it can be value from resource block like the following.
provider azurerm {
tags = {
"Key": "1"
}
}
resource "azurerm_xxx" "test" {
tags = {
"Key": "1"
}
}
This will cause problem when default value changes like the following, we don't know how to produce the plan: resource maybe use the default value and should be updated because default value updates; resource maybe not use the default value and no need to update.
provider azurerm {
tags = {
"Key": "2"
}
}
Provider aws introduced a computed tags_all
to store all tags and keep tags
optional. And this will solve the problem.
About
we need to change property tags from optional to optional + computed. After that, if we run plan command again, we don't know where the tags in state came from, it can be default value from provider, it can be value from resource block like the following.
I have some updates, this is unblocked by a function called GetRawConfig
introduced in plugin-sdk v2.8.0 . This function will return userβs config and can solve above problem.
@ms-henglu that'd make sense if we made it only possible to specify these on the provider block, but that'd just cause conflicts with every resource specifying them inline otherwise.
Unfortunately we can't make Tags Optional and Computed since it breaks the Policy-as-Code use-cases, instead we'll have to determine which tags should be specified at/filtered out from the Provider block when setting these into the state - which becomes more of a design question more than a technical one (although this also involves a non-trivial number of changes in the Provider too).
For that reason we likely won't be exposing Tags in the same manner as we do against each Resource - it could be that we do this via something like global_tags
which get combined with the tags defined against the resource (or some tags which should be ignored/can/can't be overridden etc) - at this point this is mostly a design question which still requires investigation.
Hi @tombuildsstuff ,
that'd make sense if we made it only possible to specify these on the provider block, but that'd just cause conflicts with every resource specifying them inline otherwise.
I thought that tags defined against the resource can override the whole default tags. But this may not be easy to just extend some tags.
we'll have to determine which tags should be specified at/filtered out from the Provider block when setting these into the state
Sounds good!
How about location
and resource_group_name
, they don't need to be extended, maybe they can be defined as O+C and use GetRawConfig
to know whether user overrides the default value.
But it would break the Policy-as-Code use-cases π€
@ms-henglu unfortunately you can't dynamically change the schema at runtime, so Location and Resource Group can't become O&C in some cases and Required in others.
But it would break the Policy-as-Code use-cases π€
I don't recall the specific details off the top of my head unfortunately - this was an issue in 1.x where this broke some policy-as-code tools in subtle ways - since this shouldn't have been computed in the first place, we fixed this in 2.0 and it's been fine since.
It's worth calling out that tags get really complicated fwiw - since (as I'm sure you know from the upstream issues) whilst tags look consistent in Azure, different resources have different limitations (some are lower-case only keys/values, others are an array rather than a dictionary etc) - which'll likely end up guiding the direction we take here.
How complete does this need to be? Are there constraints that could be documented to win for most resources. I'm not an expert in all azure resources, but so far I've found two exceptions (the first may be a class of exceptions):
Our use case could be more of an ignore use case for tags. We could apply values via policy, but then would want to tell terraform to ignore to policy driven tags. Like the case above, we'd still want to be able to override tag values for certain resources to add specificity.
Hi everyone, any news on that specific feature ?
π hey folks
We've spent some time digging into this and unfortunately it looks like this isn't going to be possible in it's current form.
Whilst it's technically possible to add a resource_group_name
and location
field to the provider
block, as mentioned above unfortunately Terraform doesn't have a means of dynamically changing the schema available based on fields within the provider
block.
As such to implement this we'd need to instead source this from an Environment Variable - and whilst that's technically possible we believe this could lead to user confusion when running on different machines with/without this variable defined (as the error would be "the field location
must be set", which is unclear).
What this means is that unfortunately we won't be able to support location
or resource_group_name
on the Provider block at this time - but this is something we'll consider in the future.
Tags are unfortunately considerably more complicated than they may first appear - whilst most of the Resources within Resource Manager support Tags, not all Tags are created equally - with some API's having different limitations (either the number of tags available, the available casing on the tags, certain values which can't be set etc).
What this means is that whilst we could look to support this on all applicable resources, the user experience would vary wildly, with some resources supporting it and others not. Were we to implement this on all resources and remove all applicable validation for Tags - users would be faced with indirect validation issues (similar to the Location/Resource Group problems described above).
Whilst it would have been nice to have a clear divide as @etaham has suggested above by supporting this for Resource Manager and not Data Plane resources - unfortunately that doesn't appear to be possible with how tags work today.
All that said, whilst we're currently not in a position to support this with the user experience it needs - I believe there is a path forward to supporting either default_tags
/tag_keys_to_ignore
within the Provider block in the future.
One of the things we'll be working through over the coming months is to split the Create and Update function to allow the ignore_changes
functionality to work consistently across all fields/resources.
Whilst this may sound like an unrelated change, this'd ultimately solve the "policy as code" problem above, meaning that the tags
field didn't need to become computed - which solves this problem in a different manner.
Notably this won't fix the problem described above with the different possibilities for tags, we may be able to do something else at that point in time, with/without some additional upstream changes.
However since this for the moment this isn't possible unfortunately I'm going to have to remove this from the 3.0 milestone, but I'm going to assign this to the 4.0 milestone so that we can take a look at this in the future.
Thanks!
Hope to see the tag_keys_to_ignore
functionality soon. This is really cumbersome to manage currently with default tags being added from 'higher up' (cloud management).
Strongly support the feature.
Implementation can be copy/pasted from the AWS one: https://registry.terraform.io/providers/hashicorp/aws/latest/docs#default_tags. It is great and developers won't be annoyed.
Example:
provider "azurerm" {
default_tags = {
app = "app-xx"
last_update = timestamp()
managed_by = "Terraform"
sources = "https://github.com/xxx/app-xx"
}
}
Additionally to the "tag_keys_to_ignore" feature it would be useful if this could be based on prefix. For example a use case could be a tags are added based on config management i.e. ansible_xxxx with varying tag names. This would prevent having to manually list them as well over time if they expand the code block wouldn't have to be updated.
We at @swisspost miss the feature to inherit tags to resources created by the azurerm provider. The AWS Provider has this feature since 3.38.0 (https://registry.terraform.io/providers/hashicorp/aws/latest/docs/guides/resource-tagging#propagating-tags-to-all-resources). This has been done 3 Years ago
would be very beneficial and appreciated if the azurerm provider was set on the same level as the aws provider :)
Does anyone know when 4.0 is going to be released? This is a feature we really need at my organization.
Has anyone discovered a workaround other than adding on a lifecycle
argument to every single resource?
Thanks!
@AMoghrabi I use an implementation where i have a default_tags
variable and then i set use this on every resource and if i want to add something additional for a specific resource I merge it:
variable "default_tags" {
type = map(string)
default = {
BillingCode = "internal-code"
Service = "my-service"
Terraform = "my-service-stack"
}
}
Then on a resource:
resource "azurerm_storage_account" "web_app" {
name = join("", [var.service, terraform.workspace])
resource_group_name = azurerm_resource_group.web_app.name
location = azurerm_resource_group.web_app.location
account_kind = "StorageV2"
account_tier = "Standard"
account_replication_type = "LRS"
access_tier = "Cool"
public_network_access_enabled = true
min_tls_version = "TLS1_2"
# shared_access_key_enabled = false -- need to understand the consequences of setting the default from true to false
blob_properties {
versioning_enabled = false
delete_retention_policy {
days = 365
}
container_delete_retention_policy {
days = 7
}
}
tags = var.azure_default_tags
}
And then if I want to add something specific like a webapp version above the defaults, I merge them:
resource "azurerm_windows_web_app" "web_app" {
name = local.web_app_name
resource_group_name = var.app_service_resource_name
location = var.az_region
service_plan_id = var.app_service_plan_id
virtual_network_subnet_id = var.app_uses_ase == false ? var.virtual_network_subnet_id : null
enabled = true
https_only = true
client_affinity_enabled = false
public_network_access_enabled = true
client_certificate_enabled = false
app_settings = local.app_settings
site_config {
always_on = true
http2_enabled = true
local_mysql_enabled = false
ftps_state = "Disabled"
managed_pipeline_mode = "Integrated"
minimum_tls_version = "1.2"
remote_debugging_enabled = false
use_32_bit_worker = false
vnet_route_all_enabled = true
websockets_enabled = true
default_documents = ["default.aspx"]
health_check_path = ""
application_stack {
current_stack = "dotnet"
dotnet_version = "v4.0"
}
virtual_application {
physical_path = "site\\wwwroot"
virtual_path = "/"
preload = false
}
}
connection_string {
name = "PawaDatabase"
type = "SQLServer"
value = module.user_connection.connection_string
}
connection_string {
name = "PawaDatabaseAdmin"
type = "SQLServer"
value = module.admin_connection.connection_string
}
connection_string {
name = "StorageAccount"
type = "Custom"
value = azurerm_storage_account.web_app.primary_connection_string
}
connection_string {
name = "FileStorageAccount"
type = "Custom"
value = azurerm_storage_account.web_app_files.primary_connection_string
}
sticky_settings {
connection_string_names = [
"StorageAccount",
"FileStorageAccount"
]
}
tags = merge({
Version = local.web_app_version
Environment = terraform.workspace
}, var.azure_default_tags)
}
@danpetitt I guess we all use similar solution like yours, it's just it's too cumbersome. It would be nice to have default tags where applicable, and just add additional tags where needed.
@danpetitt although I appreciate you sharing a workaround it is not solving the actual problem. The provider should be aware of which resources support tags and allow you to add them without unnecessary code lines. Currently one must check each and every resource for tag support first and then add the respective attribute to the resource. Default tags exist for ages on the AWS provider and it is so much better.
This functionality is extremely valuable and something we have successfully utilized in AWS at a previous company. It automatically applies tags to every resource that supports tagging, generated by the provider. This feature is particularly beneficial when working with complex module hierarchies (modules within modules). By specifying tags in a single location, you streamline the tagging process across all nested resources. Having a similar capability in AzureRM would be incredibly useful and enhance resource management significantly.
Same here, we are using the feature since years! for AWS. This issue is now open for three years.
Also looking for that
Bump. Need this for cloud-agnostic tag management.
Also looking for that
Could you please just add your thumbs-up to the top post, like the other 326 people did, so not everybody subscribed to this issue gets spammed with irrelevant emails? Thanks!
Community Note
Description
Investigate adding default values for location & resource_group to the provider block. This might also include tags depending on how feasible it is given the varying tag limitations of different resources/apis.
New or Affected Resource(s)
Potential Terraform Configuration
comments on the potential schema for this more then welcome!