Strategy for 1.0 and beta API features

manicminer commented 2 years ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritise this request
Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritise the request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

Work out a strategy for managing v1.0 versus beta features for resources. Where an API supports simultaneous access via both APIs, we should be able to support individual beta features/properties and document them as such. For some resources (e.g. conditional access), this will not be possible as the use of beta features precludes using the v1.0 API.

For beta-only properties, this could potentially be implicit (e.g. if you specify the property, we'll use the beta API)
For beta-only attributes (on resources and data sources), this may need to be a feature flag to explicitly populate those attributes via the beta API
- We already populate some beta attributes, e.g. azuread_user.onpremises_domain_name - how to handle these?
Beta-only resources could also potentially be implicit
If we gate beta-only attributes behind a feature flag, it likely makes sense to force the same flag for beta-only resources/properties too
- Alternatively, do the reverse, and have a feature flag for disabling beta features which seems more user friendly

New or Affected Resource(s)

All resources and data sources

Potential Terraform Configuration

# Opt-in feature flag
provider "azuread" {
  use_beta_features = true
}

# Opt-out feature flag
provider "auread" {
  disable_beta_features = true
  # or env var: AAD_DISABLE_BETA_FEATURES=true
}

# Implicit beta-only property
resource "azuread_service_principal" "example" {
  application_id = "00000000..."
  saml_metadata_url = "https://example.com/path"
}

References

None

mariussm commented 2 years ago

I would suggest that explicitly choosing to use beta features is the best way to go, following Microsoft's recommendation on not using the beta endpoint for production stuff by default.

What I normally do when using a beta endpoint, is that 90% of the code using v1.0, while only the endpoints that require beta is actually using the beta endpoints. And then, when the beta endpoints are made available in v1.0, I switch in the code from beta to v1.0. I would like to do the same thing in Terraform, by enabling only certain resources to use the beta endpoint, moving to v1.0 when ready. Having a provider wide "use beta" configuration, would mean that all of the 90% that could use v1.0, will be using beta too. This is way more prone to breaking as far as I see it. Could this be solved by using provider alias like the following, possibly not causing the resource to be recreated?

provider "azuread" {

}

provider "azuread" {
  use_beta_features = true
  alias = "beta"
}

# SP using the beta endpoint
resource "azuread_service_principal" "example" {
  application_id = "00000000..."
  saml_metadata_url = "https://example.com/path"

  provider = azuread.beta
}

# Group not using the beta endpoint
resource "azuread_group" "example" {
  display_name     = "example"
  mail_enabled     = true
  mail_nickname    = "ExampleGroup"
  security_enabled = true
  types            = ["Unified"]
}

manicminer commented 2 years ago

@mariussm Thanks for weighing in! That's pretty much my thinking that 1.0 features should use the 1.0 endpoint regardless of whether beta features are enabled, but beta features should only work when a user specifically opts in. I think this would satisfy a broad range of requirements and give confidence to users who must use 1.0 exclusively for internal compliance.

For stability reasons, it's unlikely that we'd manage 1.0 features via the beta endpoint when beta features are specifically opted in. Those would continue to use the respective 1.0 endpoint and, as you describe, only the code paths for beta features would be different. I say unlikely because a hard rule could lead to compatibility problems (as in the case of conditional access policies) - for these we'd have to approach it in a way that makes sense for that resource and we'd seek to clearly document the behavior.

Unfortunately I don't think relying on provider aliases would scale well. In v2.0 of the provider we largely moved away from computed fields to a model of explicit management, so attempting to manage the same underlying resource with two provider blocks would present a conflict (even if we worked around the aspect of non-uniqueness in display names).

In short, my (unproven) suggested approach at this time would look something like this:

provider "azuread" {
  use_beta_features = true
}

resource "azuread_fictional" "foo" {
  stable_property_one = "bar"
  stable_property_two = true
  beta_property = "blah"
}

... where we'd create the resource using the 1.0 endpoint and set the stable properties, then subsequently patch the resource using the beta endpoint to set the final property. If use_beta_features is unset/false, we'd throw an error at plan time if a beta property was detected in a configuration.

WDYT?

kaovd commented 2 years ago

Sounds good!

Guessing in order to make use of this feature imagine upstream enums would have to be both stable and beta enums to avoid duplicating resources. Downstream depending on toggle to have a param option in the go whether certain blocks are beta or not and ignore elsewise? If not we'd have latest table and beta go files downstream - maybe that would be easier than trying to fit it all into one file?

hashicorp / terraform-provider-azuread