hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.06k stars 9.47k forks source link

Ability to create custom types. #27365

Open adamdonahue opened 3 years ago

adamdonahue commented 3 years ago

Instead of something like this:

module-inputs.tf

variable "configuration" {
  type = object({
      x = string
      y = string
      z = list(string)
   })
}

infra-inputs.tf

variable "named-configurations" {
  type = map(
    object({
      x = string
      y = string
      z = list(string)
   })
  )
}

which, here and especially for multiple top-level modules, involves a ridiculous amount of boilerplate, we'd like to see something along the lines of:

types.tf

type "configuration" {
  definition = map(
    object({
      x = string
      y = string
      z = list(string)
   })
  )
} 

and then

module-inputs.tf

variable "configurtion" {
  type = types.configuration
}

infra-inputs.tf

variable "named-configurations" {
  type = map(types.configuration)
}
adamdonahue commented 3 years ago

Just wanted to check on the status of this -- it's very useful.

It'd actually be useful to somehow be able to export a type definition from a module, as well. So we can define the input type(s) for a module and reuse them in other places.

JohannesMoersch commented 3 years ago

Has there been any discussion or consideration of this enhancement? I think this would be super valuable, and would be one of the best things that could be done to improve consistency and code quality within terraform.

binte commented 2 years ago

This would indeed be very helpful. Any chance we might be able to see this at some point?

juarezr commented 2 years ago

Complementing this feature request, it would be pretty useful to also have the same features as input variable in custom type definitions:

Below an example:

type "db_password" {  
  description = "Database administrator password"
  type        = string  
  sensitive   = true
}

type "timestamp" {
  description = "Example of how to validate a type a formula defined by an expression"
  type        = string
  validation {
    condition     = can(formatdate("", type.timestamp))
    error_message = "value for timestamp type requires a valid RFC 3339 timestamp"
  }
  default = timestamp()
}

type "result" {
  description = "Example of validation for enum/option type"
  type        = string
  validation {
    condition = anytrue([
      type.result == "good",
      type.result == "bad",
      type.result == "ugly"
    ])
    error_message = "value for variable of type result must be 'good', 'bad', or 'ugly'"
  }
  default = "good"
}

local {
  ip_address_regex = "^(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)$"
}

type "ip_address" {
  description = "Example of using a local constant for type validation"
  type = string
  validation {
   condition = can(regex(local.ip_address_regex, type.value))
        error_message = "Invalid IP address provided."
  }
}

type "configuration" {
  description = "Example of using other types for type definition"
  type = map(
    object({
      x = type.timestamp
      y = type.result
      z = list(type.ip_address)
   })
  )
}

variable "configuration" {
  type = type.configuration
}

variable "multiple-configurations" {
  type = list(type.configuration)
}
Vetrenik commented 2 years ago

Truely interested in this feature!

julienbeaussier commented 2 years ago

That feature would be indeed very useful and a great complement of this optional var improvement coming in v1.3.

ImadYIdrissi commented 1 year ago

I adhere to the declarative paradigm when it is available, because it adds visibility and versionning to previously unused branches (by me at least) such as infrastructure build and evolution, and it also describes "what we want" instead of "how we obtain" it... Unfortunately, it is still a pain today as many basic coding tools that exist in the imperative paradigm are quite far behind in Terraform.

I'm quite new to contributing to opensource, I've seen in the contribution guide for Terraform that a new feature must be discussed and receive feedback before implementations are even considered. I'd say this issue is it, and would love to dip my toe in building a solution. As such, I'd love some pointers on where to start.

Kudos for this feature request.

icicimov commented 1 year ago

I wished this existed more than once in the past. Every time a have to declare multiple variables of the same structure I wish I could declare a custom type once and use it multiple times declaratively instead of replicating the same structure over and over again in each var.

JonRoma commented 1 year ago

This would be an incredibly useful feature, particularly if there is a mechanism to reuse definitions of complex types across multiple Terraform modules.

If this goes forward, please avoid repeating the moderately annoying fandango with inconsistent nameing – e.g., the locals {} blocks are declared as a plural, while references to local variable uses the singular form as local.foo.

bwinter commented 1 year ago

It would also be useful to specify another module as a type. So that a sub field of your config could be held to that modules interface, rather than having to redefine it. e.g.

module/infra.tf

variable "configuration" {
  type = object({
      x = string
      y = string
      z = list(string)
   })
}

module/consumer.tf

variable "configurations" {
  type = object({
    clusterName = string
    configs = list(module.infra)
  }
}

This way you could have nested relationships / dependencies; but, wouldn't have to redefine the interface to ensure the correct variables were passed.

dylanturn commented 1 year ago

#

It would also be useful to specify another module as a type. So that a sub field of your config could be held to that modules interface, rather than having to redefine it. e.g.

module/infra.tf

variable "configuration" {
  type = object({
      x = string
      y = string
      z = list(string)
   })
}

module/consumer.tf

variable "configurations" {
  type = object({
    clusterName = string
    configs = list(module.infra)
  }
}

This way you could have nested relationships / dependencies; but, wouldn't have to redefine the interface to ensure the correct variables were passed.

I like the idea of some of the more complex suggestions, but honestly even just this would be nice. Often times simple resource modules are used in more complex module compositions and being able to pass complex variable types forward would make life so much easier.

I think a great example of this is found in the AWS EKS module. As it currently stands the terraform-aws-eks module defines a variable of eks_managed_node_groups with a type of any. This means that when you're using your IDE to try deploy an EKS cluster the Terraform language server is powerless to give you any suggestions. Being able to pass the entirety of the eks_managed_node_group as a type would be a HUGE improvement to the developer trying to use it.

Example of what it could look like:

module "eks_managed_node_group" {
  source = "./modules/eks-managed-node-group"

  for_each = { for k, v in var.eks_managed_node_groups : k => v if var.create && !local.create_outposts_local_cluster }

  ... rest of the config ...
}

variable "eks_managed_node_groups" {
  description = "Map of EKS managed node group definitions to create"
  type        = map(module.eks_managed_node_group)
  default     = {}
}

Additionally, what would also be nice is something like a data source that could be used to lookup and reference a non-local modules variable type. Something like this:

# This data source gets the source modules variable types.
data "terraform_remote_types" "example_module" {
  source  = "terraform-aws-modules/eks/aws"
  version = "~> 19.0"
}

variable "module_as_variable" {
  type = data.terraform_remote_types.example_module
}

Building on this idea, perhaps we could even reference the type of a single module variable too:

variable "module_subset_as_variable" {
  type = data.terraform_remote_types.example_module.self_managed_node_group_defaults
}
maroux commented 1 year ago

One workaround I've found for this:

variables.tf

variable "config_a" {}

typed_variables.tf

module "config_a" {
  source = "./modules/type-config"
  input    = var.config_a
}

./modules/type-config/variables.tf

variable "input" {
  type = object({
      x = string
      y = string
      z = list(string)
  })
}

./modules/type-config/outputs.tf

output "value" {
  value = var.input
}

Now you can use module.config_a.value as a typed value.

juarezr commented 1 year ago

One workaround I've found for this: ...

Now you can use module.config_a.value as a typed value.

Nice! I still miss a shared type for variables instead of workarounds. Maybe someday.

Another (ugly) workaround is using symlinks for variable definitions files:

Pros:

Cons:

Example:

In ../shared/vars/vars-env.tf

variable "env" {
  description = "Environment Variables"
  type = object({
    name    = string
    project = string
    region  = string
    zone    = string
    tags = map(string)
  })

In ../shared/config/vars-env-dev.tfvars

env = {
  name    = "dev"
  project = "POC-project-123456"
  region  = "us-east4"
  zone    = "us-east4-a"
  tags = {
    "costs": "owned",
    "env": "dev",
  }
}

In ../shared/config/tfstate-dev.tfbackend one can put the remote state configuration like:

bucket  = "tfstate-dev"

In ../tf/module1/tf-providers-module1.tf one can use the var.env reference:


terraform {
  required_version = ">= 1.3.3, < 1.5.0"
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = ">= 4.61.0, < 5.0.0"
    }
  }
}

provider "google" {
  project = var.env.project
  region  = var.env.region
  zone    = var.env.zone
}

terraform {
  backend "gcs" {
    prefix = "tf/module1"
  }
}

In ./tf/module1/ one should run the following commands to link to the shared files:

$ ls -s ../../shared/vars/env.tf env.tf
$ ls -s ../../shared/config config

After this, one can run terraform as usual. In ./tf/module1/ it would be:

$ terraform init -backend-config=./config/tfstate-dev.tfbackend
$ terraform plan  -var-file=./config/vars-env-aws-dev.tfvars

TLDR: A lot of effort to share some types.

aaziz993 commented 3 months ago

Are there any official solutions about this issue nowadays.