Allow specifying "project" property in the "cloud" block

[aaabramov]

Terraform Version

Terraform v1.3.7
on darwin_amd64

Use Cases

It would be awesome to allow specifying "project" property in the "cloud" block. This way, cloud workspace will be created in the proper from the beginning. 

Now if you run "terraform init" it will be added to the "Default" project which could temporarily expose workspace data to other cloud users.

Attempted Solutions

terraform {
  cloud {
    organization = "my-org"
    project = "my-project"
    workspaces {
      name = "my-workspace"
    }
  }
}

Produces:

Initializing Terraform Cloud...
╷
│ Error: Unsupported argument
│
│   on main.tf line 5, in terraform:
│    5:     project = "my-project"
│
│ An argument named "project" is not expected here.

Proposal

For example, "project" property could be added to the "cloud" block like this:

terraform {
  cloud {
    organization = "my-org"
    project = "my-project"
    workspaces {
      name = "my-workspace"
    }
  }
}

It is clear what to do with the first run. But there is an open question: "What to do if "project" will be updated/deleted after some time? Should we update it? Or ignore consecutive changes?"

References

No response

P.S. I tried submitting this suggestion to https://support.hashicorp.com/hc/en-us/requests but seems that it fails to file a ticket for some reason.

[crw]

Thanks for this request!

[apparentlymart]

Hi @aaabramov! Thanks indeed for this feature request.

I understand from your request that you are primarily concerned with having terraform workspace new always create new workspaces with the designated project, and that does make sense to me.

I'm curious as to whether you'd also expect project to act as a filter to exclude any workspaces that are not already part of the designated project.

Of course in your example using the name argument inside workspaces that is a strange question because that one workspace is either in the project or not, but the workspaces block also supports a tags argument to select potentially many different workspaces that all have the specified tags. I'm wondering if you'd expect project to also exclude from the set of active workspaces any workspace that does match the tags but doesn't match the project.

If it were to act as a workspace filter then perhaps it would be more consistent for the argument to appear inside the workspaces block, to make it explicit that it's acting as another constraint on the selected workspaces:

terraform {
  cloud {
    organization = "my-org"
    workspaces {
      name    = "my-workspace"
      project = "my-project"
    }
  }
}

terraform {
  cloud {
    organization = "my-org"
    workspaces {
      tags    = ["a", "b"]
      project = "my-project"
    }
  }
}

In the first case I imagine Terraform would return an error if my-workspace already exists and doesn't already belong to my-project, but if the workspace didn't already exist then it would create it in that project.

In the second case I imagine it excluding any workspace that doesn't belong to my-project even if it matches the tags.

In both cases it would effectively exclude any workspace that isn't part of the designated project from being used with the current configuration.

Does that seem like intuitive/useful behavior to you?

Thanks again!

[aaabramov]

Hi @apparentlymart. Sorry for the late reply.

Yes, I agree that placing the project property in the workspaces block would be logical.

Here is an example how it may work:

Case 1

# 1. No workspace existed previously in TF cloud:
# 2. Declare backend
terraform {
  cloud {
    organization = "my-org"
    workspaces {
      name    = "my-workspace"
      project = "my-project"
    }
  }
}

$ terraform init
# Created new "my-workspace" workspace in "my-project" project

Case 2

# 1. Workspace already exists in TF cloud (e.g. this is done by new developer or CI):
# 2. Declare backend
terraform {
  cloud {
    organization = "my-org"
    workspaces {
      name    = "my-workspace"
      project = "my-project"
    }
  }
}

$ terraform init
# Pulled "my-workspace" workspace from "my-project" project

P.S.

I migrated my projects to use backend "remote" simply because it uses more intuitive workspaces (dev/stage/prod in my case). I know that "remote" backend is about to be removed but for now it works just fine. And I would expect something like this to work also:

terraform {
  backend "remote" {
    organization = "my-org"
    workspaces {
      project = "services" # this would fetch & create new cli workspaces (dev/stage/prod/etc.) in "services" project
      prefix = "user-service_" # this creates user-service_dev, user-service_stage, user-service_prod for me, but currently only in the Default projects.
    }
  }
}

P.P.S

Moreover, I am currently pre-bootstrapping my workspaces also with terraform. But resource "tfe_workspace" also does no support specifying projects.

locals {
  services = [
    "service-1",
    "service-2",
    "service-3",
  ]
  service_envs = [
    "dev",
    "stage",
    "prod"
  ]

  workspaces = [
    for pair in setproduct(local.services, local.service_envs) : {
      service = pair[0]
      env     = pair[1]
    }
  ]
}

resource "tfe_workspace" "my-workspaces" {
  organization = "my-org"

  for_each = {for pair in local.workspaces : "${pair.service}_${pair.env}" => pair}

  name           = "${each.value.service}_${each.value.env}"
  execution_mode = "local" # This is the main reason, I want terraform to execute locally!
  tag_names      = [each.value.env]
  # And again, no way to specify project :(
}

[apparentlymart]

Thanks for that additional context, @aaabramov! I expect that all of this will be helpful to the Terraform Cloud teams that will ultimately be considering this request.

I believe there's already been some work to make projects available in the hashicorp/tfe provider; I'd suggest opening an issue in that provider's repository to capture what you're looking for there, since that'll make the need more visible to the team that maintains that provider.

[omri-shilton]

any news of this request? out team would really benefit from it. when running in AWS and working with dynamic credentials the workspace must be attached to the project that handles its credentials to AWS. so we need to manually attach the workspace to the project each time we create a new aws service (our directories/workspaces are separated per service as recommended by Hashicorp).

[crw]

~@omri-shilton no update, but this also is not the team that would be working on it. Since this is concerning a feature of the paid product, I'd suggest boosting it through the support channels e.g. please email tf-cloud@hashicorp.support or open a new request. Thanks!~

[radditude]

I have good news! This work is in progress and is currently anticipated to be a part of the 1.6 release.

[gurretl]

Hi @aaabramov, It seems supported now : https://github.com/hashicorp/terraform/pull/33489 Just use the environment variable : TF_CLOUD_PROJECT

[bdorplatt]

It appears this was released as part of the 1.6.0 release, however, it doesn't seem to work quite right in our case. We are doing the following in a backend.tf that is created by Terragrunt. The workspace is getting created but the project isn't applied to the workspace in Terraform Cloud and shows up as just "Default Project" Is there an equivalent command to set the project name when using "backend "remote" { in a terraform block?

terraform {
  cloud {
    hostname     = "app.terraform.io"
    organization = "XXXXXX"
    workspaces {
      project = "Test"
      name = "azure-${local.tenant}-${local.region_abbr}-${local.subscription}-${local.application}-${replace(path_relative_to_include(), "/", "-")}"
    }
  } 
} 

[rmcolbert]

@bdorplatt - Your use case appears to work in 1.6.6. I just tested and my new workspace was created in the correct project.

It would be nice if project could make tags completely optional in the workspaces block though as I'd like to be able to create a new workspace from the CLI w/o having to also bind tags to it and just leverage the project filter.

terraform {
  cloud {
    organization = "XXXXXX"
    workspaces {
      project = "Test"
    }
  } 
} 

We segment workspaces by projects instead of tags and with the project option in the cloud config, it almost meets our use case but forcing name or tags is still restrictive. It would be better if it were project or name or tags. I have tested this w/ 1.6.6, 1.7.5, 1.8.5 and 1.9.0.