Dedicated GitLab State Backend Provider #29444

Open ChevronTango opened 3 years ago

Current Terraform Version

v1.0.3

Use-cases

It would be really good for users who have their state stored in GitLab.com or a Self Hosted GitLab instance to be able to store their state properly with a dedicated provider, that works well with workspaces and other features that GitLab can support

Attempted Solutions

Currently users wanting to make use of GitLab for state storage can do so via the HTTP backend however HTTP does not support elements like workspaces. HTTP is also quite verbose, requiring all of the endpoint to be specified.

terraform {
  backend "http" {
    address="https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-STATE-NAME>"
    lock_address="https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-STATE-NAME>/lock"
    unlock_address="https://gitlab.com/api/v4/projects/<YOUR-PROJECT-ID>/terraform/state/<YOUR-STATE-NAME>/lock"
    username="<YOUR-USERNAME>
    password="<YOUR-ACCESS-TOKEN>" 
    lock_method="POST" 
    unlock_method="DELETE" 
    retry_wait_min="5"
  }
}

Since variables cannot be specified in backend config, a user cannot use ${terraform.workspace} to get around this shortcoming. Instead a user must call terraform init with a set of overridden variables for the backend addresses, which is verbose and cumbersome.

Proposal

A dedicated GitLab backend provider for Terraform that reduces the amount of config required to connect and use GitLab as a state storage

terraform {
  backend "gitlab" {
    address="https://gitlab.com/api/v4/" # Optional - Required for Self Hosted GitLab instances
    project_id="<YOUR-PROJECT-ID>"
    username="<YOUR-USERNAME>
    password="<YOUR-ACCESS-TOKEN>" 
  }
}

Users can then use workspaces appropriately for their project and they will be added in place of <YOUR-STATE-NAME>

This would also allow for some element of nice error reporting, such as when a project doesn't exist, or a user does not have appropriately elevated permissions on it.

References

https://docs.gitlab.com/ee/user/infrastructure/terraform_state.html

This issue has already been discussed in the following:

The conclusion from what I can gather is that Hashicorp is planning to split the backend part into separate plugins, like the providers, and until that point all development for adding other backends is blocked.

I haven't found where this splitting work is being tracked so we can follow. @ash2k can you please help us with the tracking ticket?

@madpipeline If you are after ticket tracking work on the GitLab side, then it's the one you linked above (https://gitlab.com/gitlab-org/gitlab/-/issues/207348). If you are asking about the work Hashicorp wants to do to extract backends, then I have no idea, sorry. Perhaps someone from Hashicorp could clarify.

Looks like they are completely ignoring all backends-related PRs at the moment - https://github.com/hashicorp/terraform/pull/24639

hashicorp / terraform