What Should We Do With Terraform Cloud Integrations?

[Magnitus-]

In the process of scrubbing "Terraform" and "Hashicorp" out of the code base at least for areas that end users will see for trademark reasons, it has come to my attention that the terraform code has a lot of terraform cloud integration baked into it.

For my part, I just conservatively left it there, but I think we'll have to make a conscious decision about this.

Given that my workplace doesn't use Terraform Cloud, its easy for me to say "lets rip it out and it will make the code simpler", but I'm guessing a decent number of Terraform users are using Terraform Cloud and the set of people that would want to use OpenTF AND Terraform Cloud might not be empty (I have no idea how large it is).

However, if we keep Terraform Cloud into the code, Hashicorp won't maintain it as Terraform diverges from the OpenTF repo and we would have to shoulder the constant reconciliation of that divergence ourselves.

Alternatively, something we could try to do is remove Terraform Cloud integration from the repo and optionally support it as a separate plugin if we need (I'm naively guessing that is possible, though it would require significant architectural work and either way I think it is safe to guess that Hashicorp would not join the party in maintaining Terraform Cloud as a separate plugin either even if they would potentially have a commercial interest in it).

[anton-yurchenko]

I bet people relying on the TF Cloud will stick with HashiCorp Terraform rather than OpenTF because of compatibility and support. Moreover, keeping Terraform Cloud integration will primarily benefit HashiCorp and allow it to monetize the community work.

[Magnitus-]

Fair point, we should strive to make the project vendor-neutral with support for plugin integration that vendors (and free solutions) can take advantage of to integrate services.

It would set the right tone (neutral) for the project and make it less cluttered.

[tedivm]

If we rip the cloud part out then it will fragment the community. There are a lot of Terraform Cloud users, and we should want them to be able to use the open source version. Maintaining that functionality will make it easier for people to migrate to the fork over time. If we leave it in then people won't have to immediately "pick a side" (which would also mean migrate to a different TACOS for people on Terraform Cloud), and since status quo tends to win out I think it would really harm this project.

I also think that the "cloud" protocol itself shouldn't be specific to Terraform Cloud. If the protocol has benefits it should be opened up so other TACOS can also use that same cloud block. Although Hashicorp could in theory introduce regular backwards compatibility breaks, the reality is that would create a negative experience for their own users as they get locked into smaller version windows. So they have an incentive to have a stable protocol as well, which should make it easier for this project to keep working with terraform cloud.

[marcinwyszynski]

The current Terraform Cloud backend is used by at least some of the vendors offering similar services. We at Spacelift took a different path, but I know some of the other vendors chose to implement it in their offerings. So that's one thing. Another thing is that dropping the support for it would suddenly make OpenTF incompatible with legacy Terraform, so I think the best course of action will be to treat it as a "cloud backend" - a brand-agnostic API that anyone can implement if they so wish.

[i4ki]

In Terraform there are two similar concepts called TFC (Terraform Cloud) and TFE (Terraform Enterprise) where both implement the same API but TFC is the official SaaS cloud whereas TFE is the on-premise.

The TFC/TFE API implements the .well-known URI concept, so which means the client can discovery which services the cloud implements. This is the services exposed by the TFC: https://app.terraform.io/.well-known/terraform.json

For integrating with TFE, the user can provide an alternative base URL, for example:

terraform login my.cloud.example.org

and if the domain implements the TFE API, Terraform just works.

The Terraform codebase makes use of the go-tfe library which abstracts the cloud type, so in principle, it could work with any third-party clouds. Of course, TFC/TFE are Hashicorp products and probably there are important legal matters in this case. Still, maybe an interesting alternative would be if OpenTF also documents a community cloud API (which initially can be compatible with the MPL go-tfe SDK). Then we can keep all the cloud-related code in the fork but disable/remove the use of the TFC domain (Hashicorp default API domain). Have a look here.

Then all the Terraform Cloud customers can switch to opentf fork and the only change is that they will need to configure the Hashicorp domain (as people using TFE do).

[marcinwyszynski]

TL;DR is that TFC users should still be able to use OpenTF as a drop-in replacement.

FYI I was handling that in this PR. This is a name change TBH, the remote backend remained unchanged, and the cloud backend uses vendor-agnostic language with no change in the functionality.

Removing TFC as the default for terraform login could make sense, though, because it gives privilege to a particular vendor.