Docs: describe provider aliases in the JSON format

izeau commented 4 years ago

Terraform Version

terraform v0.12.24

Terraform Configuration Files

{
  "provider": {
    "local": {},
    "local.x": {
      "alias": "x"
    }
  }
}

Debug Output

https://gist.github.com/izeau/49d383db7ac598bd3b33ef804dd16561

Expected Behavior

This used to work fine in 0.11. The JSON format surely has changed, but the spec link given in the docs is broken and the documentation does not give an example of aliased providers.

Edit: found the spec at https://github.com/hashicorp/hcl/blob/hcl2/json/spec.md

Actual Behavior

Error: Failed to instantiate provider "local.x" to obtain schema: unknown provider "local.x"

Steps to Reproduce

cd $(mktemp -d)
cat > provider.tf.json <<EOF
{
  "provider": {
    "local": {
    },
    "local.x": {
      "alias": "x"
    }
  }
}
EOF
terraform init
terraform plan

Additional Context

We are migrating our Terrafrom 0.11 environment to Terraform 0.12 (better late than never, I guess). Our provider.tf.json files are generated by a script, and using HCL would be manageable but more work.

izeau commented 4 years ago

So after reading the spec and some fiddling, we managed to find a working solution:

{
  "provider": {
    "local": [
      {},
      {"alias": "x"}
    ]
  }
}

Still I’d like to keep this issue open to track that the docs may need an upgrade.

danieldreier commented 4 years ago

@izeau are you open to making a PR to fix the docs? It sounds like you've found the answer

apparentlymart commented 4 years ago

Thanks for reporting this, @izeau! I'm glad you figured out the correct syntax, and sorry for the unintended compatibility break here.

I'm honestly rather baffled as to how the syntax you had before was working in Terraform 0.11, but I believe you! That JSON would've been equivalent to the following in native syntax, which was never intentionally supported:

provider "local" {
}
provider "local.x" {
  alias = "x"
}

I guess we can chalk it up to another oddity in the old HCL JSON implementation, which had plenty of bugs and quirky behaviors. (That's why HCL 2, which Terraform 0.12 and later use, has an entirely new implementation which takes a different design approach.)

I think the tricky thing about updating the docs here is that this seems to have stemmed from a misunderstanding about the Terraform 0.11 syntax that was reinforced by a Terraform 0.11 parsing bug failing to validate it.

Technically the information about the correct syntax is already in the docs, but it's split into two parts due to how the documentation is organized:

The alias argument documentation shows a native syntax example where both of the blocks have provider "aws", and the aliased one isn't provider "aws.west".
The JSON syntax page then includes an example of an aliased provider in JSON syntax which shows almost a JSON equivalent of the second block in the alias argument docs, albeit with the alias set to "usw1" instead of "west", probably because these examples were written totally independently of one another.

When I think about how to organize docs I try to think about what mindset the reader is in when reading it. The current organization is assuming that folks will first learn about provider aliases in the native syntax, and then maybe at some later point they'll want to convert their native syntax configuration into JSON and so will want to learn how to systematically convert all of the features into JSON.

It seems unlikely to me that there would be many other users who had inadvertently discovered this unintended alternative syntax in Terraform 0.11 and so would now be looking for docs on how to fix it for 0.12, so I wouldn't want to approach the doc updates from that perspective, but I think it's reasonable to want to quickly find the JSON equivalent of any specific native-syntax example you've seen in the docs.

Some time ago we'd prototyped a way to include a Native vs. JSON language switcher on most examples in the documentation as a generalized solution to this, but it was depending on some web framework improvements that haven't made it to the Terraform website yet, because the Terraform website is still on our legacy web stack. It feels kinda awkward to include both native and JSON configuration examples on the main docs page only for the case of aliased providers... it would then raise the question as to why all of the other docs aren't like that, but for many of the larger examples having two syntaxes without the UI element to present them compactly would be too unwieldy and would make the content harder to read.

So with all of that said ... I feel a little unsure as to what short-term action we could take here. Long term I think the answer is to have a syntax switch on all of the examples in our docs, but the web framework updates are outside of our team's scope and, as far as I know, not likely to happen in the very near future. My instinct then is to close this specific issue for now (because it isn't actionable in isolation), and plan to address this problem in a more general way once our website has caught up with the web framework improvements that are gradually making their way across all of the HashiCorp open source projects.

izeau commented 4 years ago

Hey @apparentlymart! It’s alright, don’t feel bad: Terraform is a terrific product!

I’m not sure an HCL2 / JSON language switcher on the website would be that useful – I may be biased here but I don’t think most users of Terraform are even aware of the JSON support. There would be a need to translate a large portion of the docs for little to no added value. As a matter of fact, our team only use it because we run a “dependency update” script twice a month to make sure we install the latest versions of the providers in our CI pipeline; and while HCL2 is great for configuration, JSON was selected for this job since reading and writing in JSON typically doesn’t require the installation of third-party libraries.

I think a “missing” example in the JSON Configuration Syntax page would be one that shows how to have multiple blocks with the same type / name. There is an example in the HCL2 JSON spec, although the link in the Terraform docs is broken (it has an extra /hcl/ path in the URL that may refer to an older directory structure).

Anyway, since provider aliasing is relatively common – and I believe most providers only require aliases for “duplicate” providers – maybe the provider section example in the JSON Configuration Syntax page can be modified to include multiple providers:

{
  "provider": {
    "aws": [
      {
        "region": "us-east-1"
      },
      {
        "alias": "west",
        "region": "us-west-1"
      }
    ]
  }
}

@danieldreier I’d be more than happy to contribute!

izeau commented 4 years ago

The broken link issue has already been fixed in #24630.

ghost commented 4 years ago

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

hashicorp / terraform