search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io
Other
43.16k stars 9.58k forks source link

Provider installations fails when a provider has a non-zip archive in the release artifacts #26282

Open louy opened 4 years ago

louy commented 4 years ago

TL;DR: I made a terraform provider with the exact same config suggested (github actions & goreleaser config) except for a minor change: I told goreleaser to create a tar.gz archive as well as a zip one

TF then tried to install the tar.gz and run unzip on it, failing to do so and leaving the user in a broken state where TF thinks the provider is installed but it wasn’t

Terraform Version

Terraform v0.13.3

Terraform Configuration Files


terraform {
  required_providers {
    uptimerobot = {
      source = "louy/uptimerobot"
      version = "0.5.0"
    }
  }
}
provider "uptimerobot" {}
data "uptimerobot_account" "account" {}

Debug Output

Fresh setup:

 TF_LOG=trace ./terraform init
2020/09/17 13:46:29 [INFO] Terraform version: 0.13.3  
2020/09/17 13:46:29 [INFO] Go runtime version: go1.14.7
2020/09/17 13:46:29 [INFO] CLI args: []string{"/home/runner/terraform-provider-uptimerobot/terraform/terraform", "init"}
2020/09/17 13:46:29 [DEBUG] Attempting to open CLI config file: /home/runner/.terraformrc
2020/09/17 13:46:29 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/09/17 13:46:29 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/09/17 13:46:29 [DEBUG] ignoring non-existing provider search directory /home/runner/.terraform.d/plugins
2020/09/17 13:46:29 [DEBUG] ignoring non-existing provider search directory /home/runner/.local/share/terraform/plugins
2020/09/17 13:46:29 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2020/09/17 13:46:29 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2020/09/17 13:46:29 [INFO] CLI command args: []string{"init"}
2020/09/17 13:46:29 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2020/09/17 13:46:29 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/09/17 13:46:29 [DEBUG] New state was assigned lineage "f791a7b0-72d0-6c8a-ac34-a5787de8b673"
2020/09/17 13:46:29 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2020/09/17 13:46:29 [TRACE] Meta.Backend: instantiated backend of type <nil>
2020/09/17 13:46:29 [DEBUG] checking for provisioner in "."
2020/09/17 13:46:29 [DEBUG] checking for provisioner in "/home/runner/terraform-provider-uptimerobot/terraform"
2020/09/17 13:46:29 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2020/09/17 13:46:29 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend

Initializing the backend...

Initializing provider plugins...
2020/09/17 13:46:29 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2020/09/17 13:46:29 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2020/09/17 13:46:29 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2020/09/17 13:46:29 [TRACE] statemgr.Filesystem: read nil snapshot
2020/09/17 13:46:29 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/09/17 13:46:29 [TRACE] getproviders.SearchLocalDirectory: failed to resolve symlinks for .terraform/plugins: lstat .terraform: no such file or directory
2020/09/17 13:46:29 [TRACE] providercache.fillMetaCache: error while scanning directory .terraform/plugins: cannot search .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/09/17 13:46:29 [WARN] Failed to scan provider cache directory .terraform/plugins: cannot search .terraform/plugins: lstat .terraform/plugins: no such file or directory
2020/09/17 13:46:29 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
2020/09/17 13:46:29 [TRACE] HTTP client GET request to https://registry.terraform.io/.well-known/terraform.json
- Finding louy/uptimerobot versions matching "0.5.0"...
2020/09/17 13:46:30 [DEBUG] GET https://registry.terraform.io/v1/providers/louy/uptimerobot/versions
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/louy/uptimerobot/versions
2020/09/17 13:46:30 [DEBUG] GET https://registry.terraform.io/v1/providers/louy/uptimerobot/0.5.0/download/linux/amd64
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://registry.terraform.io/v1/providers/louy/uptimerobot/0.5.0/download/linux/amd64
2020/09/17 13:46:30 [DEBUG] GET https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_SHA256SUMS
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_SHA256SUMS
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://github-production-release-asset-2e65be.s3.amazonaws.com/135330373/7e1cd200-eff7-11ea-9b60-34d77a9aef15?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200917%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200917T134524Z&X-Amz-Expires=300&X-Amz-Signature=9823ae5ae8c6d9f57d25aa263cc90f85db18d6e93f9989b2c80c45ef999141a1&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=135330373&response-content-disposition=attachment%3B%20filename%3Dterraform-provider-uptimerobot_0.5.0_SHA256SUMS&response-content-type=application%2Foctet-stream
2020/09/17 13:46:30 [DEBUG] GET https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_SHA256SUMS.sig
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_SHA256SUMS.sig
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://github-production-release-asset-2e65be.s3.amazonaws.com/135330373/7c530e80-eff7-11ea-8cae-c080650d7ff4?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200917%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200917T134524Z&X-Amz-Expires=300&X-Amz-Signature=04e98ef8bde6f923fe1a1c67fa76f43157c8c7a9643e8fd8178a9bbf49bf97c2&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=135330373&response-content-disposition=attachment%3B%20filename%3Dterraform-provider-uptimerobot_0.5.0_SHA256SUMS.sig&response-content-type=application%2Foctet-stream
2020/09/17 13:46:30 [TRACE] providercache.Dir.InstallPackage: installing registry.terraform.io/louy/uptimerobot v0.5.0 from https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_linux_amd64.tar.gz
- Installing louy/uptimerobot v0.5.0...
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_linux_amd64.tar.gz
2020/09/17 13:46:30 [TRACE] HTTP client GET request to https://github-production-release-asset-2e65be.s3.amazonaws.com/135330373/7ceba500-eff7-11ea-9229-4535aeb915c5?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200917%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200917T134630Z&X-Amz-Expires=300&X-Amz-Signature=52502711cfee48892a24a3659f6ab0c73353033b5f704eda173fd44bb75aed37&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=135330373&response-content-disposition=attachment%3B%20filename%3Dterraform-provider-uptimerobot_0.5.0_linux_amd64.tar.gz&response-content-type=application%2Foctet-stream
2020/09/17 13:46:31 [DEBUG] Provider signed by 2CC2FD55D1C66DF3 Louay Alakkad <louay@alakkad.me>

Error: Failed to install provider

Error while installing louy/uptimerobot v0.5.0: zip: not a valid zip file

Subsequent tries:

 TF_LOG=trace ./terraform init
2020/09/17 13:48:24 [INFO] Terraform version: 0.13.3  
2020/09/17 13:48:24 [INFO] Go runtime version: go1.14.7
2020/09/17 13:48:24 [INFO] CLI args: []string{"/home/runner/terraform-provider-uptimerobot/terraform/terraform", "init"}
2020/09/17 13:48:24 [DEBUG] Attempting to open CLI config file: /home/runner/.terraformrc
2020/09/17 13:48:24 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/09/17 13:48:24 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/09/17 13:48:24 [DEBUG] ignoring non-existing provider search directory /home/runner/.terraform.d/plugins
2020/09/17 13:48:24 [DEBUG] ignoring non-existing provider search directory /home/runner/.local/share/terraform/plugins
2020/09/17 13:48:24 [DEBUG] ignoring non-existing provider search directory /usr/local/share/terraform/plugins
2020/09/17 13:48:24 [DEBUG] ignoring non-existing provider search directory /usr/share/terraform/plugins
2020/09/17 13:48:24 [INFO] CLI command args: []string{"init"}
2020/09/17 13:48:24 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2020/09/17 13:48:24 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2020/09/17 13:48:24 [DEBUG] New state was assigned lineage "c02efa31-5571-cc1c-b55d-b3936a753c49"
2020/09/17 13:48:24 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
2020/09/17 13:48:24 [TRACE] Meta.Backend: instantiated backend of type <nil>
2020/09/17 13:48:24 [DEBUG] checking for provisioner in "."
2020/09/17 13:48:24 [DEBUG] checking for provisioner in "/home/runner/terraform-provider-uptimerobot/terraform"
2020/09/17 13:48:24 [INFO] Failed to read plugin lock file .terraform/plugins/linux_amd64/lock.json: open .terraform/plugins/linux_amd64/lock.json: no such file or directory
2020/09/17 13:48:24 [TRACE] Meta.Backend: backend <nil> does not support operations, so wrapping it in a local backend
2020/09/17 13:48:24 [TRACE] backend/local: state manager for workspace "default" will:
 - read initial snapshot from terraform.tfstate
 - write new snapshots to terraform.tfstate
 - create any backup at terraform.tfstate.backup
2020/09/17 13:48:24 [TRACE] statemgr.Filesystem: reading initial snapshot from terraform.tfstate
2020/09/17 13:48:24 [TRACE] statemgr.Filesystem: snapshot file has nil snapshot, but that's okay
2020/09/17 13:48:24 [TRACE] statemgr.Filesystem: read nil snapshot
2020/09/17 13:48:24 [TRACE] providercache.fillMetaCache: scanning directory .terraform/plugins
2020/09/17 13:48:24 [TRACE] getproviders.SearchLocalDirectory: .terraform/plugins is a symlink to .terraform/plugins
2020/09/17 13:48:24 [TRACE] getproviders.SearchLocalDirectory: found registry.terraform.io/louy/uptimerobot v0.5.0 for linux_amd64 at .terraform/plugins/registry.terraform.io/louy/uptimerobot/0.5.0/linux_amd64
2020/09/17 13:48:24 [TRACE] providercache.fillMetaCache: including .terraform/plugins/registry.terraform.io/louy/uptimerobot/0.5.0/linux_amd64 as a candidate package for registry.terraform.io/louy/uptimerobot 0.5.0
2020/09/17 13:48:24 [TRACE] providercache.fillMetaCache: using cached result from previous scan of .terraform/plugins

Initializing the backend...

Initializing provider plugins...

Error: Failed to validate installed provider

- Using previously-installed louy/uptimerobot v0.5.0
Validating provider louy/uptimerobot v0.5.0 failed: provider binary not found:
could not find executable file starting with terraform-provider-uptimerobot

Expected Behavior

Terraform shouldn’t care what else is in the release artifacts

Actual Behavior

Very bad UX. Error seen in logs above

Steps to Reproduce

Create empty folder with config file above, then run terraform init

Additional Context

Issue was fixed by removing any other archive formats other than zip and re-releasing the provider.

alisdair commented 4 years ago

Thanks for reporting this! I can reproduce the issue with the configuration you provided.

I believe this is a bug in the Terraform Registry. Terraform core intentionally only supports Zip archives, so I believe that the registry should not have ingressed the tarball, and certainly not have provided it as a download URL:

$ curl -s https://registry.terraform.io/v1/providers/louy/uptimerobot/0.5.0/download/linux/amd64 | jq -r .download_url
https://github.com/louy/terraform-provider-uptimerobot/releases/download/v0.5.0/terraform-provider-uptimerobot_0.5.0_linux_amd64.tar.gz

I'm glad you were able to find a workaround! I'm passing this upstream to the Terraform Registry maintainers.