Unable to initialize Terraform OpenAPI provider

zjpiazza commented 4 years ago

Describe the bug

Followed the steps in the documentation to download the binary plugin and move it to the appropriate directory (%AppData%\Roaming\terraform.d\plugins, for Windows in my case). Created a main.tf file with a single resource definition:

provider "dtr" { apikey_auth = "" }

resource "dtr_repositories" "apache_repo" { namespace = " my_dtr_username / terraformtest" }

However when I try to perform "terraform init" it fails because it's querying the main hashicorp terraform registry for a provider named "dtr", which obviously fails. I've tried passing the "-plugin-dir" cli parameter and it fails in both cases. I'm sure it's something trivial I'm overlooking but I cannot get past this issue.

Any advice is appreciated!

dikhan commented 4 years ago

Hi @zjpiazza ,

Can you try running terraform in debug mode? That might give more info about what's going on. You can turn on debug mode using the TF_LOG=DEBUG before the terraform command. For instance: TF_LOG=DEBUG terraform init

Dani

zjpiazza commented 4 years ago

Sure thing. And thanks for the prompt reply. Also maybe I misunderstood the purpose of the project but I was under the impression that any OpenAPI compliant service can make use of this provider. But when I ran the provider binary after setting the OTF_VAR_x_SWAGGER_URL environment variable, I got back a ton of messages saying resource paths are not terraform data source compliant.

2020/09/24 20:14:20 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
  Use TF_LOG=TRACE to see Terraform's internal logs.
  ----
2020/09/24 20:14:20 [INFO] Terraform version: 0.13.2
2020/09/24 20:14:20 [INFO] Go runtime version: go1.14.7
2020/09/24 20:14:20 [INFO] CLI args: []string{"C:\\ProgramData\\chocolatey\\lib\\terraform\\tools\\terraform.exe", "init"}
2020/09/24 20:14:20 [DEBUG] Attempting to open CLI config file: C:\Users\piazza\AppData\Roaming\terraform.rc
2020/09/24 20:14:20 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2020/09/24 20:14:20 [DEBUG] checking for credentials in "C:\\Users\\piazza\\AppData\\Roaming\\terraform.d\\plugins"
2020/09/24 20:14:20 [DEBUG] ignoring non-existing provider search directory terraform.d/plugins
2020/09/24 20:14:20 [DEBUG] will search for provider plugins in C:\Users\piazza\AppData\Roaming\terraform.d\plugins
2020/09/24 20:14:20 [DEBUG] ignoring non-existing provider search directory C:\Users\piazza\AppData\Roaming\HashiCorp\Terraform\plugins
2020/09/24 20:14:20 [INFO] CLI command args: []string{"init"}
2020/09/24 20:14:20 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
  Use TF_LOG=TRACE to see Terraform's internal logs.
  ----

Initializing the backend...2020/09/24 20:14:20 [DEBUG] New state was assigned lineage "a1583c94-c5b8-7bab-e984-08debfa8c62f"

2020/09/24 20:14:20 [DEBUG] checking for provisioner in "."
2020/09/24 20:14:20 [DEBUG] checking for provisioner in "C:\\ProgramData\\chocolatey\\lib\\terraform\\tools"
2020/09/24 20:14:20 [DEBUG] checking for provisioner in "C:\\Users\\piazza\\AppData\\Roaming\\terraform.d\\plugins"
2020/09/24 20:14:20 [INFO] Failed to read plugin lock file .terraform\plugins\windows_amd64\lock.json: open .terraform\plugins\windows_amd64\lock.json: The system cannot find the path specified.

Initializing provider p2020/09/24 20:14:20 [DEBUG] Service discovery for registry.terraform.io at https://registry.terraform.io/.well-known/terraform.json
lugins...
- Finding latest version of hashicorp/dtr...
2020/09/24 20:14:21 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
  Use TF_LOG=TRACE to see Terraform's internal logs.
  ----
2020/09/24 20:14:21 [DEBUG] GET https://registry.terraform.io/v1/providers/hashicorp/dtr/versions
2020/09/24 20:14:21 [WARN] Log levels other than TRACE are currently unreliable, and are supported only for backward compatibility.
  Use TF_LOG=TRACE to see Terraform's internal logs.
  ----
2020/09/24 20:14:21 [DEBUG] GET https://registry.terraform.io/v1/providers/-/dtr/versions

dikhan commented 4 years ago

Hi @zjpiazza

The logging output suggests that your are using Terraform 0.13.2 which is the new Terraform version released and requires the plugins to be installed in a different path. This provider is compatible with Terraform 0.13 but in order for you to be able to use it you will need to follow Terraform 0.13 installation instructions to install the plugin in the right location (this is something that 0.13 introduced; the instructions in this repo on how to install this provider are based on Terraform 0.12 plugin installation instructions).

Terraform 0.13 requires custom plugins to be located in what Terraform refers to as placeholder hostnames. In your example that could be for instance: terraform.zjpiazza.com. This will be used in both the terraform configuration as well as the folder structure expected by terraform to find the plugin inside the terraform plugins folder (example below):

Your terraform config in the tf file could look like:

terraform {
  required_providers {
    dtr = {
      source  = "terraform.zjpiazza.com/apache/dtr" # Note 'terraform.zjpiazza.com' does not need to be resolvable, terraform will use this value to figure out the location of the plugin inspecting the file system (see more below). You can also choose any namespace (in this example it's "apache") and type (the plugin name "dtr") you wish to represent your in-house provider 
      version = "0.31.1"
    }
  }
}

To make version 0.31.1 of this provider available for installation from the local filesystem, choose one of the implied local mirror directories and create a directory structure under it like this (example below assumes windows_amd64):

terraform.zjpiazza.com/apache/dtr/0.31.1

Under that 0.31.1 directory, create one additional directory representing the platform where you are running Terraform, such as linux_amd64 for Linux on an AMD64/x64 processor, and then place the provider plugin executable and any other needed files in that directory.

Thus, on a Windows system, the provider plugin executable file might be at the following path:

terraform.zjpiazza.com/apache/dtr/0.31.1/windows_amd64/terraform-provider-dtr.exe

The OpenAPI Terraform provider expects the OpenAPI document to follow certain format following Restful style APIs to consider endpoints Terraform compliant. You can read more about this in the How To section.

Hope this helps! Dani

zjpiazza commented 4 years ago

Ah I didn't even think about the terraform version....I really appreciate the help, thank you!

dikhan commented 4 years ago

Considering this is not an issue with the plugin but rather Terraform configuration related, I will go ahead and close this issue.

dikhan / terraform-provider-openapi

Unable to initialize Terraform OpenAPI provider #270

Describe the bug