"Error running `workspace list`" with no output

[DanielJonesEB]

I've got no idea what the problem is here... Using 0.12.6. Ran some Terraform through the resource once, and it worked. Running an update through and getting this:

▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ Terraform Apply ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ Terraform Apply ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲
Failed To Run Terraform Apply!
2019/08/19 11:33:21 Apply Error: Error running `workspace list`: exit status 1, Output: 

Here's the config in the pipeline, all of which have sensible values:

- name: bosh-tf
  type: terraform
  source:
    backend_type: gcs
    backend_config:
      prefix: bosh
      <<: &tf-backend-config
        bucket: ((project_id))
        credentials: ((gcp_credentials_json))
    env:
      GOOGLE_CLOUD_KEYFILE_JSON: ((gcp_credentials_json))
      GOOGLE_PROJECT: ((project_id))
      GOOGLE_REGION: ((region))
      GOOGLE_ZONE: ((zone))

and the put:

- name: bosh-tf
  serial: true
  plan:
  - get: cfvk8s
    trigger: true
    passed: [setup-ci]
  - put: bosh-tf
    attempts: 2
    params:
      terraform_source: cfvk8s/k8s/tf/bosh/
      env_name: default
      vars:
        bucket_location: ((bucket_location))
        project_id: ((project_id))

Have I missed something obvious?

[ljfranklin]

Hi @DanielJonesEB, I just pushed this commit to the latest image which prints STDERR when the workspace list command fails. Can you try running it again and see if you get any helpful output?

[DanielJonesEB]

Thanks for the help, @ljfranklin. I think you may have pushed to 0.12.6 too, as I didn't have to update my resource_types with the new tag :)

Here's the output now, running against the same input

▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ Terraform Apply ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼
▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ Terraform Apply ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲ ▲
Failed To Run Terraform Apply!
2019/08/20 11:38:11 Apply Error: Error running `workspace list`: exit status 1, Output: 2019/08/20 11:38:11 [INFO] Terraform version: 0.12.6  
2019/08/20 11:38:11 [INFO] Go runtime version: go1.12.4
2019/08/20 11:38:11 [INFO] CLI args: []string{"/usr/local/bin/terraform", "workspace", "list"}
2019/08/20 11:38:11 [DEBUG] Attempting to open CLI config file: /root/.terraformrc
2019/08/20 11:38:11 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/08/20 11:38:11 [INFO] CLI command args: []string{"workspace", "list"}

Error: Invalid expression

  on bosh.tf line 32, in resource "google_compute_firewall" "director":
  32:     "${google_project_service.compute}",

A single static variable reference is required: only attribute access and
indexing with constant keys. No calculations, function calls, template
expressions, etc are allowed here.

I'm wondering if 0.12.6 is being updated by CI, maybe it wasn't as 'pinned' as I thought, and I was effectively using latest and maybe an output-swallowing bug was introduced previously? Now I've got output it definitely looks like PEBKAC and my dodgy Terraform, but I wouldn't have expected behaviour to have changed when I was using a tagged version of the image.

[DanielJonesEB]

Ah, just realised that the tags on the Docker images refer to the version of Terraform, not the version of the resource.

Would it be possible to introduce some other versioning mechanism so we can use the latest version of Terraform without necessarily picking up new resource changes?

Just to show I'm not completely insane:

Perhaps there was something wrong with my worker that was causing the Terraform executable to not be runnable on build #6.

[ljfranklin]

@DanielJonesEB the current tagging implementation is maybe a little surprising, at the end of the resource's CI pipeline it tags the current Terraform + resource image with latest and <terraform_version>. So the current Terraform version tag does float on new resource code until a newer Terraform version comes out, at which point the previous Terraform version tag really is static.

Aside from a larger change to the resource earlier this year to update the resource to handle Terraform Backends, I don't often add new functionality to the resource other than bug fixes. The lack of STDERR from workspace list has always been there for example. Also newer versions of Terraform often introduce breaking changes in the CLI which require updates to the resource to work. Given that I'm not sure that versioning the resource separately from Terraform itself provides much value.