Lack of output/troubleshooting help

[pezhore]

I'm trying to figure out what I'm missing - no matter what I try, I can't seem to get reasonable output from the terraform resource in concourse. Success or failures don't seem to provide much output in the UI/console:

This makes troubleshooting particularly difficult and I'm open to suggestions on how to fix this.

Pipeline:

resource_types:
- name: merge-request
  type: docker-image
  source:
    repository: samcontesse/gitlab-merge-request-resource
- name: terraform
  type: docker-image
  source:
    repository: harbor.example.local:443/custom-certs/tf-docker
    tag: latest
    ca_certs:
    - domain: harbor.example.local:443
      cert: ((harbor.cert))

resources:
  - name: gated-vm-deploy
    type: merge-request
    icon: gitlab
    check_every: 1s
    source:
      uri: https://gitlab.example.local/pezhore/gated-vm-deploy.git
      private_token: ((gitlab.access_token))
      insecure: true
  - name: terraform
    type: terraform
    icon: terraform
    source:
      env_name: staging
      backend_type: s3
      backend_config:
        bucket: range-infra
        access_key: ((terraform.backend_access_key))
        secret_key: ((terraform.backend_secret_key))
        endpoint: https://minio.example.local
        key: gated-vm-deploy/terraform.tfstate
        region: main
        force_path_style: true
        skip_credentials_validation: true
        skip_metadata_api_check: true
        skip_region_validation: true
      env:
        VSPHERE_USER: ((terraform.vsphere_user))
        VSPHERE_PASSWORD: ((terraform.vsphere_password))
        VSPHERE_SERVER: vcenter.example.local
        TF_LOG: DEBUG
      vars:
        num_cpus: 2

jobs:
- name: merge-request
  plan:
  - get: gated-vm-deploy
    trigger: true
  - put: gated-vm-deploy
    params:
      repository: gated-vm-deploy
      status: running
  - put: terraform
    params:
      env_name: staging
      terraform_source: gated-vm-deploy/terraform
      plan_only: true
    get_params:
      output_planfile: true
  on_failure:
    put: gated-vm-deploy
    params:
      repository: gated-vm-deploy
      status: failed
  on_success:
    put: gated-vm-deploy
    params:
      repository: gated-vm-deploy
      status: success
      labels: ['unit-test', 'stage']
      comment:
        file: terraform/plan.json
        text: |
          plan output:
          $FILE_CONTENT
- name: terraform-apply
  plan:
  - get: gated-vm-deploy
    trigger: false
    passed: [merge-request]
  - get: terraform
    trigger: false
    passed: [merge-request]
  - put: terraform
    params:
      env_name: staging
      terraform_source: gated-vm-deploy/terraform
      plan_run: true

[ljfranklin]

Even successful runs print no output? My guess would be there's something wrong with the harbor.example.local:443/custom-certs/tf-docker image. How are you building that image? Would it be possible to try a dummy run with the normal ljfranklin/terraform-resource:latest image?

[pezhore]

Yup, even successful runs produce no output. All I'm doing for the custom-certs/tf-docker container is injecting our internal ca certs to ljfranklin/terraform-resource:latest. Otherwise, the terraform init freaks out with our internally signed Minio cluster S3 backend.

FROM ljfranklin/terraform-resource

RUN apk update && apk add ca-certificates && rm -rf /var/cache/apk/*

COPY ./certs/internal-ca.crt /usr/local/share/ca-certificates/internal-ca.crt
COPY ./certs/internal-int-ca.crt /usr/local/share/ca-certificates/internal-int-ca.crt
RUN update-ca-certificates

[ljfranklin]

Otherwise, the terraform init freaks out with our internally signed Minio cluster S3 backend.

So if you use the ljfranklin/terraform-resource:latest image then you at least get output showing terraform init failing? If so still seems like an issue with the custom image although your Dockerfile looks right to me.

Afraid I'm a bit at a loss here. Maybe try fly watch to see if you get output that way, could be an issue with streaming your build logs to the web UI. You could also try manually adding some fmt.Fprintf(os.Stderr, "TESTING") to the resource and manually building an image: https://github.com/ljfranklin/terraform-resource/blob/master/DEVELOPMENT.md#testing-your-changes-in-concourse.

[pezhore]

I'll take a look at building the image from scratch - thanks @ljfranklin

[pezhore]

@ljfranklin update: there's something funky with our concourse version because on a more recent version in stage the pipeline works just fine - output and all.