Something strange since resource bump to terraform 0.15.0

[jppitout]

IaaS: Azure Workspaces: Yes Terraform vestions: 0.11.14

Since the resource got bumped it seems either this resource is ignoring the env_name: prod parameter or the resource type is ignoring the tag: "0.11.14" parameter.

Even though both above parameters are set we are seeing a newly created terraform.tfstate file in our Azure blob storage container (i.e. instead of per workspace tfstate files e.g. terraform.tfstateenv:prod) and it is populated with the below despite the above tag and env_name being set:

{
  "version": 4,
  "terraform_version": "0.15.0",
  "serial": 0,
  "lineage": "5cd1a576-9c56-ac9b-ed19-7484f7266fb4",
  "outputs": {},
  "resources": []
}

Now we are seeing these errors in our pipeline:

Terraform doesn't allow running any operations against a state
that was written by a future Terraform version. The state is
reporting it is written by Terraform '0.15.0'

A newer version of Terraform is required to make changes to the current
workspace.

But it is looking at a tfstate file that shouldn't even exist. Not sure why or how this tfstate file is being created/referenced but the issue started shortly after this commit: https://github.com/ljfranklin/terraform-resource/commit/b59eb92c040a43983ec758fbbb4409f744f8afb5 I don't see any breaking changes here so really not sure...

Example code:

resource_types:
- name: terraform
  type: docker-image
  source:
    repository: ljfranklin/terraform-resource
    tag: "0.11.14"

resources:
- name: infrastructure
  icon: terraform
  type: terraform
  source:
    env_name: ((env))
    backend_type: azurerm
    backend_config:
      resource_group_name: ((pipeline_resource_group_name))
      storage_account_name: ((pipeline_storage_account_name))
      container_name: ((pipeline_tfstate_container_name))
      key: terraform.tfstate
    env:
      ARM_SUBSCRIPTION_ID: ######
      ARM_CLIENT_ID: ######
      ARM_CLIENT_SECRET: ######
      ARM_TENANT_ID: ######

Any help would be appreciated.

[jppitout]

For further clarity... we're running two Terraform verisons namely 0.11.14 and 0.14.2. It turns out that not all the 0.14.2 tasks had the 0.14.2 tag specified and so was pulling the latest image instead. This was ultimately the root cause but I'm still not sure why the resource created 0.15.0 terraform.tfstate file instead of updating the workspaces... I'm grateful it didn't though because it might have been harder to fix.

I'll need to read up on 0.15.0 behaviour to see why the 0.15.0 tfstate was created. Any input on this would be greatly valued!

[ljfranklin]

Glad you found the missing tag!

I'm still not sure why the resource created 0.15.0 terraform.tfstate file instead of updating the workspaces

Terraform has the concept of a "default" workspace that gets used if you don't explicitly specify a workspace name. It's possible Terraform is auto-creating an empty statefile for the default workspace at some point, e.g. during terraform init. Given the statefile was empty, i.e. "resources": [], I'm not really concerned. If you wanted to investigate further you could delete that statefile and manually run some Terraform commands to see if the default statefile gets populated.

[jppitout]

Thanks for the info! I find it strange that 0.14.2 does not create the empty statefile but for now this issue can be closed. We have Terraform upgrades planned for the future and will revisit this then :)