Error: reading token file: reading OIDC Token

[soyetoro]

Error: reading token file: reading OIDC Token

Config Definition

I am having some issues performing auth to Azure when using the runner. My setup is detailed below which detailed how my setup and config looks like

apiVersion: infra.contrib.fluxcd.io/v1alpha2
kind: Terraform
metadata:
  name: azure-resources-tf-sa1
  namespace: flux-system
spec:
  interval: 2m
  approvePlan: auto
  path: ./Terraform
  tfstate:
    forceUnlock: auto
  backendConfig:
    customConfiguration: |
      backend "azurerm" {
        resource_group_name  = "d-aks-rg"
        storage_account_name = "storageaccount"
        container_name       = "tfstate"
        key                 = "azure-resources.tfstate"
        use_oidc            = true
      }
  sourceRef:
    kind: GitRepository
    name: dnm
    namespace: flux-system
  serviceAccountName: azure-tf-controller-sa
  runnerPodTemplate:
    metadata:
      labels:
        aadpodidbinding: azure-tf-controller-identity
      annotations:
        azure.workload.identity/client-id: "xxxx-xxxx-xxxxx-xxxx"
    spec:
      image: ghcr.io/weaveworks/tf-runner-azure:v0.15.1 #ghcr.io/weaveworks/tf-runner:latest
      env:
        - name: ARM_USE_OIDC
          value: "true"
        - name: ARM_SUBSCRIPTION_ID
          value: "xxxx-xxxx-xxxxx-xxxx"
        - name: ARM_TENANT_ID
          value: "xxxx-xxxx-xxxxx-xxxx"
        - name: ARM_CLIENT_ID
          value: "xxxx-xxxx-xxxxx-xxxx"
        - name: ARM_OIDC_TOKEN_FILE_PATH
          value: "/var/run/secrets/azure/tokens/azure-identity-token"
      # tolerations:
      #   - key: "terraform"
      #     operator: "Equal"
      #     value: "true"
      #     effect: "NoSchedule"
      # nodeSelector:
      #   purpose: terraform

---

apiVersion: source.toolkit.fluxcd.io/v1
kind: GitRepository
metadata:
  name: dnm
  namespace: flux-system
spec:
  interval: 30s
  url: https://github.com/xxxx-xxxx-xxxxx-xxxx/repo
  ref:
    branch: main

---

apiVersion: v1
kind: ServiceAccount
metadata:
  name: azure-tf-controller-sa
  namespace: flux-system
  annotations:
    azure.workload.identity/client-id: "xxxx-xxxx-xxxxx-xxxx"
    azure.workload.identity/tenant-id: "xxxx-xxxx-xxxxx-xxxx" 

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: terraform-runner-role
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
- apiGroups: ["infra.contrib.fluxcd.io"]
  resources: ["*"]
  verbs: ["*"]

---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: terraform-runner-rolebinding
subjects:
- kind: ServiceAccount
  name: azure-tf-controller-sa
  namespace: flux-system
roleRef:
  kind: ClusterRole
  name: terraform-runner-role
  apiGroup: rbac.authorization.k8s.io

Error Log

The error noticed in the runner:

{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"generating the template founds"}
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"main.tf.tpl not found, skipping"}
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"initializing","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6"}
{"level":"info","ts":"2024-11-01T15:29:15.626Z","logger":"runner.terraform","msg":"mapping the Spec.BackendConfigsFrom","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6"}
{
  "terraform_version": "1.3.9",
  "platform": "linux_amd64",
  "provider_selections": {},
  "terraform_outdated": true
}
Upgrading modules...
- aks in modules/aks

Initializing the backend...

Error: reading token file: reading OIDC Token "/var/run/secrets/azure/tokens/azure-identity-token": open /var/run/secrets/azure/tokens/azure-identity-token: no such file or directory

{"level":"error","ts":"2024-11-01T15:29:15.787Z","logger":"runner.terraform","msg":"unable to initialize","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6","error":"exit status 1

Error: reading token file: reading OIDC Token "/var/run/secrets/azure/tokens/azure-identity-token": open /var/run/secrets/azure/tokens/azure-identity-token: no such file or directory

"}
{"level":"info","ts":"2024-11-01T15:29:15.787Z","logger":"runner.terraform","msg":"cleanup TmpDir","instance-id":"2fc52e85-d74a-4bd6-84e7-8ff9a0487cd6","tmpDir":"/tmp/flux-system-azure-resources-tf-sa1"}

Notes

Additional Context I am using Azure Pod Identity Binding and have created a federated identity on the managed identity with access to the Kubernetes cluster.

Possible Issue The error suggests that the OIDC token file at /var/run/secrets/azure/tokens/azure-identity-token is missing, which is likely related to the Azure Pod Identity setup. I have configured the Service Account with appropriate annotations for Azure Workload Identity, but it appears the runner cannot access the required token file.

Steps Taken Verified the Service Account and managed identity setup. Ensured aadpodidbinding and other annotations are correctly configured. Any insights or guidance on resolving the missing OIDC token file issue would be greatly appreciated.

[soyetoro]

@chanwit Can I get some support on this? :)

[chanwit]

@soyetoro please email me: chanwit at gmail.