Bug while migrating to flux_bootstrap_git resource

[ldunkum]

Hi, thanks for the work on this provider, it helps us a lot to manage our environments.

We encountered an error while trying to upgrade to the new flux_bootstrap_git resource in 2 out of 6 environments. In the other environments, the import step ran without problems. All environments are managed by the same Terraform code, so theoretically the configurations shouldn't be different.

terragrunt import flux_bootstrap_git.main flux-system 

[...]

╷
│ Error: Getting expected repository files
│
│ Could not generate install manifests: accumulating resources: accumulation
│ err='merging resources from 'helm-controller.yaml': may not add resource
│ with an already registered id:
│ CustomResourceDefinition.v1.apiextensions.k8s.io/helmreleases.helm.toolkit.fluxcd.io.[noNs]':
│ must build at directory: '/tmp/flux-system3197074009/helm-controller.yaml':
│ file is not directory
╵

If necessary, I can post a full debug log, but it might take me some time to remove all secrets, references, etc.

[phillebaba]

This sounds like it could be an issue with what the provider expects to be present in the repository during import. As you stated the same Terraform has setup all environments so the manifests in the repositories should be the same.

As an initial step could you check your git repository to validate that there are no additional files committed to the specific directory. Alternatively that files are named something different.

[ldunkum]

Hi, thanks for your reply, we use a Monorepo for our clusters with the following structure:

clusters/
|-- cluster1
|--|--flux-system
|-- cluster2 
|--|--flux-system

In the path for the flux_bootstrap_git resource we just exchange the path in our clusters repo, e.g. path = "clusters/cluster1" for each environment. I've just checked our repo, and there doesn't seem to be an obvious difference between the flux yaml files, a search for helm-controller didn't return any irregularities either.

[phillebaba]

What I am interested in is what helm-controller.yaml is in your repository. Does this file exist in all of your environment directories?

[ldunkum]

There is no helm-controller.yaml file in the repo, the helm-controller is only present in the gotk-components.yaml file created by flux.

[phillebaba]

@ldunkum could you also share your Terraform configuration?

[ldunkum]

Sure, our flux configuration looks like this:

data "aws_eks_cluster_auth" "cluster_auth" {
  name = var.cluster_name
}

provider "flux" {
  host                   = data.aws_eks_cluster.cluster.endpoint
  cluster_ca_certificate = base64decode(data.aws_eks_cluster.cluster.certificate_authority[0].data)
  token                  = data.aws_eks_cluster_auth.cluster_auth.token
}

data "gitlab_project" "main" {
  id = "${var.gitlab_owner}/${var.repository_name}"
}

resource "gitlab_deploy_key" "main" {
  title    = "flux-${var.cluster_name}-${data.aws_caller_identity.current.account_id}"
  project  = data.gitlab_project.main.id
  key      = tls_private_key.main.public_key_openssh
  can_push = true

  depends_on = [data.gitlab_project.main]
}

resource "flux_bootstrap_git" "main" {
  depends_on = [gitlab_deploy_key.main]

  url     = "ssh://git@gitlab.com/${var.gitlab_owner}/${var.repository_name}.git"
  path    = var.target_path
  version = var.flux_version

  ssh = {
    username    = "git"
    private_key = tls_private_key.main.private_key_pem
  }
}