Atlantis apply fails because of git dubious ownership

BenoitLF commented 1 year ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Overview of the Issue

Atlantis plan works but the apply does not because of a git error (similar to : this issue)

Logs

running git ls-files . --others: fatal: detected dubious ownership in repository at '/home/atlantis/.atlantis/repos/org/repo-name/10/default'
To add an exception for this directory, call:

    git config --global --add safe.directory /home/atlantis/.atlantis/repos/org/repo-name/10/default
: exit status 128

--->

Environment details

Atlantis version: latest
Deployment method: ecs, terraform-atlantis module

BenoitLF commented 1 year ago

Was able to fix the issue like this comment

nitrocode commented 1 year ago

Thanks @BenoitLF. This would be nice to add to our troubleshooting page. We welcome PRs if you have the time

jiba21 commented 1 year ago

@nitrocode and @BenoitLF for me it's not working, why enable_ephemeral_storage = true?

jiba21 commented 1 year ago

BenoitLF commented 1 year ago

enable_ephemeral_storage does not work ?

jiba21 commented 1 year ago

@BenoitLF
If enable_ephemeral_storage == true, the EFS config is disabled as you could see in the image: count = true = 0

I use that config

resource "aws_efs_access_point" "this" {
  file_system_id = local.file_system

  root_directory {
    path = "/${var.application}/${var.environment}"
    creation_info {
      owner_gid   = local.gid (1000)
      owner_uid   = local.uid (100)
      permissions = 0750
    }
  }

  posix_user {
    gid = local.gid (1000)
    uid = local.uid (100)
  }
}

But is failing with: detected dubious ownership in repository

jseiser commented 1 year ago

This has broke our Atlantis as well.

We do not use access points, we provision EFS, and then allow the EFS csi to do the provisioning.

We are failing to download modules

nitrocode commented 1 year ago

Perhaps this command needs to be running Atlantis on a new pr or in a pre workflow hook?

git config --global --add safe.directory '*'

https://stackoverflow.com/questions/72978485/git-submodule-update-failed-with-fatal-detected-dubious-ownership-in-repositor

jseiser commented 1 year ago

OK, i actually figured out whats going on, but im not sure why it just now started failed.

We have atlantis in 2 clusters, handling deployments for like 20+ repos. So both are heavily used and have went through multiple upgrades etc.

We provision EFS using the EFS CSI helm chart. The default setting for your provisioner is to use a random range for the UID/GID for the EFS mount.

So the atlantis helm chart allows you to lock the uid/gid of the user to 1000:1000 ( this matches the debian image before it was broken). But the EFS mount may be owned by say :1008:10008 . I had to re-create a new EFS provisioner just for atlantis, locking the UID/GID to 1000:1000.

The other atlantis deployment is currently working.. and the EFS mount is owned by 1009:1009 so im not sure what needs to happen to trigger it to fail.

bml1g12 commented 1 year ago

Encountered this issue from today so will try https://github.com/runatlantis/atlantis/issues/3139#issuecomment-1432882110 and it seems to have worked around the issue for now at least

jamengual commented 1 year ago

you can do this too:

repos:
  - id: /.*/
    # allowed_overrides specifies which keys can be overridden by this repo in
    # its atlantis.yaml file.
    allowed_overrides: [apply_requirements, workflow]
    apply_requirements: [approved,mergeable]
    # allow_custom_workflows defines whether this repo can define its own
    # workflows. If false (default), the repo can only use server-side defined
    # workflows.
    allow_custom_workflows: true
    # This hook is to work around https://github.com/runatlantis/atlantis/issues/2221
    pre_workflow_hooks:
      - run: git config --global --add safe.directory '*'

aalvarezaph commented 1 year ago

you can do this too:

repos:
  - id: /.*/
    # allowed_overrides specifies which keys can be overridden by this repo in
    # its atlantis.yaml file.
    allowed_overrides: [apply_requirements, workflow]
    apply_requirements: [approved,mergeable]
    # allow_custom_workflows defines whether this repo can define its own
    # workflows. If false (default), the repo can only use server-side defined
    # workflows.
    allow_custom_workflows: true
    # This hook is to work around https://github.com/runatlantis/atlantis/issues/2221
    pre_workflow_hooks:
      - run: git config --global --add safe.directory '*'

Using this to overcome the issue as I think persistent storage makes sense (maybe I am wrong). A not so nice side effect is that now the hook, that we use as a workaround, will show up on PR status

https://github.com/runatlantis/atlantis/issues/3173 would allow getting rid of this if worked on

cjsrkd3321 commented 1 year ago

I'm going through it, too. But, my apply worked when with -d option.. I don't know why..

RafPe commented 7 months ago

you can do this too:
repos:
  - id: /.*/
    # allowed_overrides specifies which keys can be overridden by this repo in
    # its atlantis.yaml file.
    allowed_overrides: [apply_requirements, workflow]
    apply_requirements: [approved,mergeable]
    # allow_custom_workflows defines whether this repo can define its own
    # workflows. If false (default), the repo can only use server-side defined
    # workflows.
    allow_custom_workflows: true
    # This hook is to work around https://github.com/runatlantis/atlantis/issues/2221
    pre_workflow_hooks:
      - run: git config --global --add safe.directory '*'
Using this to overcome the issue as I think persistent storage makes sense (maybe I am wrong). A not so nice side effect is that now the hook, that we use as a workaround, will show up on PR status

3173 would allow getting rid of this if worked on

You can just add description field that looks more appealing ;)

      pre_workflow_hooks:
        - run: |
            git config --global --add safe.directory '*'        
          description: configure-git-credentials

runatlantis / atlantis