2.12.x throws could not read Username for 'https://gitlab.com' from the UI

c-buisson commented 2 months ago

Checklist:

[X] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
[X] I've included steps to reproduce the bug.
[X] I've pasted the output of argocd version.

Describe the bug

Since 2.12.x, the UI throws this error message:

Unable to load data: error acquiring repo lock: failed to initialize repository resources: rpc error: code = Internal desc = Failed to fetch default: `git fetch origin --tags --force --prune` failed exit status 128: fatal: could not read Username for 'https://gitlab.com/': terminal prompts disabled

To Reproduce

Upgrade to 2.12.x from 2.11.x with connectivity to Gitlab repositories. Using Helm chart version 7.4.4

Expected behavior

UI able to refresh Applications from Gitlab.

Screenshots

argocd: v2.11.2+25f7504
  BuildDate: 2024-05-23T13:58:43Z
  GitCommit: 25f7504ecc198e7d7fdc055fdb83ae50eee5edd0
  GitTreeState: clean
  GoVersion: go1.21.10
  Compiler: gc
  Platform: linux/amd64
argocd-server: v2.12.1+26b2039
  BuildDate: 2024-08-16T16:42:13Z
  GitCommit: 26b2039a55b9bdf807a70d344af8ade5171d3d39
  GitTreeState: clean
  GoVersion: go1.22.4
  Compiler: gc
  Platform: linux/amd64
  Kustomize Version: v5.4.2 2024-05-22T15:19:38Z
  Helm Version: v3.15.2+g1a500d5
  Kubectl Version: v0.29.6
  Jsonnet Version: v0.20.0

Logs

`git fetch origin --tags --force --prune` failed exit status 128: fatal: could not read Username for 'https://gitlab.com': terminal prompts disabled

argo1984 commented 2 months ago

Hi together,

i have the same issue with 2.12.X and it seems that the login to the repository works (it is green) but the Projects and Applications are not able to access the credentials.

blakepettersson commented 2 months ago

@c-buisson and @argo1984 can you give the specs of the applications which are failing, and the (redacted) specs of the repo credentials that are expected to work with these applications? I suspect shenanigans from #18388.

crenshaw-dev commented 2 months ago

+1 to Blake's question, but to anyone who's experiencing this issue. :-)

mozemke commented 2 months ago

hi there,

we're facing the same problems and I read through #19587 IMO we should not be affected with our setup but are still getting the listed error.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
  name: project-1-appset
spec:
  goTemplate: true
  goTemplateOptions: ["missingkey=error"]
  generators:
    - git:
        repoURL: https://gitlab.xxxxx/project-1/deployment.git
        revision: dev
        directories:
        - path: '*/overlays/*'
        - path: '*/overlays/prod*'
          exclude: true
        - path: '*/overlays/xx*'
          exclude: true
        values:
          appName: '{{index .path.segments 0}}-{{.path.basenameNormalized}}'
  template:
    metadata:
      name: '{{.values.appName}}'
      labels:
        env: '{{.path.basenameNormalized}}'
    spec:
      project: < project-1 >
      source:
        repoURL: https://gitlab.xxxxx/project-1/deployment.git
        targetRevision: dev
        path: '{{.path.path}}'
      destination:
        server: https://kubernetes.default.svc
        namespace: 'xx-{{.values.appName}}'
      syncPolicy:
        automated:
          prune: true
          selfHeal: true
        syncOptions:
          - CreateNamespace=true
        managedNamespaceMetadata:
          labels:
            app.kubernetes.io/instance: '{{.values.appName}}'
          annotations: 
            field.cattle.io/projectId: xx
            argocd.argoproj.io/tracking-id: xx
---
# add creds for appset in admin project
apiVersion: v1
kind: Secret
metadata:
  name: xxx-admin-repo-creds
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  type: git
  url: https://gitlab.xxxxx/project-1/deployment.git
  username: xxxx
  password: xxxx
  project: < admin-project >
---
# add creds for project-1 apps
apiVersion: v1
kind: Secret
metadata:
  name: xxx-xxx-repo-creds
  labels:
    argocd.argoproj.io/secret-type: repository
stringData:
  type: git
  url: https://gitlab.xxxxx/project-1/deployment.git
  username: xxxx
  password: xxxx
  project: < project-1 >

the appset of project-1 is deployed to the admin-project, that's why we are having the same secret two times, but with different project scope, so that the admin project is able to generate the apps. If now the new way to match repo creds to apps is using project + url - I don't see why our setup is failing :(

The apps themselves have no connection problems.

error generating params from git: error getting directories from repo: error retrieving Git Directories: rpc error: code = Internal desc = unable to checkout git repo https://gitlab.xxxxx/project-1/deployment.git with revision e32c9a160962137771b976eff8c82c0c28bad41a: failed to initialize repository resources: rpc error: code = Internal desc = Failed to fetch default: `git fetch origin --tags --force --prune` failed exit status 128: fatal: could not read Username for 'https://gitlab.xxxx': terminal prompts disabled

We are facing this problem with all our Appsets, above is just one example.

blakepettersson commented 2 months ago

I suspect it's the same thing actually :( An applicationset doesn't have a project; it's admin-scoped. I believe the resolution is exactly the same as for #19587 - having a repo-cred that doesn't have a project set.

mozemke commented 2 months ago

yep that did indeed fix it. I now understand what you mean :D

We actually deploy our appsets, with one bootstrapping app, that one has set a project. But since AppSets do not have the key, it all makes sense now! :)

The downside now is, that the repo creds can now be used by all projects.

thank you! :)

blakepettersson commented 2 months ago

The downside now is, that the repo creds can now be used by all projects.

Yes, that is true; that was however the same behavior in < 2.12. A workaround for that would be to disallow that source to be used in your AppProjects:

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
  name: < project-1 >
spec:
  sourceRepos:
    - '!https://gitlab.xxxxx/project-1/deployment.git'

mozemke commented 2 months ago

that was however the same behavior in < 2.12.

I actually never knew that before I read through the issue/pr to enhance the situation :D We'll implement your workaround! :)

thanks! :)