Git Submodules should be configurable to ignore upstream, private submodules

[soudaburger]

Checklist:

• [X] I've searched in the docs and FAQ for my answer: https://bit.ly/argocd-faq.
• [X] I've included steps to reproduce the bug.
• [X] I've pasted the output of argocd version.

Describe the bug

We instead attempted to leverage git submodules to clone down the teleport repository with a specific tag, e.g., v10.1.0. We were able to get this to properly work locally, but because argocd performs a git submodule --init --recursive, it ALSO pulled in the .gitmodules from the teleport repository. However, the e repository at git@github.com:gravitational/teleport.e.git appears to be a private repository and since we don't have access to the repository, argocd applications simply fail to render at all when submodules simply exist in our repo.

To Reproduce

• Use a github repository with a submodule.
• Cloned submodule should have private, non-accessible submodule.
• Watch argocd fail to render charts from repositories that attempt to leverage initial submodule.

Expected behavior

I expect that argocd should simply either silently skip the nested submodule it can't clone or otherwise be provided a configuration parameter to exclude specifically matched git submodules so that argocd does NOT attempt to clone them.

Version ArgoCD Version: 2.4.8 ArgoCD Helm Version: 4.10.4

Logs

[argocd-repo-server-bb57b5f6d-m24j2 repo-server] time="2022-08-02T13:53:52Z" level=error msg="`git submodule update --init --recursive` failed exit status 1: Submodule 'kustomize/teleport/charts/teleport' (https://github.com/gravitational/teleport) registered for path 'kustomize/teleport/charts/teleport'\nCloning into '/tmp/_argocd-repo/0c504034-ef49-4a45-bc3c-4a63f836d33c/kustomize/teleport/charts/teleport'...\nSubmodule 'e' (git@github.com:gravitational/teleport.e.git) registered for path 'kustomize/teleport/charts/teleport/e'\nSubmodule 'webassets' (https://github.com/gravitational/webassets.git) registered for path 'kustomize/teleport/charts/teleport/webassets'\nCloning into '/tmp/_argocd-repo/0c504034-ef49-4a45-bc3c-4a63f836d33c/kustomize/teleport/charts/teleport/e'...\ngit@github.com: Permission denied (publickey).\r\nfatal: Could not read from remote repository.\n\nPlease make sure you have the correct access rights\nand the repository exists.\nfatal: clone of 'git@github.com:gravitational/teleport.e.git' into submodule path

[crenshaw-dev]

I think we could tackle this and https://github.com/argoproj/argo-cd/issues/3799 at the same time.

I'd propose adding these fields:

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  source:
    fetch:
      submodules:  # submodules: {} would mean fetch with default "recursive" value (true)
        recursive: false

I've seen other issues where folks wanted finer-grained control over how fetches happened. This new field would be a great place to add those features.

[crenshaw-dev]

Would a per-app config be sufficient, or would you want a global setting?

[soudaburger]

I think I'm more concerned with being able to restrict pulling certain upstream, recursive submodules.

For example: My helm chart would like to use the teleport helm chart as a submodule. So we add that submodule in our local .gitmodules.. But the teleport helm has it's own .gitmodules. It's THOSE submodule(s) that I'd like to be able to tell argo NOT to pull or otherwise skip.

So if https://github.com/argoproj/argo-cd/issues/3799 meets those expectations, then I'm all for it, but with the way I read that, it's not going to solve my expectation.

[crenshaw-dev]

Yeah, I don't think #3799 would solve your use case. But I wonder if fetch.submodules.recursive: false would. i.e., if we ran git submodule --init instead of git submodule --init --recursive, could you get the top-level submodule that you need without also pulling the secondary submodules?

[soudaburger]

Yes that might work, especially if it's per argo application. That might just be sufficient! Thanks!

[crenshaw-dev]

I'll try to piece together a PR real quick, but if it ends up being involved, will probably have to leave it as a draft to see if someone else can pick it up.

[crenshaw-dev]

This is an example of something else that could be configured in a new fetch field: https://github.com/argoproj/argo-cd/issues/8938

[crenshaw-dev]

@ashutosh16 does this draft PR look like something you could pick up?

[ashutosh16]

@soudaburger You can manage submodules at git config level.. Add the option fetchRecurseSubmodules = false in your .gitmodules file that should restrict access to private repo. Can your try and let me know if that resolve your issue?

[submodule "<private module>"]
    path = <<path-to-private-module>>
    url = <<giturl-to-private-module>>
    branch = main
    shallow = true
    fetchRecurseSubmodules = false