search

travisghansen / argo-cd-helmfile

Integration between argo-cd and helmfile
MIT License
213 stars 55 forks source link

Autodiscover with apps in app #38

Closed oniram88 closed 1 year ago

oniram88 commented 1 year ago

Hi, first of all thanks for the work.

I migrated a month ago to the new system with sidecars and everything works except for one case:

APPs in APP deployed with argo.

the structure of the directory is this:

root_manifest:

the discovery command is successful, in fact the result of the find is this:

 find . -type f -name "helmfile.yaml"
./application/redis/helmfile.yaml
./application/mysql/helmfile.yaml
./application/helmfile.yaml
./application/app/helmfile.yaml
./system-utils/external-secrets/helmfile.yaml
./system-utils/helmfile.yaml

is there a way to make the discovery command fail?

travisghansen commented 1 year ago

I think that should be easy enough. I’m assuming you want it on a per-app basis vs per-instance basis?

oniram88 commented 1 year ago

yes, i think that it should be great to have a ENV variable to set in the Argo Application spec to disable the autodiscover

like:

apiVersion: argoproj.io/v1alpha1
kind: Application
spec:
  source:
    plugin:
      env:
        - name: HELMFILE_DISCOVERY_DISABLE
          value: anything
travisghansen commented 1 year ago

https://github.com/travisghansen/argo-cd-helmfile/commit/e9951021760e4d6f51f172bb7fb39ef4a31e1dc1

In your case to fail the check do HELMFILE_DISCOVERY_RESPONSE=<0|false|no>

oniram88 commented 1 year ago

I tried with the new version Now the plugin correctly stops at discovery

msg="`argo-cd-helmfile.sh discover` failed exit status 1: v3.11.2+g912ebc1\nhelmfile version 0.152.0\nstarting discover" execID=4ad8b

with this spec for the application(redatted):

spec:
  destination:
    server: https://kubernetes.default.svc
  project: my-project
  source:
    path: manifests
    plugin:
      env:
        - name: HELMFILE_DISCOVERY_RESPONSE
          value: 'no'
    repoURL: https://XXXXXXXX
    targetRevision: master

but the last status is:

      message: >-
        rpc error: code = Unknown desc = plugin sidecar failed. could not find
        plugin supporting the given repository
      type: ComparisonError

and the logs in the argo server are:

time="2023-04-12T09:51:59Z" level=error msg="repository /tmp/_argocd-repo/42efe776-27f3-4dc5-a2fa-86bcb9bb9281/manifests is not the match because error receiving stream response: rpc error: code = Unknown desc = match repository error: error running find command: `argo-cd-helmfile.sh discover` failed exit status 1: v3.11.2+g912ebc1\nhelmfile version 0.152.0\nstarting discover" CWE=775 security=2
time="2023-04-12T09:51:59Z" level=error msg="finished unary call with code Unknown" error="plugin sidecar failed. could not find plugin supporting the given repository" grpc.code=Unknown grpc.method=GenerateManifest grpc.service=repository.RepoServerService grpc.start_time="2023-04-12T09:51:58Z" grpc.time_ms=482.042 span.kind=server system=grpc

I think that if I set the plugin configuration, but NO plugin is discovered, then argo fail. I'm right? it's now an argo problem?

travisghansen commented 1 year ago

Good question. What version of argocd are you running?

oniram88 commented 1 year ago

{ "Version": "v2.6.7+5bcd846", "BuildDate": "2023-03-23T14:57:27Z", "GitCommit": "5bcd846fa16e4b19d8f477de7da50ec0aef320e5", "GitTreeState": "clean", "GoVersion": "go1.18.10", "Compiler": "gc", "Platform": "linux/amd64", "KustomizeVersion": "v4.5.7 2022-08-02T16:35:54Z", "HelmVersion": "v3.10.3+g835b733", "KubectlVersion": "v0.24.2", "JsonnetVersion": "v0.19.1" }

crenshaw-dev commented 1 year ago

Yeah, by setting the plugin field, you're basically telling Argo CD "idk what plugin you should use, but you should definitely use a plugin!"

I think you could use this: ARGOCD_APP_SOURCE_PATH. If that variable doesn't start with application, then fail.

Another way to look at it: you could craft the find to only return true if there's a helmfile in the root of ARGOCD_APP_SOURCE_PATH.

travisghansen commented 1 year ago

@oniram88 I think the only sane way to handle your use-case is to introduce file-based logic in the repo itself. For example in the root of the app check for a .argo-cd-helmfile-env file or something where it's checked and sourced during execution. Then the above mentioned var could be checked without defining env vars in the plugin block.

travisghansen commented 1 year ago

This commit should make it just work:

This commit will give you extreme control (including implementing your own discover/generate/etc logic if desired inside your gitops repo) by simply creating a .argo-cd-helmfile-env file and export HELMFILE_DISCOVERY_RESPONSE=no for example.

In short, upgrade to v0.3.4 without making any changes at all and let me know how it goes.

oniram88 commented 1 year ago

Perfect!!! I removed:

    plugin:
      env:
        - name: HELMFILE_DISCOVERY_RESPONSE
          value: 'no'

and then the discovery exited correctly.

time="2023-04-18T08:04:38Z" level=error msg="`argo-cd-helmfile.sh discover` failed exit status 1: helm version v3.11.2+g912ebc1\nhelmfile version 0.152.0\nstarting discover" execID=f234b
time="2023-04-18T08:04:38Z" level=error msg="finished streaming call with code Unknown" error="match repository error: error running find command: `argo-cd-helmfile.sh discover` failed exit status 1: helm version v3.11.2+g912ebc1\nhelmfile version 0.152.0\nstarting discover" grpc.code=Unknown grpc.method=MatchRepository grpc.service=plugin.ConfigManagementPluginService grpc.start_time="2023-04-18T08:04:38Z" grpc.time_ms=161.219 span.kind=server system=grpc

the sub applications that uses helmfile works correctly.

Then I tested with all the other normal single level helmfile projects (~100 projects) and all works with v0.3.4 and HardRefresh