Open moreiramarti opened 1 year ago
argocd app get ephemeral-pull-helmfile --hard-refresh
WARN[0000] Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web.
WARN[0003] Failed to invoke grpc call. Use flag --grpc-web in grpc calls. To avoid this warning message, use flag --grpc-web.
Name: argocd/ephemeral-pull-helmfile
Project: infra
Server: testbed
Namespace: ephemeral-pull-helmfile
URL: https://argocd.voodoo-infra-sandbox.io/applications/ephemeral-pull-helmfile
Repo: https://github.com/VoodooTeam/argocd-sandbox-infra.git
Target: master
Path: ephemeral/
SyncWindow: Sync Allowed
Sync Policy: Automated (Prune)
Sync Status: Unknown
Health Status: Healthy
CONDITION MESSAGE LAST TRANSITION
ComparisonError rpc error: code = Unknown desc = plugin sidecar failed. error generating manifests in cmp: rpc error: code = Unknown desc = error generating manifests: `bash -c /usr/local/bin/argo-cd-helmfile.sh generate` failed exit status 1: v3.10.3+g835b733
helmfile version 0.151.0
starting generate
Templating release=db, chart=bitnami/postgresql
Templating release=db-migration, chart=../charts/db-migration
Pulling 456072703506.dkr.ecr.eu-west-1.amazonaws.com/helm/stateless-service:6.0.1
in helmfile.d/01_application.yaml: [release "ephemeral": command "/usr/local/bin/helm" exited with non-zero status:
PATH:
/usr/local/bin/helm
ARGS:
0: /usr/local/bin/helm (19 bytes)
1: pull (4 bytes)
2: oci://456072703506.dkr.ecr.eu-west-1.amazonaws.com/helm/stateless-service (73 bytes)
3: --version (9 bytes)
4: 6.0.1 (5 bytes)
5: --destination (13 bytes)
6: /tmp/helmfile2452714502/ephemeral-pull-helmfile/ephemeral/stateless-service/6.0.1 (81 bytes)
7: --untar (7 bytes)
8: --kube-version=1.23 (19 bytes)
9: --api-versions=acme.cert-manager.io/v1 (38 bytes)
10: --api-versions=acme.cert-manager.io/v1/Challenge (48 bytes)
11: --api-versions=acme.cert-manager.io/v1/Order (44 bytes)
12: --api-versions=admissionregistration.k8s.io/v1 (46 bytes)
13: --api-versions=admissionregistration.k8s.io/v1/MutatingWebhookConfiguration (75 bytes)
14: --api-versions=admissionregistration.k8s.io/v1/ValidatingWebhookConfiguration (77 bytes)
15: --api-versions=apiextensions.k8s.io/v1 (38 bytes)
16: --api-versions=apiextensions.k8s.io/v1/CustomResourceDefinition (63 bytes)
17: --api-versions=apiregistration.k8s.io/v1 (40 bytes)
18: --api-versions=apiregistration.k8s.io/v1/APIService (51 bytes)
19: --api-versions=apps/v1 (22 bytes)
20: --api-versions=apps/v1/ControllerRevision (41 bytes)
21: --api-versions=apps/v1/DaemonSet (32 bytes)
22: --api-versions=apps/v1/Deployment (33 bytes)
23: --api-versions=apps/v1/ReplicaSet (33 bytes)
24: --api-versions=apps/v1/StatefulSet (34 bytes)
25: --api-versions=argoproj.io/v1alpha1 (35 bytes)
26: --api-versions=argoproj.io/v1alpha1/AppProject (46 bytes)
27: --api-versions=argoproj.io/v1alpha1/Application (47 bytes)
28: --api-versions=argoproj.io/v1alpha1/ApplicationSet (50 bytes)
29: --api-versions=argoproj.io/v1alpha1/ArgoCDExtension (51 bytes)
30: --api-versions=autoscaling/v1 (29 bytes)
31: --api-versions=autoscaling/v1/HorizontalPodAutoscaler (53 bytes)
32: --api-versions=autoscaling/v2 (29 bytes)
33: --api-versions=autoscaling/v2/HorizontalPodAutoscaler (53 bytes)
34: --api-versions=autoscaling/v2beta1 (34 bytes)
35: --api-versions=autoscaling/v2beta1/HorizontalPodAutoscaler (58 bytes)
36: --api-versions=autoscaling/v2beta2 (34 bytes)
37: --api-versions=autoscaling/v2beta2/HorizontalPodAutoscaler (58 bytes)
38: --api-versions=batch/v1 (23 bytes)
39: --api-versions=batch/v1/CronJob (31 bytes)
40: --api-versions=batch/v1/Job (27 bytes)
41: --api-versions=batch/v1beta1 (28 bytes)
42: --api-versions=batch/v1beta1/CronJob (36 bytes)
43: --api-versions=bitnami.com/v1alpha1 (35 bytes)
44: --api-versions=bitnami.com/v1alpha1/SealedSecret (48 bytes)
45: --api-versions=cert-manager.io/v1 (33 bytes)
46: --api-versions=cert-manager.io/v1/Certificate (45 bytes)
47: --api-versions=cert-manager.io/v1/CertificateRequest (52 bytes)
48: --api-versions=cert-manager.io/v1/ClusterIssuer (47 bytes)
49: --api-versions=cert-manager.io/v1/Issuer (40 bytes)
50: --api-versions=certificates.k8s.io/v1 (37 bytes)
51: --api-versions=certificates.k8s.io/v1/CertificateSigningRequest (63 bytes)
52: --api-versions=coordination.k8s.io/v1 (37 bytes)
53: --api-versions=coordination.k8s.io/v1/Lease (43 bytes)
54: --api-versions=crd.k8s.amazonaws.com/v1alpha1 (45 bytes)
55: --api-versions=crd.k8s.amazonaws.com/v1alpha1/ENIConfig (55 bytes)
56: --api-versions=discovery.k8s.io/v1 (34 bytes)
57: --api-versions=discovery.k8s.io/v1/EndpointSlice (48 bytes)
58: --api-versions=discovery.k8s.io/v1beta1 (39 bytes)
59: --api-versions=discovery.k8s.io/v1beta1/EndpointSlice (53 bytes)
60: --api-versions=elbv2.k8s.aws/v1alpha1 (37 bytes)
61: --api-versions=elbv2.k8s.aws/v1alpha1/TargetGroupBinding (56 bytes)
62: --api-versions=elbv2.k8s.aws/v1beta1 (36 bytes)
63: --api-versions=elbv2.k8s.aws/v1beta1/IngressClassParams (55 bytes)
64: --api-versions=elbv2.k8s.aws/v1beta1/TargetGroupBinding (55 bytes)
65: --api-versions=events.k8s.io/v1 (31 bytes)
66: --api-versions=events.k8s.io/v1/Event (37 bytes)
67: --api-versions=events.k8s.io/v1beta1 (36 bytes)
68: --api-versions=events.k8s.io/v1beta1/Event (42 bytes)
69: --api-versions=flowcontrol.apiserver.k8s.io/v1beta1 (51 bytes)
70: --api-versions=flowcontrol.apiserver.k8s.io/v1beta1/FlowSchema (62 bytes)
71: --api-versions=flowcontrol.apiserver.k8s.io/v1beta1/PriorityLevelConfiguration (78 bytes)
72: --api-versions=flowcontrol.apiserver.k8s.io/v1beta2 (51 bytes)
73: --api-versions=flowcontrol.apiserver.k8s.io/v1beta2/FlowSchema (62 bytes)
74: --api-versions=flowcontrol.apiserver.k8s.io/v1beta2/PriorityLevelConfiguration (78 bytes)
75: --api-versions=irsa.voodoo.io/v1alpha1 (38 bytes)
76: --api-versions=irsa.voodoo.io/v1alpha1/IamRoleServiceAccount (60 bytes)
77: --api-versions=irsa.voodoo.io/v1alpha1/Policy (45 bytes)
78: --api-versions=irsa.voodoo.io/v1alpha1/Role (43 bytes)
79: --api-versions=k6.io/v1alpha1 (29 bytes)
80: --api-versions=k6.io/v1alpha1/K6 (32 bytes)
81: --api-versions=monitoring.coreos.com/v1 (39 bytes)
82: --api-versions=monitoring.coreos.com/v1/Alertmanager (52 bytes)
83: --api-versions=monitoring.coreos.com/v1/PodMonitor (50 bytes)
84: --api-versions=monitoring.coreos.com/v1/Probe (45 bytes)
85: --api-versions=monitoring.coreos.com/v1/Prometheus (50 bytes)
86: --api-versions=monitoring.coreos.com/v1/PrometheusRule (54 bytes)
87: --api-versions=monitoring.coreos.com/v1/ServiceMonitor (54 bytes)
88: --api-versions=monitoring.coreos.com/v1/ThanosRuler (51 bytes)
89: --api-versions=monitoring.coreos.com/v1alpha1 (45 bytes)
90: --api-versions=monitoring.coreos.com/v1alpha1/AlertmanagerConfig (64 bytes)
91: --api-versions=monitoring.grafana.com/v1alpha1 (46 bytes)
92: --api-versions=monitoring.grafana.com/v1alpha1/GrafanaAgent (59 bytes)
93: --api-versions=monitoring.grafana.com/v1alpha1/Integration (58 bytes)
94: --api-versions=monitoring.grafana.com/v1alpha1/LogsInstance (59 bytes)
95: --api-versions=monitoring.grafana.com/v1alpha1/MetricsInstance (62 bytes)
96: --api-versions=monitoring.grafana.com/v1alpha1/PodLogs (54 bytes)
97: --api-versions=networking.k8s.io/v1 (35 bytes)
98: --api-versions=networking.k8s.io/v1/Ingress (43 bytes)
99: --api-versions=networking.k8s.io/v1/IngressClass (48 bytes)
100: --api-versions=networking.k8s.io/v1/NetworkPolicy (49 bytes)
101: --api-versions=node.k8s.io/v1 (29 bytes)
102: --api-versions=node.k8s.io/v1/RuntimeClass (42 bytes)
103: --api-versions=node.k8s.io/v1beta1 (34 bytes)
104: --api-versions=node.k8s.io/v1beta1/RuntimeClass (47 bytes)
105: --api-versions=policy/v1 (24 bytes)
106: --api-versions=policy/v1/PodDisruptionBudget (44 bytes)
107: --api-versions=policy/v1beta1 (29 bytes)
108: --api-versions=policy/v1beta1/PodDisruptionBudget (49 bytes)
109: --api-versions=policy/v1beta1/PodSecurityPolicy (47 bytes)
110: --api-versions=rbac.authorization.k8s.io/v1 (43 bytes)
111: --api-versions=rbac.authorization.k8s.io/v1/ClusterRole (55 bytes)
112: --api-versions=rbac.authorization.k8s.io/v1/ClusterRoleBinding (62 bytes)
113: --api-versions=rbac.authorization.k8s.io/v1/Role (48 bytes)
114: --api-versions=rbac.authorization.k8s.io/v1/RoleBinding (55 bytes)
115: --api-versions=scheduling.k8s.io/v1 (35 bytes)
116: --api-versions=scheduling.k8s.io/v1/PriorityClass (49 bytes)
117: --api-versions=storage.k8s.io/v1 (32 bytes)
118: --api-versions=storage.k8s.io/v1/CSIDriver (42 bytes)
119: --api-versions=storage.k8s.io/v1/CSINode (40 bytes)
120: --api-versions=storage.k8s.io/v1/StorageClass (45 bytes)
121: --api-versions=storage.k8s.io/v1/VolumeAttachment (49 bytes)
122: --api-versions=storage.k8s.io/v1beta1 (37 bytes)
123: --api-versions=storage.k8s.io/v1beta1/CSIStorageCapacity (56 bytes)
124: --api-versions=v1 (17 bytes)
125: --api-versions=v1/ConfigMap (27 bytes)
126: --api-versions=v1/Endpoints (27 bytes)
127: --api-versions=v1/Event (23 bytes)
128: --api-versions=v1/LimitRange (28 bytes)
129: --api-versions=v1/Namespace (27 bytes)
130: --api-versions=v1/Node (22 bytes)
131: --api-versions=v1/PersistentVolume (34 bytes)
132: --api-versions=v1/PersistentVolumeClaim (39 bytes)
133: --api-versions=v1/Pod (21 bytes)
134: --api-versions=v1/PodTemplate (29 bytes)
135: --api-versions=v1/ReplicationController (39 bytes)
136: --api-versions=v1/ResourceQuota (31 bytes)
137: --api-versions=v1/Secret (24 bytes)
138: --api-versions=v1/Service (25 bytes)
139: --api-versions=v1/ServiceAccount (32 bytes)
140: --api-versions=velero.io/v1 (27 bytes)
141: --api-versions=velero.io/v1/Backup (34 bytes)
142: --api-versions=velero.io/v1/BackupStorageLocation (49 bytes)
143: --api-versions=velero.io/v1/DeleteBackupRequest (47 bytes)
144: --api-versions=velero.io/v1/DownloadRequest (43 bytes)
145: --api-versions=velero.io/v1/PodVolumeBackup (43 bytes)
146: --api-versions=velero.io/v1/PodVolumeRestore (44 bytes)
147: --api-versions=velero.io/v1/ResticRepository (44 bytes)
148: --api-versions=velero.io/v1/Restore (35 bytes)
149: --api-versions=velero.io/v1/Schedule (36 bytes)
150: --api-versions=velero.io/v1/ServerStatusRequest (47 bytes)
151: --api-versions=velero.io/v1/VolumeSnapshotLocation (50 bytes)
152: --api-versions=vpcresources.k8s.aws/v1beta1 (43 bytes)
153: --api-versions=vpcresources.k8s.aws/v1beta1/SecurityGroupPolicy (63 bytes)
ERROR:
exit status 1
EXIT STATUS
1
STDERR:
Error: unknown flag: --kube-version
COMBINED OUTPUT:
Error: unknown flag: --kube-version] 2023-03-03 10:31:41 +0100 CET
GROUP KIND NAMESPACE NAME STATUS HEALTH HOOK MESSAGE
Secret ephemeral-pull-helmfile ephemeral-postgres Unknown
Service ephemeral-pull-helmfile ephemeral Unknown Healthy
Service ephemeral-pull-helmfile ephemeral-postgres Unknown Healthy
Service ephemeral-pull-helmfile ephemeral-postgres-hl Unknown Healthy
ServiceAccount ephemeral-pull-helmfile ephemeral Unknown
apps Deployment ephemeral-pull-helmfile ephemeral Unknown Healthy
apps StatefulSet ephemeral-pull-helmfile ephemeral-postgres Unknown Healthy
batch Job ephemeral-pull-helmfile ephemeral-db-migrate Unknown Healthy
networking.k8s.io Ingress ephemeral-pull-helmfile ephemeral Unknown Healthy
policy PodDisruptionBudget ephemeral-pull-helmfile ephemeral Unknown
Can you send over the app definition so I can see what you have set?
Or the helmfile content if you have the args defined there.
Like I'm using multiple yaml in helmfile.d directory, I don't have a helmfile.yaml. This is my helmfile yaml :
bases:
releases:
name: db chart: bitnami/postgresql version: 12.1.0 values:
name: db-migration chart: ../charts/db-migration version: 0.1.0 values:
01_application.yaml
bases:
../common/repositories.yaml
../common/environments.yaml
releases:
And my app definition :
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata:
name: ephemeral-as
spec:
generators:
- pullRequest:
github:
owner: VoodooTeam
repo: ephemeral-env
tokenRef:
secretName: argocd-github-secret
key: pat
requeueAfterSeconds: 1800
filters:
- branchMatch: pull-.*
template:
metadata:
name: ephemeral-{{branch}}
spec:
source:
repoURL: https://github.com/xxx/argocd-sandbox.git
targetRevision: master
path: ephemeral/
plugin:
parameters:
- name: ephemeral_values
map:
image.tag: "{{head_sha}}"
ingress.host: "{{branch_slug}}.xxx.xx"
environment.CSRF_TRUSTED_ORIGINS: "https://{{branch_slug}}.xxx.xx"
project: infra
destination:
name: testbed
namespace: ephemeral-{{branch}}
syncPolicy:
automated:
prune: true
selfHeal: true
syncOptions:
- CreateNamespace=true
I suspect this has some behavior difference due to the repository being oci
. Conceptually the goal is to make sure all repos are added and necessary charts downloads during the init
phase (which occurs before generate
phase). The error above is occurring during the generate
phase which leads me to believe something fundamentally different with how helmfile handles oci vs non-oci registries. Can you send the (cleansed) content of the repositories.yaml
file?
I've opened this: https://github.com/helmfile/helmfile/discussions/727
Sorry for the delay The fiie :
repositories:
- name: bitnami
url: https://charts.bitnami.com/bitnami
- name: localstack
url: https://localstack.github.io/helm-charts
- name: ecr
url: xxx.dkr.ecr.eu-west-1.amazonaws.com/helm
oci: true
@travisghansen I have the same issue with oci repositories. I did some investigation and can say:
helmfile before pass template
command passing pull
command for oci repositories to helm.
helm 'pull' command doesn't support --kube-version and --api-versions arguments..
also I checked 'fetch' command before 'template' command. it is doesn't help. Template still pulling charts.
you are always passing this arguments to 'template' command:
# TODO: support post process pipeline here
${helmfile} \
template \
--skip-deps ${INTERNAL_HELMFILE_TEMPLATE_OPTIONS} \
--args "${INTERNAL_HELM_TEMPLATE_OPTIONS} ${HELM_TEMPLATE_OPTIONS}" \
${HELMFILE_TEMPLATE_OPTIONS}
;;
So, my suggestion is make this arguments are optional, or add a new ENV to possibility disable this args.
@welderpb thanks! I'm unaware of another option to ensure proper kubeapi versions etc can be passed down to helm, am I missing something?
@travisghansen it is not necessary to pass this arguments to helm templating, it is should be optional.
In many situations they absolutely are required unfortunately :( we’ll work closely with helmfile to make sure we sanely cover the necessary use cases and the issue will go away.
Hello,
I'm experiencing a strange behavior when I'm using helmfile.d tree structure. Some args are passed to helm pull command but they shouldn't (cf output below)
It seems passing args from helmfile to helm is not a good practice (doc)
Do you think it could be improved or make it optional ? For now, I removed the --args parameter from the script like I'm not using it.