Helm Capabilities built-in object is not supported by repository server

[alexmt]

Capabilities built-in helm object contains information about capabilities which target K8s cluster supports. Argo CD uses helm template to generate chart manifests without specifying the target cluster version. As a result Capabilities field is always empty and charts which depend on it cannot work properly.

The helm template is going to get --api-version flag in https://github.com/helm/helm/pull/5392 . Argo CD should leverage --api-version flag as soon as PR is merged.

[alexec]

I should probably look into this as I'm doing a Helm refactoring.

[alexec]

Customer benefit - they will be able to use charts with capabilities.

[alexec]

This is currently blocked.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[alexec]

Blocked waiting on an upstream fix.

[jessesuen]

Just ran into this with the prometheus-operator chart. Without this fix, prometheus dashboards are not loaded. helm template now has --kube-version string which we need to run as part of helm template.

$ helm template --help

Render chart templates locally and display the output.

This does not require Tiller. However, any values that would normally be
looked up or retrieved in-cluster will be faked locally. Additionally, none
of the server-side testing of chart validity (e.g. whether an API is supported)
is done.

To render just one template in a chart, use '-x':

    $ helm template mychart -x templates/deployment.yaml

Usage:
  helm template [flags] CHART

Flags:
  -x, --execute stringArray      only execute the given templates
  -h, --help                     help for template
      --is-upgrade               set .Release.IsUpgrade instead of .Release.IsInstall
      --kube-version string      kubernetes version used as Capabilities.KubeVersion.Major/Minor (default "1.9")

[alexec]

@jessesuen I think we have two issues here. One for --kube-version and one for --api-version. The latter depends on a Helm upstream fix. I'll create a new issue.

[EppO]

FYI, https://github.com/helm/helm/pull/5392 has been merged! Using --kube-version and now --api-version, ArgoCD has all required information to process correctly Helm Charts that leverages Capabilities Object.

[alexec]

Now we need to upgrade the Helm version.

[EppO]

Now that #2587 upgraded Helm to 2.15.2 that supports API versions, can we put this issue in milestone 1.4 as well? That would improve support of many Helm Charts.

[dimitarKiryakov]

Hi @alexec
When we could expect this feature to be released?

[jessesuen]

This should be in 1.4

[pfyod]

Is this still planned for 1.4(.x?)? Lots of resources got deprecated in k8s 1.16 and helm charts that rely on .Capabilities.APIVersions.Has do not work as expected.

[EppO]

I see that feature is coming after ArgoCD v1.5 which introduces Helm 3 support. The thing is, Helm 3 template command is not on-par with Helm 2 template, it's missing the --kube-version parameter. So we're back to square one on this, there is a bunch of Charts in stable (65 as of today) that relies on Capabilities.KubeVersion.GitVersion object:

cluster-autoscaler
cluster-overprovisioner
cockroachdb
concourse
consul
datadog
dokuwiki
drone
drupal
elasticsearch-curator
eventrouter
external-dns
fluent-bit
ghost
hackmd
jasperreports
joomla
kibana
kube-slack
kube2iam
kuberhealthy
kubewatch
kured
lamp
magento
mariadb
mediawiki
mercure
metabase
minio
moodle
mssql-linux
nats
nfs-client-provisioner
nginx-ingress
node-problem-detector
odoo
opencart
orangehrm
osclass
owncloud
pgadmin
phabricator
phpbb
postgresql
prestashop
prometheus
prometheus-blackbox-exporter
prometheus-cloudwatch-exporter
prometheus-operator
prometheus-pushgateway
redis
redmine
rocketchat
sealed-secrets
selenium
seq
spark
spotify-docker-gc
suitecrm
testlink
traefik
weave-cloud
weave-scope
wordpress

[alexmt]

Helm2 is still supported. ArgoCD is using Helm3 binary only if chart has apiVersion: v2.

[pfyod]

@alexmt @EppO As far as I understand, even Helm2 in ArgoCD still does not properly support the Capabilities object and many of the charts do not work because of that. One needs to explicitly set both --kube-version and --api-versions (for all resources) to be able to support both Capabilities.KubeVersion (--kube-version) and Capabilities.APIVersions (--api-versions). These are the "stable" charts that use Capabilities.APIVersions (and thus not really installable via ArgoCD in all cases):

chaoskube coredns envoy filebeat fluent-bit gangway grafana jenkins kong minio nginx-ingress opa pgadmin prometheus prometheus-cloudwatch-exporter prometheus-nats-exporter prometheus-operator prometheus-postgres-exporter prometheus-redis-exporter redis-ha reloader rethinkdb sematext-agent sematext-docker-agent sonarqube

[EppO]

You're right and there are big boys in that list (prometheus*, grafana, nginx-ingress, fluentbit...). For now, the only workaround I found is to fork the Helm Charts and remove the Capabilities checks to make it run using ArgoCD. Can't wait to drop these forks though...

[alexmt]

PR https://github.com/argoproj/argo-cd/pull/3243 adds --api-versions flag support to both helm2 and helm3