search

bitnami / readme-generator-for-helm

Auto generate READMEs for Helm Charts
https://bitnami.com
Apache License 2.0
212 stars 48 forks source link

some bitnami charts fail to render the readme or metadata #21

Closed vrabbi closed 2 years ago

vrabbi commented 2 years ago

the following bitnami charts fail to render via this project due to multiple issues: "KEY_OVER_1024_CHARS":

  1. mongodb
  2. grafana-operator

"Wrong Metadata":

  1. mariadb-cluster
  2. mean
  3. prometheus-operator
  4. sugarcrm
  5. tensorflow-inception

the wrong metadata seems to be because these charts do not have any metadata in them used by this project which i think is an issue as it should be standard in all bitnami charts. the key over 1024 chars issue seems to be an underlying YAML parser issue but should be considered how to fix in this project or in the chart as its a bad UX when not all charts render as expected

miguelaeh commented 2 years ago

Hi @vrabbi , Are you using the last version of this tool? Could you share more information on how to obtain that error? I just executed the tool for all the charts, and everything worked as expected: https://github.com/bitnami/charts/pull/7790

vrabbi commented 2 years ago 
root@Scott-scott-0354:/tmp# readme-generator -v /tmp/charts/prometheus-operator/values.yaml --metadata a.json
INFO: Checking missing metadata...

######
The following errors must be fixed before proceeding
######

ERROR: Missing metadata for key: global.labels
ERROR: Missing metadata for key: rbac.create
ERROR: Missing metadata for key: rbac.apiVersion
ERROR: Missing metadata for key: rbac.pspEnabled
ERROR: Missing metadata for key: operator.enabled
ERROR: Missing metadata for key: operator.image.registry
ERROR: Missing metadata for key: operator.image.repository
ERROR: Missing metadata for key: operator.image.tag
ERROR: Missing metadata for key: operator.image.pullPolicy
ERROR: Missing metadata for key: operator.serviceAccount.create
ERROR: Missing metadata for key: operator.securityContext.enabled
ERROR: Missing metadata for key: operator.securityContext.runAsUser
ERROR: Missing metadata for key: operator.securityContext.fsGroup
ERROR: Missing metadata for key: operator.service.type
ERROR: Missing metadata for key: operator.service.port
ERROR: Missing metadata for key: operator.service.annotations
ERROR: Missing metadata for key: operator.createCustomResource
ERROR: Missing metadata for key: operator.serviceMonitor.enabled
ERROR: Missing metadata for key: operator.serviceMonitor.interval
ERROR: Missing metadata for key: operator.serviceMonitor.metricRelabelings
ERROR: Missing metadata for key: operator.serviceMonitor.relabelings
ERROR: Missing metadata for key: operator.resources
ERROR: Missing metadata for key: operator.podAnnotations
ERROR: Missing metadata for key: operator.nodeAffinity
ERROR: Missing metadata for key: operator.podAntiAffinity
ERROR: Missing metadata for key: operator.podAffinity
ERROR: Missing metadata for key: operator.nodeSelector
ERROR: Missing metadata for key: operator.tolerations
ERROR: Missing metadata for key: operator.priorityClassName
ERROR: Missing metadata for key: operator.livenessProbe.initialDelaySeconds
ERROR: Missing metadata for key: operator.livenessProbe.periodSeconds
ERROR: Missing metadata for key: operator.livenessProbe.timeoutSeconds
ERROR: Missing metadata for key: operator.livenessProbe.failureThreshold
ERROR: Missing metadata for key: operator.livenessProbe.successThreshold
ERROR: Missing metadata for key: operator.readinessProbe.initialDelaySeconds
ERROR: Missing metadata for key: operator.readinessProbe.periodSeconds
ERROR: Missing metadata for key: operator.readinessProbe.timeoutSeconds
ERROR: Missing metadata for key: operator.readinessProbe.failureThreshold
ERROR: Missing metadata for key: operator.readinessProbe.successThreshold
ERROR: Missing metadata for key: operator.logLevel
ERROR: Missing metadata for key: operator.logFormat
ERROR: Missing metadata for key: operator.kubeletService.enabled
ERROR: Missing metadata for key: operator.kubeletService.namespace
ERROR: Missing metadata for key: operator.configmapReload.image.registry
ERROR: Missing metadata for key: operator.configmapReload.image.repository
ERROR: Missing metadata for key: operator.configmapReload.image.tag
ERROR: Missing metadata for key: operator.prometheusConfigReloader.image
ERROR: Missing metadata for key: prometheus.enabled
ERROR: Missing metadata for key: prometheus.image.registry
ERROR: Missing metadata for key: prometheus.image.repository
ERROR: Missing metadata for key: prometheus.image.tag
ERROR: Missing metadata for key: prometheus.serviceAccount.create
ERROR: Missing metadata for key: prometheus.securityContext.enabled
ERROR: Missing metadata for key: prometheus.securityContext.runAsUser
ERROR: Missing metadata for key: prometheus.securityContext.fsGroup
ERROR: Missing metadata for key: prometheus.podDisruptionBudget.enabled
ERROR: Missing metadata for key: prometheus.podDisruptionBudget.minAvailable
ERROR: Missing metadata for key: prometheus.service.type
ERROR: Missing metadata for key: prometheus.service.port
ERROR: Missing metadata for key: prometheus.service.annotations
ERROR: Missing metadata for key: prometheus.serviceMonitor.enabled
ERROR: Missing metadata for key: prometheus.serviceMonitor.interval
ERROR: Missing metadata for key: prometheus.serviceMonitor.metricRelabelings
ERROR: Missing metadata for key: prometheus.serviceMonitor.relabelings
ERROR: Missing metadata for key: prometheus.ingress.enabled
ERROR: Missing metadata for key: prometheus.ingress.certManager
ERROR: Missing metadata for key: prometheus.ingress.annotations
ERROR: Missing metadata for key: prometheus.ingress.hosts[0].name
ERROR: Missing metadata for key: prometheus.ingress.hosts[0].path
ERROR: Missing metadata for key: prometheus.ingress.tls
ERROR: Missing metadata for key: prometheus.resources
ERROR: Missing metadata for key: prometheus.nodeAffinity
ERROR: Missing metadata for key: prometheus.podAntiAffinity
ERROR: Missing metadata for key: prometheus.podAntiAffinityTopologyKey
ERROR: Missing metadata for key: prometheus.podAffinity
ERROR: Missing metadata for key: prometheus.nodeSelector
ERROR: Missing metadata for key: prometheus.tolerations
ERROR: Missing metadata for key: prometheus.scrapeInterval
ERROR: Missing metadata for key: prometheus.evaluationInterval
ERROR: Missing metadata for key: prometheus.listenLocal
ERROR: Missing metadata for key: prometheus.enableAdminAPI
ERROR: Missing metadata for key: prometheus.alertingEndpoints
ERROR: Missing metadata for key: prometheus.externalLabels
ERROR: Missing metadata for key: prometheus.replicaExternalLabelName
ERROR: Missing metadata for key: prometheus.replicaExternalLabelNameClear
ERROR: Missing metadata for key: prometheus.prometheusExternalLabelName
ERROR: Missing metadata for key: prometheus.prometheusExternalLabelNameClear
ERROR: Missing metadata for key: prometheus.secrets
ERROR: Missing metadata for key: prometheus.configMaps
ERROR: Missing metadata for key: prometheus.querySpec
ERROR: Missing metadata for key: prometheus.ruleNamespaceSelector
ERROR: Missing metadata for key: prometheus.ruleSelector
ERROR: Missing metadata for key: prometheus.serviceMonitorSelector
ERROR: Missing metadata for key: prometheus.serviceMonitorNamespaceSelector
ERROR: Missing metadata for key: prometheus.podMonitorSelector
ERROR: Missing metadata for key: prometheus.podMonitorNamespaceSelector
ERROR: Missing metadata for key: prometheus.retention
ERROR: Missing metadata for key: prometheus.retentionSize
ERROR: Missing metadata for key: prometheus.disableCompaction
ERROR: Missing metadata for key: prometheus.walCompression
ERROR: Missing metadata for key: prometheus.paused
ERROR: Missing metadata for key: prometheus.replicaCount
ERROR: Missing metadata for key: prometheus.logLevel
ERROR: Missing metadata for key: prometheus.logFormat
ERROR: Missing metadata for key: prometheus.podMetadata.labels
ERROR: Missing metadata for key: prometheus.podMetadata.annotations
ERROR: Missing metadata for key: prometheus.remoteRead
ERROR: Missing metadata for key: prometheus.remoteWrite
ERROR: Missing metadata for key: prometheus.storageSpec
ERROR: Missing metadata for key: prometheus.persistence.enabled
ERROR: Missing metadata for key: prometheus.persistence.accessModes
ERROR: Missing metadata for key: prometheus.persistence.size
ERROR: Missing metadata for key: prometheus.priorityClassName
ERROR: Missing metadata for key: prometheus.containers
ERROR: Missing metadata for key: prometheus.volumes
ERROR: Missing metadata for key: prometheus.volumeMounts
ERROR: Missing metadata for key: prometheus.additionalPrometheusRules
ERROR: Missing metadata for key: prometheus.additionalScrapeConfigsExternal.enabled
ERROR: Missing metadata for key: prometheus.additionalAlertRelabelConfigsExternal.enabled
ERROR: Missing metadata for key: prometheus.thanos.create
ERROR: Missing metadata for key: prometheus.thanos.image.registry
ERROR: Missing metadata for key: prometheus.thanos.image.repository
ERROR: Missing metadata for key: prometheus.thanos.image.tag
ERROR: Missing metadata for key: prometheus.thanos.image.pullPolicy
ERROR: Missing metadata for key: prometheus.thanos.extraArgs
ERROR: Missing metadata for key: prometheus.thanos.objectStorageConfig
ERROR: Missing metadata for key: prometheus.thanos.resources.limits
ERROR: Missing metadata for key: prometheus.thanos.resources.requests
ERROR: Missing metadata for key: prometheus.thanos.service.type
ERROR: Missing metadata for key: prometheus.thanos.service.port
ERROR: Missing metadata for key: prometheus.thanos.service.annotations
ERROR: Missing metadata for key: prometheus.thanos.service.extraPorts
ERROR: Missing metadata for key: alertmanager.enabled
ERROR: Missing metadata for key: alertmanager.image.registry
ERROR: Missing metadata for key: alertmanager.image.repository
ERROR: Missing metadata for key: alertmanager.image.tag
ERROR: Missing metadata for key: alertmanager.serviceAccount.create
ERROR: Missing metadata for key: alertmanager.securityContext.enabled
ERROR: Missing metadata for key: alertmanager.securityContext.runAsUser
ERROR: Missing metadata for key: alertmanager.securityContext.fsGroup
ERROR: Missing metadata for key: alertmanager.podDisruptionBudget.enabled
ERROR: Missing metadata for key: alertmanager.podDisruptionBudget.minAvailable
ERROR: Missing metadata for key: alertmanager.service.type
ERROR: Missing metadata for key: alertmanager.service.port
ERROR: Missing metadata for key: alertmanager.service.annotations
ERROR: Missing metadata for key: alertmanager.serviceMonitor.enabled
ERROR: Missing metadata for key: alertmanager.serviceMonitor.interval
ERROR: Missing metadata for key: alertmanager.serviceMonitor.metricRelabelings
ERROR: Missing metadata for key: alertmanager.serviceMonitor.relabelings
ERROR: Missing metadata for key: alertmanager.ingress.enabled
ERROR: Missing metadata for key: alertmanager.ingress.certManager
ERROR: Missing metadata for key: alertmanager.ingress.annotations
ERROR: Missing metadata for key: alertmanager.ingress.hosts[0].name
ERROR: Missing metadata for key: alertmanager.ingress.hosts[0].path
ERROR: Missing metadata for key: alertmanager.ingress.tls
ERROR: Missing metadata for key: alertmanager.resources
ERROR: Missing metadata for key: alertmanager.nodeAffinity
ERROR: Missing metadata for key: alertmanager.podAntiAffinity
ERROR: Missing metadata for key: alertmanager.podAntiAffinityTopologyKey
ERROR: Missing metadata for key: alertmanager.podAffinity
ERROR: Missing metadata for key: alertmanager.nodeSelector
ERROR: Missing metadata for key: alertmanager.tolerations
ERROR: Missing metadata for key: alertmanager.config.global.resolve_timeout
ERROR: Missing metadata for key: alertmanager.config.route.group_by
ERROR: Missing metadata for key: alertmanager.config.route.group_wait
ERROR: Missing metadata for key: alertmanager.config.route.group_interval
ERROR: Missing metadata for key: alertmanager.config.route.repeat_interval
ERROR: Missing metadata for key: alertmanager.config.route.receiver
ERROR: Missing metadata for key: alertmanager.config.route.routes[0].match.alertname
ERROR: Missing metadata for key: alertmanager.config.route.routes[0].receiver
ERROR: Missing metadata for key: alertmanager.config.receivers[0].name
ERROR: Missing metadata for key: alertmanager.externalConfig
ERROR: Missing metadata for key: alertmanager.replicaCount
ERROR: Missing metadata for key: alertmanager.logLevel
ERROR: Missing metadata for key: alertmanager.logFormat
ERROR: Missing metadata for key: alertmanager.podMetadata.labels
ERROR: Missing metadata for key: alertmanager.podMetadata.annotations
ERROR: Missing metadata for key: alertmanager.secrets
ERROR: Missing metadata for key: alertmanager.configMaps
ERROR: Missing metadata for key: alertmanager.retention
ERROR: Missing metadata for key: alertmanager.storageSpec
ERROR: Missing metadata for key: alertmanager.persistence.enabled
ERROR: Missing metadata for key: alertmanager.persistence.accessModes
ERROR: Missing metadata for key: alertmanager.persistence.size
ERROR: Missing metadata for key: alertmanager.paused
ERROR: Missing metadata for key: alertmanager.listenLocal
ERROR: Missing metadata for key: alertmanager.containers
ERROR: Missing metadata for key: alertmanager.volumes
ERROR: Missing metadata for key: alertmanager.volumeMounts
ERROR: Missing metadata for key: alertmanager.priorityClassName
ERROR: Missing metadata for key: alertmanager.additionalPeers
ERROR: Missing metadata for key: exporters.node-exporter.enabled
ERROR: Missing metadata for key: exporters.kube-state-metrics.enabled
ERROR: Missing metadata for key: node-exporter.service.labels.jobLabel
ERROR: Missing metadata for key: node-exporter.serviceMonitor.enabled
ERROR: Missing metadata for key: node-exporter.serviceMonitor.jobLabel
ERROR: Missing metadata for key: node-exporter.extraArgs.collector.filesystem.ignored-mount-points
ERROR: Missing metadata for key: node-exporter.extraArgs.collector.filesystem.ignored-fs-types
ERROR: Missing metadata for key: kube-state-metrics.serviceMonitor.enabled
ERROR: Missing metadata for key: kubeApiServer.enabled
ERROR: Missing metadata for key: kubeApiServer.serviceMonitor.interval
ERROR: Missing metadata for key: kubeApiServer.serviceMonitor.metricRelabelings
ERROR: Missing metadata for key: kubeApiServer.serviceMonitor.relabelings
ERROR: Missing metadata for key: kubeProxy.enabled
ERROR: Missing metadata for key: kubeProxy.endpoints
ERROR: Missing metadata for key: kubeProxy.namespace
ERROR: Missing metadata for key: kubeProxy.service.enabled
ERROR: Missing metadata for key: kubeProxy.service.port
ERROR: Missing metadata for key: kubeProxy.service.targetPort
ERROR: Missing metadata for key: kubeProxy.serviceMonitor.https
ERROR: Missing metadata for key: kubeProxy.serviceMonitor.interval
ERROR: Missing metadata for key: kubeProxy.serviceMonitor.metricRelabelings
ERROR: Missing metadata for key: kubeProxy.serviceMonitor.relabelings
ERROR: Missing metadata for key: kubelet.enabled
ERROR: Missing metadata for key: kubelet.namespace
ERROR: Missing metadata for key: kubelet.serviceMonitor.https
ERROR: Missing metadata for key: kubelet.serviceMonitor.interval
ERROR: Missing metadata for key: kubelet.serviceMonitor.metricRelabelings
ERROR: Missing metadata for key: kubelet.serviceMonitor.relabelings
ERROR: Missing metadata for key: kubelet.serviceMonitor.cAdvisorMetricRelabelings
ERROR: Missing metadata for key: kubelet.serviceMonitor.cAdvisorRelabelings
/root/readme-generator-for-helm/lib/checker.js:92
    throw new Error('ERROR: Wrong metadata!');
    ^

Error: ERROR: Wrong metadata!
    at checkKeys (/root/readme-generator-for-helm/lib/checker.js:92:11)
    at getValuesSections (/root/readme-generator-for-helm/index.js:23:3)
    at runReadmeGenerator (/root/readme-generator-for-helm/index.js:45:20)
    at Object.<anonymous> (/root/readme-generator-for-helm/bin/index.js:21:1)
    at Module._compile (internal/modules/cjs/loader.js:1085:14)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1114:10)
    at Module.load (internal/modules/cjs/loader.js:950:32)
    at Function.Module._load (internal/modules/cjs/loader.js:790:12)
    at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:76:12)
    at internal/main/run_main_module.js:17:47
vrabbi commented 2 years ago

that is after updating npm and node to the latest stable versions as well as installing the latest version of this tool for all othe bitnami charts this works. its the ones i mentioned above that fail

vrabbi commented 2 years ago

and this is the error in full for mongodb:

root@Scott-scott-0354:/tmp# readme-generator -v /tmp/charts/mongodb/values.yaml --metadata a.json
/root/readme-generator-for-helm/node_modules/yaml/dist/public-api.js:75
            throw doc.errors[0];
            ^

YAMLParseError: The : indicator must be at most 1024 chars after the start of an implicit block mapping key at line 916, column 1:

##
volumePermissions:
^^^^^^^^^^^^^^^^^

    at Composer.onError (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/composer.js:71:34)
    at Object.resolveBlockMap (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/resolve-block-map.js:69:21)
    at Object.composeCollection (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/compose-collection.js:13:36)
    at Object.composeNode (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/compose-node.js:30:38)
    at Object.composeDoc (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/compose-doc.js:31:23)
    at Composer.next (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/composer.js:152:40)
    at next (<anonymous>)
    at Composer.compose (/root/readme-generator-for-helm/node_modules/yaml/dist/compose/composer.js:134:25)
    at compose.next (<anonymous>)
    at parseDocument (/root/readme-generator-for-helm/node_modules/yaml/dist/public-api.js:47:16) {
  code: 'KEY_OVER_1024_CHARS',
  pos: [ 35873, 35890 ],
  linePos: [ { line: 916, col: 1 }, { line: 916, col: 18 } ]
}
miguelaeh commented 2 years ago

Hi @vrabbi , We dno't have a prometheus-operator chart. It is called kube-prometheus, so it is possible that you have a really old version of the charts repository, or that chart was not renamed in your local repo. Could you please update your charts repo and try again?

Also, for MongoDB I am unable to reproduce it, this is my output:

7:38:27 › readme-generator -v values.yaml -r README.md
Skipping check for metrics.service.annotations.prometheus.io/scrape
Skipping check for metrics.service.annotations.prometheus.io/port
Skipping check for metrics.service.annotations.prometheus.io/path
INFO: Checking missing metadata...
INFO: Metadata is correct!
INFO: Found parameters section at: ## Parameters
INFO: Found section end at: Specify each parameter using the `--set key=value[,key=value]` argument to `helm install`. For example,
INFO: Inserting the new table into the README...

Could you share more details on how to reproduce it?

vrabbi commented 2 years ago

so the mongodb issue was a bad node installation i had which the commander module was broken in. regardin the other charts, the issue is that they are still in the chart repo. they have been removed from github but if you add the repo and run a search they are still there. i have been doing some automation around generating carvel packages from the bitnami charts and was doing the helm search programitcally against the repo. is there a way in helm search to not show the bitnami deprecated charts? are the bitnami team planning on removing those charts completely?

miguelaeh commented 2 years ago

Hi @vrabbi ,] We marc them as DEPRECATED in the index, for example:

bitnami/mean                                    6.1.2           4.6.2           DEPRECATED MEAN is a free and open-source JavaS...

but we don't remove them. Isn't it possible for you to search the charts on the GH repo?

MikaelElkiaer commented 2 years ago

so the mongodb issue was a bad node installation i had which the commander module was broken in. regardin the other charts, the issue is that they are still in the chart repo. they have been removed from github but if you add the repo and run a search they are still there. i have been doing some automation around generating carvel packages from the bitnami charts and was doing the helm search programitcally against the repo. is there a way in helm search to not show the bitnami deprecated charts? are the bitnami team planning on removing those charts completely?

I am facing the same issue right now. What exactly did you do to fix your node installation?

miguelaeh commented 2 years ago

Hi @MikaelElkiaer , You can delete the node_modules directory and execute npm install again from inside the directory where you cloned the tool.

MikaelElkiaer commented 2 years ago

Hi @MikaelElkiaer , You can delete the node_modules directory and execute npm install again from inside the directory where you cloned the tool.

Thanks for the reply.

I ended up uninstalling the globally installed version and cloning and running instead.

miguelaeh commented 2 years ago

Since the issue is solved I will close the issue