indentation error when converting defectdojo

[marcelser]

I tried to download and convert local helm chart downloaded from product page here: https://github.com/DefectDojo/django-DefectDojo/blob/master/KUBERNETES.md

When converting the helm chart it it creates an error in the section below, where "requests" on the same level as "resources" but it must be indented as "requests" is subkey of resource. Interestingly on other blocks which have exactly the same dfinition it works perfectly Here's the wrong code block:

apiVersion: batch/v1
kind: Job
metadata:
  name: defectdojo-initializer
  labels:
    defectdojo.org/component: initializer
    app.kubernetes.io/name: defectdojo
    app.kubernetes.io/instance: defectdojo
    app.kubernetes.io/managed-by: Tiller
    helm.sh/chart: defectdojo-0.1.0
spec:
  ttlSecondsAfterFinished: 
  template:
    metadata:
      labels:
        defectdojo.org/component: initializer
        app.kubernetes.io/name: defectdojo
        app.kubernetes.io/instance: defectdojo
    spec:
      containers:
        - name: initializer
          image: "defectdojo/defectdojo-django:latest"
          imagePullPolicy: Always
          command: ['/entrypoint-initializer.sh']
          envFrom:
            - configMapRef:
                name: defectdojo
            - secretRef:
                name: defectdojo
          env:
            - name: DD_DATABASE_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: defectdojo-mysql
                  key: mysql-password
          resources:
          requests:
            cpu: 100m
            memory: 128Mi

      restartPolicy: Never
  backoffLimit: 1

[JeanMertz]

Hey @marcelser, thank you for the report.

Could you provide a minimal reproducible example so that I can test this locally and hopefully fix the problem?

[marcelser]

Ok, I'll try to make it as simple as possible. I'm trying to convert defectdojo which only has instructions for a helm installation to convert to K8S deployment. I've attached the "helm" directory of the project as the attached "helm.zip" which I hope is enough.

Otherwise you can git clone the whole project from here: https://github.com/DefectDojo/django-DefectDojo

here's the attached helm directoy. And I the cofig I use a relative path to it helm.zip

# apiVersion defines the version of the charts.yaml structure. Currently,
# only "v1" is supported.
apiVersion: v1

# name is the .Release.Name template value that charts can use in their
# templates, which can be overridden by the "--name" CLI flag. If omitted,
# "--name" is required.
name: defectdojo

# namespace is the .Release.Namespace template value that charts can use in
# their templates. Note that since kubecrt does not communicate with
# Kubernetes in any way, it is up to you to also use this namespace when
# doing kubectl apply [...]. Can be overridden using "--namespace".  If omitted,
# "--namespace" is required.
namespace: defectdojo

# charts is an array of charts you want to compile into Kubernetes resource
# files.
#
# A single chart might be used to deploy something simple, like a memcached pod,
# or something complex, like a full web app stack with HTTP servers, databases,
# caches, and so on.
charts:

# A Chart can either be in the format REPO/NAME, or a PATH to a local chart.
#
# If using REPO/NAME, kubecrt knows by-default where to locate the "stable"
# repository, all other repositories require the "repo" configuration (see
# below).
- ./django-DefectDojo/helm/defectdojo:
    values:

Also when I fix the indentation error another problem with this is also that all deployed parts show CreateContainerConfigError in K8s. I'm currently trying to find out what's wrong. But it seems the conversion is not done properly or there are some values that needs to be set or else it wouldn't work.

[JeanMertz]

Thank you for the example. I won't have time to try this today, but will do so tomorrow.

As for this part:

Also when I fix the indentation error another problem with this is also that all deployed parts show CreateContainerConfigError in K8s

Looking at your last two lines of your example:

- ./django-DefectDojo/helm/defectdojo:
    values:

You are not actually providing any values to be used by the chart. If you look at the chart's values.yml file, you can see there are many configuration values you can set, so perhaps you need to set some of those first?

In any case, I will report back tomorrow with whatever I could reproduce with your example.

[marcelser]

Ah ok I see thanks for the explanation but I still have 2 questions regarding values.yml if I may. The problem is a bit that I know k8s quite well but I never used helm at all but lots of packages come only as helm charts so please forgive me for asking:

1.) Are any of those values automatically used in conversion to k8s (defaults?) or are all values in values.yml ignored 2.) If I want to set/override (in case defaults exist) any those, is the values section in the crt template exactly as the structure of the values.yml (for key & value names and tree-structure (indentation))?

And thanks a lot for trying out why indentation level is wrong in just one generated section whereas other sections contain exactly the same cpu/memory sections where the indentation is correct. I guess it's just one type of K8S resource where it's wrong.

[JeanMertz]

At this point I should probably suggest you read the (excellent) Helm documentation to understand how Helm charts work. Kubecrt is only a small wrapper around Helm, so anything documented in there that does not involve the actual helm CLI is applicable to Kubecrt.

To give you two quick answers to your questions:

1. Anything in values.yml is used as the default values
2. Yes

[marcelser]

Hi. I debugged this problem a bit further by installing helm locally and run it with --dry-run --debug which also outputs what would be installed onto my minikube cluster. I then compared output of kubecrt against helm output and it turns out a lot of k8s resources are missing. All of the missing ones are in subdirectorires inside the charts directory. Here a few examples as helm --dry-run also lists the corresponding files: defectdojo/charts/mysql/templates/secrets.yaml defectdojo/charts/mysql/templates/tests/test-configmap.yaml defectdojo/charts/rabbitmq/templates/configuration.yaml defectdojo/charts/mysql/templates/pvc.yaml etc.

Can I somehow make it work by changing the config file for kubecrt or is this a shortcoming of kubecrt?

Of course I would still like to use kubecrt as I don't want to install helm on our pipelines host especially not on production pipeline but at the moment apart from the indentation error this is a much bigger problem which makes the output not deployable at all.

I have attached the output of kubecrt and of helm dry-run (NOTE: kubecrt output was rearranged to match the order in helm)

converted_defectdojo_charts.zip

[JeanMertz]

Quick FYI that I didn't get around to this yet. I've still got it on my list of things to get done this week though.

[marcelser]

No problem. I'm not in such a hurry.

But a quick hint to the indentation problem. I think kubecrt works fine. I got the same error als in the helm generated templates So I guess there's some error in the helm templates.

But anway the much bigger problem is that kubecrt doesn't process subdirs and makes the resulting output non-deployable

[JeanMertz]

I agree, that is a big problem. We've never encountered charts that did this, which is why this flew under the radar for so long. I'll look into this soon.

Thank you for reporting back on the indentation issue, glad it's not a bug in Kubecrt, your best step to solve that would be to send a PR to the relevant chart repo.