quintush / helm-unittest

BDD styled unit test framework for Kubernetes Helm charts as a Helm plugin.
MIT License
345 stars 69 forks source link

[bug] - contains fails when expected and actual content is the same #174

Closed froy001 closed 1 year ago

froy001 commented 2 years ago

The template:

 volumes:
        - name: config-volume
          configMap:
            name: davf-app-02
        - name: all-vault
          projected:
            sources:
              - secret:
                  name: common-vault
        - configMap:
            name: foo-configMap-2
          name: foo-configmap-volume-2
        - name: foo-secret-volume-2
          secret:
            items:
            - key: foo-key-2
              path: foo-key.txt
            secretName: 'dev-foo-secret-2'

The test:

      - contains:
          path: spec.template.spec.volumes[3].secret.items
          content:
            key: foo-key-2 
            path: foo-key.txt 
          count: 1
          any: true

The failure:

- asserts[1] `contains` fail
     Template:  charts-integration-test/charts/app-02/templates/deployment.yaml
      DocumentIndex:    0
      Path: spec.template.spec.volumes[3].secret.items
      Expected to contain:
      - key: foo-key-2
        path: foo-key.txt
       Actual:
      - key: foo-key-2
        path: foo-key.txt

Obviously this should pass.

quintush commented 1 year ago

Hello @froy001,

I would agree, however I have seen this behavior earlier. I will try to reproduce this issue.

Greetings, @quintush

cristiadu commented 1 year ago

Had a similar issue, but in my case I do have other elements in the array.

            Template:   my-chart/templates/network/gateway.yaml
            DocumentIndex:  0
            Path:   spec.servers
            Expected to contain:
                - hosts:
                  - my.host
                  port:
                    name: http
                    number: 80
                    protocol: HTTP
                  tls:
                    httpsRedirect: true
            Actual:
                - hosts:
                  - my.host
                  port:
                    name: http
                    number: 80
                    protocol: HTTP
                  tls:
                    httpsRedirect: true
                - hosts:
                  - my.host2
                  port:
                    name: https
                    number: 443
                    protocol: HTTPS
                  tls:
                    credentialName: service1-RELEASE-NAME
                    mode: SIMPLE
cristiadu commented 1 year ago

PS: I could use equals on spec.servers[0] with the whole YAML structure, so that can be a workaround if you know for sure the array index to match.

froy001 commented 1 year ago

@cristiadu Following up on your suggestion:

The template:

volumes:
        - name: config-volume
          configMap:
            name: app-02
        - name: all-vault
          projected:
            sources:
              - secret:
                  name: common-vault

The test (expecting 'false' logic to not render a volume):

  - it: should not render the rookout volume when rookout.enabled is not given
    templates:
      - charts/app-01/templates/deployment.yaml
      - charts/app-02/templates/deployment.yaml
    asserts:
      - isNull:
          path: spec.template.spec.volumes[2]

test result:

    - should not render the rookout volume when rookout.enabled is not given

        - asserts[0] `isNull` fail
            Template:   charts-integration-test/charts/app-01/templates/deployment.yaml
            DocumentIndex:  0
            Error:
                [2] :
                - configMap:
                    name: foobar-ns-app-01
                  name: config-volume
                - name: all-vault
                  projected:
                    sources:
                    - secret:
                        name: common-vault
            Template:   charts-integration-test/charts/app-02/templates/deployment.yaml
            DocumentIndex:  0
            Error:
                [2] :
                - configMap:
                    name: foobar-ns-app-02
                  name: config-volume
                - name: all-vault
                  projected:
                    sources:
                    - secret:
                        name: common-vault

 PASS  test integration Ingress tests/ingress_test.yaml
 PASS  test integration service tests/service_test.yaml

Charts:      1 failed, 0 passed, 1 total
Test Suites: 1 failed, 3 passed, 4 total
Tests:       1 failed, 97 passed, 98 total
Snapshot:    0 passed, 0 total
Time:        3.860298409s

@quintush it seems that the processing of yaml arrays has a fundumental flaw in logic. I wish I speaked go so I could inteligently look into it.

I think it has to do with go's OOB error when trying to evaluate an the 3rd item in a 2 item array. I think that in this context, OOB error should be treated as Null since when checking what the 3rd item in a 2 item array is equal to... well it equals to nothing we can comprehend or Null.

Thanks for the effort.