search

kyverno / chainsaw

Declarative K8s e2e testing
https://kyverno.github.io/chainsaw/
Apache License 2.0
308 stars 45 forks source link

[Bug] Template bindings cannot be changed #2133

Open cbandy opened 2 weeks ago

cbandy commented 2 weeks ago

chainsaw version Version

v0.2.11

Description

I tried using a step template with bindings but could not change their values using with.bindings.

Steps to reproduce

Create a simple test with one template:

# chainsaw-test.yaml
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Test
metadata:
  name: template-bindings
spec:
  steps:
  - use:
      template: template.yaml
      with:
        bindings:
        - name: input
          value: from-test # 👈
# template.yaml
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: StepTemplate
metadata:
  name: template
spec:
  bindings:
  - name: input
    value: from-template # 👈

  try:
  - create:
      resource:
        apiVersion: v1
        kind: ConfigMap
        metadata:
          name: ($input)

See that chainsaw test uses the value from-template rather than from-test:

$ chainsaw test
Version: v0.2.11
Loading default configuration...
- Using test file: chainsaw-test
- TestDirs [.]
- SkipDelete false
- FailFast false
- Namespace ''
- FullName false
- IncludeTestRegex ''
- ExcludeTestRegex ''
- ApplyTimeout 5s
- AssertTimeout 30s
- CleanupTimeout 30s
- DeleteTimeout 15s
- ErrorTimeout 30s
- ExecTimeout 5s
- DeletionPropagationPolicy Background
- Template true
- NoCluster false
- PauseOnFailure false
Loading tests...
- template-bindings (.)
Loading values...
Running tests...
=== RUN   chainsaw
=== PAUSE chainsaw
=== CONT  chainsaw
=== RUN   chainsaw/template-bindings
=== PAUSE chainsaw/template-bindings
=== CONT  chainsaw/template-bindings
    | 14:18:05 | template-bindings | @chainsaw | CREATE    | OK    | v1/Namespace @ chainsaw-settled-mosquito
    | 14:18:05 | template-bindings | step-1    | TRY       | BEGIN |
    | 14:18:05 | template-bindings | step-1    | CREATE    | RUN   | v1/ConfigMap @ chainsaw-settled-mosquito/from-template 👈
    | 14:18:05 | template-bindings | step-1    | CREATE    | OK    | v1/ConfigMap @ chainsaw-settled-mosquito/from-template 👈
    | 14:18:05 | template-bindings | step-1    | CREATE    | DONE  | v1/ConfigMap @ chainsaw-settled-mosquito/from-template 👈
    | 14:18:05 | template-bindings | step-1    | TRY       | END   |
    | 14:18:05 | template-bindings | step-1    | CLEANUP   | BEGIN |
    | 14:18:05 | template-bindings | step-1    | DELETE    | OK    | v1/ConfigMap @ chainsaw-settled-mosquito/from-template 👈
    | 14:18:05 | template-bindings | step-1    | CLEANUP   | END   |
    | 14:18:05 | template-bindings | @chainsaw | CLEANUP   | BEGIN |
    | 14:18:05 | template-bindings | @chainsaw | DELETE    | OK    | v1/Namespace @ chainsaw-settled-mosquito
    | 14:18:10 | template-bindings | @chainsaw | CLEANUP   | END   |
--- PASS: chainsaw (0.00s)
    --- PASS: chainsaw/template-bindings (5.28s)
PASS
Tests Summary...
- Passed  tests 1
- Failed  tests 0
- Skipped tests 0
Done.

Expected behavior

I expected the template bindings to be the "default" of sorts. I expected that when the test imports the template, the test gets to decide the values of bindings.

Screenshots

No response

Logs

No response

Slack discussion

No response

Troubleshooting

cbandy commented 2 weeks ago

Perhaps these are the lines that affect the template binding precedence:

https://github.com/kyverno/chainsaw/blob/54d8d4449172472d59dd34feda0fd2d2a32a1b44/pkg/discovery/load.go#L70-L71