ovh / venom

🐍 Manage and run your integration tests with efficiency - Venom run executors (script, HTTP Request, web, imap, etc... ) and assertions
Apache License 2.0
1.04k stars 142 forks source link

[bug] `skip` doesn't work with variables from previous steps #777

Open berlic opened 7 months ago

berlic commented 7 months ago

Hey. Thanks for the useful testing tool.

There is and issue that only global variables or variables from other test cases work with 'skip', but not from previous steps within a test case.

Reproducible with this example:

vars:
  static_var: 1
testcases:
- name: skip-works1
  steps:
  - name: do-step
    type: exec
    script: echo .
    vars:
      dynamic_var:
        from: result.code
        default: ""
  - name: dont-skip-step
    type: exec
    script: echo .
    skip:
      - static_var ShouldEqual 1
  - name: skip-step
    type: exec
    script: echo .
    skip:
      - static_var ShouldEqual 0
- name: skip-works2
  steps:
  - name: do-step
    type: exec
    script: echo .
  - name: dont-skip-step
    type: exec
    script: echo .
    skip:
      - skip-works1.dynamic_var ShouldEqual 0
  - name: skip-step
    type: exec
    script: echo .
    skip:
      - skip-works1.dynamic_var ShouldEqual 1
- name: skip-doesnt-work
  steps:
  - name: do-step
    type: exec
    script: echo .
    vars:
      another_dynamic_var:
        from: result.code
        default: ""
  - name: dont-skip-step
    type: exec
    script: echo another_dynamic_var={{.skip-doesnt-work.another_dynamic_var}}
    skip:
      - skip-doesnt-work.another_dynamic_var ShouldEqual 0
  - name: skip-step
    type: exec
    script: echo another_dynamic_var={{.skip-doesnt-work.another_dynamic_var}}
    skip:
      - skip-doesnt-work.another_dynamic_var ShouldEqual 1

I found some abandoned PRs on this matter: https://github.com/ovh/venom/pull/685, but the latest release is sill plagued with this issue.

berlic commented 1 month ago

@yesnault Hi. Please reopen the issue. Not sure why the bot marks issues as completed while they are clearly not.