FEATURE REQUEST: Show count of PASS/FAIL on CLI

[blame2020]

Is it possible to display the number of fail/pass results such as PASS: 2, FAIL: 1 ?

[ivan-velasco]

For individual test suite or sequential list of test suites runs, its possible.

This might get tricky with parallel execution to get results for all tests. I recommend to output to xml, then merge all xml reports into one with node package junit-report-merger

[blame2020]

Thank you your response.

For individual test suite or sequential list of test suites runs, its possible

sorry. I still don't understand what it means.

I am running the test suite in the README.

$ cat cases/01.yml 
---
name: Title of TestSuite
testcases:
- name: TestCase with default value, exec cmd. Check if exit code != 1
  steps:
  - script: echo 'foo'
    type: exec

- name: Title of First TestCase
  steps:
  - script: echo 'foo'
    assertions:
    - result.code ShouldEqual 0
  - script: echo 'bar'
    assertions:
    - result.systemout ShouldNotContainSubstring foo
    - result.timeseconds ShouldBeLessThan 1

- name: GET http testcase, with 5 seconds timeout
  steps:
  - type: http
    method: GET
    url: https://eu.api.ovh.com/1.0/
    timeout: 5
    assertions:
    - result.body ShouldContainSubstring /dedicated/server
    - result.body ShouldContainSubstring /ipLoadbalancing
    - result.statuscode ShouldEqual 200
    - result.timeseconds ShouldBeLessThan 1

- name: Test with retries and delay in seconds between each try
  steps:
  - type: http
    method: GET
    url: https://eu.api.ovh.com/1.0/
    retry: 3
    retry_if: # (optional, lets you early break unrecoverable errors)
    - result.statuscode ShouldNotEqual 403
    delay: 2
    assertions:
    - result.statuscode ShouldEqual 200

$ venom run cases/01.yml 
          [trac] writing venom.3.log
 • Title of TestSuite (cases/01.yml)
        • TestCase-with-default-value-exec-cmd-Check-if-exit-code-1 PASS
        • Title-of-First-TestCase PASS
        • GET-http-testcase-with-5-seconds-timeout FAIL
Testcase "GET http testcase, with 5 seconds timeout", step #0-0: Assertion "result.timeseconds ShouldBeLessThan 1" failed. expected: 1.036383959 less than 1 but it wasn't (01.yml:0)
        • Test-with-retries-and-delay-in-seconds-between-each-try PASS
final status: FAIL 

# I would like summary of tests in here.

This doesn't seem to fall within that description.

Could you show examples?

Sorry for my poor English.

[ivan-velasco]

@blame2020 Hello, I meant it is possible to add into venom as an enhancement. Currently not supported.

[blame2020]

@ivan-velasco

I understood.

Thank you.