search

avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
343 stars 341 forks source link

avocado --show=test run doesn't return test case result #5717

Closed chunfuwen closed 1 year ago

chunfuwen commented 1 year ago

Hello Cleber, Sometimes ,I use --show=test to dump running output into console for debugging e.g: avocado --show=test run --vt-type libvirt --vt-machine-type q35 --vt-connect-uri qemu:///system domain_life_cycle.shutdown_domain.negative.paused_state_guest

Previously, e.g running on 92lts, at least I got information indicating whether test case pass or not:

92lts: Job ID: 2cd388e92e4bd525cab07d5a4433bb59a891605f

type_specific.io-github-autotest-libvirt.domain_life_cycle.shutdown_domain.negative.paused_state_guest: STARTED type_specific.io-github-autotest-libvirt.domain_life_cycle.shutdown_domain.negative.paused_state_guest: PASS More information in /root/avocado/job-results/job-2023-06-28T04.40-2cd388e/test-results/1-type_specific.io-github-autotest-libvirt.domain_life_cycle.shutdown_domain.negative.paused_state_guest Test results available in /root/avocado/job-results/job-2023-06-28T04.40-2cd388e Not logging /sys/kernel/debug/sched_features (file not found)

But in 102.0, I got nothing about final running status.

102.0: avocado.test: 1-type_specific.io-github-autotest-libvirt.domain_life_cycle.shutdown_domain.negative.paused_state_guest: vnc_autoport = yes avocado.test: 1-type_specific.io-github-autotest-libvirt.domain_life_cycle.shutdown_domain.negative.paused_state_guest: Searching for test modules that match 'type = create_destroy_domain' and 'provider = io-github-autotest-libvirt' on this cartesian dict avocado.test: 1-type_specific.io-github-autotest-libvirt.domain_life_cycle.shutdown_domain.negative.paused_state_guest: cleaning libvirtd logs...

richtja commented 1 year ago

Hi @chunfuwen, in version 102 we separated the logging streams, now the stream test is only related to the logs generated by test itself. IMO, the best way how you can see the test results is to add the app stream into the show option. 

avocado --show=test,app run examples/tests/passtest.py
JOB ID     : 17f3e8aba4f4f7da75918280807520f3cd1d5b5c
JOB LOG    : /home/janrichter/avocado/job-results/job-2023-07-11T09.51-17f3e8a/job.log
 (1/1) examples/tests/passtest.py:PassTest.test: STARTED
avocado.test: 1-examples/tests/passtest.py:PassTest.test: INIT 1-examples/tests/passtest.py:PassTest.test
avocado.test: 1-examples/tests/passtest.py:PassTest.test: PARAMS (key=timeout, path=*, default=None) => None
avocado.test: 1-examples/tests/passtest.py:PassTest.test: Test metadata:
avocado.test: 1-examples/tests/passtest.py:PassTest.test:   filename: /home/janrichter/Avocado/avocado/examples/tests/passtest.py
avocado.test: 1-examples/tests/passtest.py:PassTest.test:   teststmpdir: /var/tmp/avocado_strguxat
avocado.test: 1-examples/tests/passtest.py:PassTest.test: START 1-examples/tests/passtest.py:PassTest.test
avocado.test: 1-examples/tests/passtest.py:PassTest.test: PASS 1-examples/tests/passtest.py:PassTest.test
avocado.test: 1-examples/tests/passtest.py:PassTest.test:
 (1/1) examples/tests/passtest.py:PassTest.test: PASS (0.01 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /home/janrichter/avocado/job-results/job-2023-07-11T09.51-17f3e8a/results.html
JOB TIME   : 1.68 s
chunfuwen commented 1 year ago

@richtja thank for your information

richtja commented 1 year ago

@chunfuwen no problem, I am closing this issue as resolved, but feel free to reopen it if you will have any additional questions.