search

fedora-modularity / meta-test-family

Meta test family (MTF) is a tool to test components of a modular Fedora:
https://docs.pagure.org/modularity/
Other
8 stars 12 forks source link

Fixes #207 Inform user, why test is cancelled. #215

Closed phracek closed 6 years ago

phracek commented 6 years ago

Signed-off-by: Petr "Stone" Hracek phracek@redhat.com

This Pull Request informs user, why test was cancelled. Because skipped is deprecated, cancel is used instead of skip.

New output if Dockerfile is missing, looks like:

JOB LOG    : /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/job.log
 (01/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/check_compose.py:ComposeTest.test_component_profile_installability: CANCEL (0.00 s)
 (02/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/modulelint.py:ModuleLintPackagesCheck.test: PASS (14.34 s)
 (03/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/dockerlint.py:DockerfileLinterInContainer.test_all_nodocs: WARN (123.80 s)
 (04/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/dockerlint.py:DockerfileLinterInContainer.test_installed_docs: PASS (14.24 s)
 (05/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/dockerlint.py:DockerfileLinterInContainer.test_docker_clean_all: PASS (14.14 s)
 (06/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/dockerlint.py:DockerLint.testLabels: PASS (3.06 s)
 (07/27) /usr/lib/python2.7/site-packages/moduleframework/tests/generic/rpmvalidation.py:rpmvalidation.testPaths: PASS (27.88 s)
 (08/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_image_name: CANCEL (0.00 s)
 (09/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_maintainer_name: CANCEL (0.00 s)
 (10/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_name: CANCEL (0.00 s)
 (11/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_description: CANCEL (0.00 s)
 (12/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_usage: CANCEL (0.00 s)
 (13/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_environment_variables: CANCEL (0.00 s)
 (14/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_security_implications: CANCEL (0.00 s)
 (15/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_from_is_first_directive: CANCEL (0.00 s)
 (16/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_from_directive_is_valid: CANCEL (0.00 s)
 (17/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_chained_run_dnf_commands: CANCEL (0.00 s)
 (18/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_chained_run_rest_commands: CANCEL (0.00 s)
 (19/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_helpmd_is_present: CANCEL (0.00 s)
 (20/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_architecture_label_exists: CANCEL (0.00 s)
 (21/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_name_in_env_and_label_exists: CANCEL (0.00 s)
 (22/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_maintainer_label_exists: CANCEL (0.00 s)
 (23/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_release_label_exists: CANCEL (0.00 s)
 (24/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_version_label_exists: CANCEL (0.01 s)
 (25/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_com_redhat_component_label_exists: CANCEL (0.00 s)
 (26/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_summary_label_exists: CANCEL (0.00 s)
 (27/27) /usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_run_or_usage_label_exists: CANCEL (0.00 s)
RESULTS    : PASS 5 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 1 | INTERRUPT 0 | CANCEL 21
JOB TIME   : 203.34 s
JOB HTML   : /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/results.html

TEST:   01-/usr/lib/python2.7/site-packages/moduleframework/tests/generic/check_compose.py:ComposeTest.test_component_profile_installability
CANCEL:  Nspawn specific test
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/01-_usr_lib_python2.7_site-packages_moduleframework_tests_generic_check_compose.py_ComposeTest.test_component_profile_installability/debug.log
-------------------------
TEST:   08-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_image_name
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/08-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_image_name/debug.log
-------------------------
TEST:   09-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_maintainer_name
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/09-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_maintainer_name/debug.log
-------------------------
TEST:   10-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_name
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/10-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_name/debug.log
-------------------------
TEST:   11-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_description
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/11-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_description/debug.log
-------------------------
TEST:   12-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_usage
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/12-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_usage/debug.log
-------------------------
TEST:   13-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_environment_variables
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/13-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_environment_variables/debug.log
-------------------------
TEST:   14-/usr/lib/python2.7/site-packages/moduleframework/tests/static/helpmd_lint.py:HelpFileSanity.test_helpmd_security_implications
CANCEL:  Help file or Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/14-_usr_lib_python2.7_site-packages_moduleframework_tests_static_helpmd_lint.py_HelpFileSanity.test_helpmd_security_implications/debug.log
-------------------------
TEST:   15-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_from_is_first_directive
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/15-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerInstructionsTests.test_from_is_first_directive/debug.log
-------------------------
TEST:   16-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_from_directive_is_valid
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/16-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerInstructionsTests.test_from_directive_is_valid/debug.log
-------------------------
TEST:   17-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_chained_run_dnf_commands
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/17-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerInstructionsTests.test_chained_run_dnf_commands/debug.log
-------------------------
TEST:   18-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_chained_run_rest_commands
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/18-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerInstructionsTests.test_chained_run_rest_commands/debug.log
-------------------------
TEST:   19-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerInstructionsTests.test_helpmd_is_present
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/19-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerInstructionsTests.test_helpmd_is_present/debug.log
-------------------------
TEST:   20-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_architecture_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/20-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_architecture_label_exists/debug.log
-------------------------
TEST:   21-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_name_in_env_and_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/21-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_name_in_env_and_label_exists/debug.log
-------------------------
TEST:   22-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_maintainer_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/22-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_maintainer_label_exists/debug.log
-------------------------
TEST:   23-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_release_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/23-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_release_label_exists/debug.log
-------------------------
TEST:   24-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_version_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/24-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_version_label_exists/debug.log
-------------------------
TEST:   25-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_com_redhat_component_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/25-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_com_redhat_component_label_exists/debug.log
-------------------------
TEST:   26-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_summary_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/26-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_summary_label_exists/debug.log
-------------------------
TEST:   27-/usr/lib/python2.7/site-packages/moduleframework/tests/static/dockerfile_lint.py:DockerLabelsTests.test_run_or_usage_label_exists
CANCEL:  Dockerfile was not found
        /home/phracek/avocado/job-results/job-2018-02-21T13.35-c398f7e/test-results/27-_usr_lib_python2.7_site-packages_moduleframework_tests_static_dockerfile_lint.py_DockerLabelsTests.test_run_or_usage_label_exists/debug.log
TomasTomecek commented 6 years ago

LGTM, looks way better and exactly what I need; can't comment on implementation

phracek commented 6 years ago

skip is deprecated, like in your issue. [WARNING: self.skip() will be deprecated. Use 'self.cancel()' or the skip decorators]

Therefore I use cancel. If I use skip, then the message is shown too. Which is weird.

TomasTomecek commented 6 years ago

Hmmm, this is now merged while @jscotka raised some concerns. Did you guys talked about it privately? Honzo, wanna meet and talk about this PR?