IBM / watson-assistant-workbench

Framework for maintaining Watson Assistant data in github repository with bundle of tools for generating, testing and uploading. Travis: https://travis-ci.com/IBM/watson-assistant-workbench. Artifactory (will be deleted after 25 Nov 2019): https://na.artifactory.swg-devops.com/artifactory/webapp/#/artifacts/browse/tree/General/iot-waw-trevis-generic-local.
Apache License 2.0
16 stars 25 forks source link

Add pytest mark to be able to "try" test #357

Closed ondrejtomas closed 5 years ago

ondrejtomas commented 5 years ago

We want to be able to mark test that to run, show errors but pass (be skipped). There is not such a support in pytest (closest it 'xfail' but it does not show the errors).

Syntax:

@pytest.mark.skipiffails()
def test_withoutLabel(self):
    ...

@pytest.mark.skipiffails(label='202 Cloud Function')
def test_withLabel(self):
    ...

Output:

ci/pytest_customization/skipiffails_test.py::TestSkipiffails::test_withoutLabel SKIPPED                                                                                                              [ 50%]
ci/pytest_customization/skipiffails_test.py::TestSkipiffails::test_withLabel SKIPPED                                                                                                                 [100%]

============================================================================================= warnings summary =============================================================================================
pytest_customization/skipiffails_test.py::TestSkipiffails::test_withoutLabel

  ================ TEMPORARY SKIPPED TEST ================
  ci/pytest_customization/skipiffails_test.py:26: in test_withoutLabel
      assert 0
  E   assert 0

pytest_customization/skipiffails_test.py::TestSkipiffails::test_withLabel

  ================ TEMPORARY SKIPPED TEST - 202 Cloud Function ================
  ci/pytest_customization/skipiffails_test.py:32: in test_withLabel
      assert 0
  E   assert 0

-- Docs: https://docs.pytest.org/en/latest/warnings.html
================================================================================== 2 skipped, 2 warnings in 0.10 seconds ===================================================================================