sclorg / container-common-scripts

Apache License 2.0
21 stars 45 forks source link

testlib improvements #338

Closed zmiklank closed 1 year ago

zmiklank commented 1 year ago
  1. kill image build after ten minutes

    The image build is run in subshell, which is killed if does not finish after 10 minutes. If the build is finished sooner no action is taken.

    This could however cause problems when sending SIGINT to the test process, as trap is not inherited by subshells.

  2. use different trap functions for EXIT and SIGINT for container tests

    so when the tests are stopped with SIGINT the final tasks are minimized and the waiting time for results is reduced. That means, that image size and stats are not shown in this case.

  3. OCP tests: examine the exit code in the trap function

    in OCP3 tests we still use set -e for discovering test failure. However, if the non-zero exit code is caught by trap function the whole bash ends with exit code from the trap function itself and not with the exit code with the tests.

    This commit solves the issue by checking what value was returned from the testsuite and exits the whole testsuite based on that.

zmiklank commented 1 year ago

[test-all]

zmiklank commented 1 year ago

Added some comments.

Thank you for the review! I will fix the issues you have found.

Additionally, one of the commits says "ct_cleanup now does not return TESTUITE_RESULT, it only sets it." but from what I can see, ct_cleanup does not set TESTUITE_RESULT in any way, so the message could be confusing to the reader.

You are right. it is not ct_cleanup function that sets it, but ct_run_tests_from_testset. Thank you for catching this.

zmiklank commented 1 year ago

[test-all]

zmiklank commented 1 year ago

@pkubatrh The changes you requested were provided. Could you please re-review?

zmiklank commented 1 year ago

I have added one more commit - adding explicit function for updating testcase result.

This will be needed for fixing https://github.com/sclorg/s2i-python-container/issues/593.

zmiklank commented 1 year ago

[test]

zmiklank commented 1 year ago

[test]

phracek commented 1 year ago

I totally missing tests in this repo that could test corner cases like it is introduced by this pull request.