utf.py should returns an exception if some of the tests fail

[karthikgvss]

When we are trying to trigger automatic build using jenkins, we dont get any exception or error from utf.py -a command. Due to this out status is always build completed successfully. We would like to see a error / exception saying that some tests failed. So, we could update the status as build failed. Please suggest.

[aaaaalbert]

sys.exit(error_number) with error_number>0?

[vladimir-v-diaz]

Note: The exit status of tests with the unittest module: https://docs.python.org/3.1/library/unittest.html#unittest.main "By default main calls sys.exit() with an exit code indicating success or failure of the tests run."

If you're running a unittest suite, the behavior is slightly different. http://stackoverflow.com/questions/24972098/unit-test-script-returns-exit-code-0-even-if-tests-fail https://docs.python.org/3.1/library/unittest.html#unittest.TestResult.wasSuccessful

[aaaaalbert]

FWIW, https://github.com/karthikgvss/utf/tree/add-seattlelib_v2 implements the proposed sys.exit approach. @kellender based his testing of Travis and AppVeyor on this, so I suppose we could build a clean patchset from Karthik's implementation and go with it.