When this fails I expect a non-zero exit code from the executable. This worked in OUnit 1.1.2.
We recently upgraded the OUnit package to 2.0.0 in our environment and found that the exit code for the above is always 0. It correctly reports in the textual output that there is a failure but returns 0.
This silent failure is major for us because any failure won't be picked up by our automated build system.
I can get this to work as desired by opening OUnit2 instead of OUnit and replacing the () parameter of the test1 to _ to match against the test_ctxt type. However, I would need to do this for all of our existing test base.
But we maintain old build environments and we'd like to be able to keep backwards compatibility.
I would have understood this change in behaviour if OUnit2 updates had been made to the OUnit module itself but the separation of the two namespaces implies to me that I should be able to open the OUnit module and expect the same behaviour regardless of whether I have OUnit package version 2.0.0 or 1.x.
The fixe is pretty straightforward (check result and exit with error code), but I encourage you to upgrade to OUnit2.
The reason of the split of interface was that there are some huge changes (i.e. test_ctxt) but the code is a lot more clean than before.... Not even considering the possible improvement of speed you'll get.
I need to do a new release but I have no ETA for that.
This bug has been migrated from artifact #1392 on forge.ocamlcore.org. It was assigned to user102. It was closed on 2019-08-27 06:50:09.
user16969 posted on 2014-05-13 13:48:35:
Here is a simple example of a failing case in the documented format:
https://gist.github.com/simonjbeaumont/9dbb0771baf616c3493d#file-test-ml
open OUnit
let test1 () = assert_failure "Test failure"
let suite = "suite" >::: ["test1" >:: test1]
let _ = run_test_tt_main suite
When this fails I expect a non-zero exit code from the executable. This worked in OUnit 1.1.2.
We recently upgraded the OUnit package to 2.0.0 in our environment and found that the exit code for the above is always 0. It correctly reports in the textual output that there is a failure but returns 0.
This silent failure is major for us because any failure won't be picked up by our automated build system.
I can get this to work as desired by opening OUnit2 instead of OUnit and replacing the () parameter of the test1 to _ to match against the test_ctxt type. However, I would need to do this for all of our existing test base.
But we maintain old build environments and we'd like to be able to keep backwards compatibility.
I would have understood this change in behaviour if OUnit2 updates had been made to the OUnit module itself but the separation of the two namespaces implies to me that I should be able to open the OUnit module and expect the same behaviour regardless of whether I have OUnit package version 2.0.0 or 1.x.
user102 replied on 2014-05-13 15:34:18:
The fixe is pretty straightforward (check result and exit with error code), but I encourage you to upgrade to OUnit2.
The reason of the split of interface was that there are some huge changes (i.e. test_ctxt) but the code is a lot more clean than before.... Not even considering the possible improvement of speed you'll get.
I need to do a new release but I have no ETA for that.