Snaipe / Criterion

A cross-platform C and C++ unit testing framework for the 21st century
MIT License
1.95k stars 177 forks source link

Using any clause with double with at least one valid condition results in assertion failure #522

Open rgomes1 opened 3 months ago

rgomes1 commented 3 months ago

An "any" clause with int comparisons where at least one condition is true works: cr_expect(any(eq(int, 1, 2), eq(int, 3, 3)));

Tried the same with double, and assertion fails for some reason: double elapsed_time;

elapsed_time = 0.10; cr_expect(ieee_ulp_eq(dbl, elapsed_time, 0.10, 4)); / this passes / cr_expect(ieee_ulp_eq(dbl, elapsed_time, 0.11, 4)); / line 20 - this fails (as expected) / / here, first condition should pass (like above), so why does this fail (line 22)? / cr_expect(any(ieee_ulp_eq(dbl, elapsed_time, 0.10, 4), ieee_ulp_eq(dbl, elapsed_time, 0.11, 4)));

Failure reported: [----] ... test_any.c:20: Assertion Failed [----] ieee_ulp_eq(dbl, elapsed_time, 0.11, 4): [----] diff: [-0.10000000000000001-]{+0.11+} [----] ulp: 4 [----] ... test_any.c:22: Assertion Failed [FAIL] test_any::any_with_double: (0.00s)

Also note that the line 22 failure does not report any value diffs, which perhaps is fine.

quarthex commented 1 month ago

It seems that it depends on the order of the assertions.

cr_expect(any(0, 0, 1), "This one passes");
cr_expect(any(0, 1, 0), "This one fails");
cr_expect(any(1, 0, 0), "This one fails");
rgomes1 commented 1 month ago

Does your observation mean this should be a candidate to be fixed such that order should not matter (i.e. a true boolean or evaluation regardless of order of assertions)?

quarthex commented 1 month ago

According to the documentation:

any() evaluates a sequence of criteria, and combines them into a single value with the logical or operator (||).

Today, the result of any(x, y, …, z) seems to be z.

rgomes1 commented 1 month ago

Ok, so it is a valid finding and hopefully will be a candidate for fixing in the next release. Thanks for looking into this. For context, I am a more recent user of both Criterion and Mimick, so expecting to gain more experience with them over time.