ERL-1143: Eunit doesn't report a timeout in parallel tests

[OTP-Maintainer]

Original reporter: JIRAUSER13400 Affected version: OTP-21.3 Component: eunit Migrated from: https://bugs.erlang.org/browse/ERL-1143

---

When a parallel test is cancelled by a timeout before it's reported, it's totally ignored in the result.
{code:erlang}
-module(ignored_cancel).
-include_lib( "eunit/include/eunit.hrl" ).

foo(N) ->
    timer:sleep(N),
    ok.

step1_test_() ->
    {inparallel, [
     {"foo1",                ?_assertEqual(ok, foo(800))},
     {"foo2",                ?_assertEqual(ok, foo(700))},
     {"foo3", {timeout, 0.2, ?_assertEqual(ok, foo(400))}}
    ]}.
 {code}
 Sample run:
{code}
Erlang/OTP 21 [erts-10.3] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [hipe]

Eshell V10.3  (abort with ^G)
1> c(ignored_cancel).
{ok,ignored_cancel}
2> eunit:test(ignored_cancel,[verbose]).
======================== EUnit ========================
module 'ignored_cancel'
  ignored_cancel:11: step1_test_ (foo1)...[0.801 s] ok
  ignored_cancel:12: step1_test_ (foo2)...[0.701 s] ok
  [done in 0.804 s]
=======================================================
  2 tests passed.
ok
{code}

Interestingly, if foo3 is moved to the top
{code}
step1_test_() ->
    {inparallel, [
     {"foo3", {timeout, 0.2, ?_assertEqual(ok, foo(400))}},
     {"foo1",                ?_assertEqual(ok, foo(800))},
     {"foo2",                ?_assertEqual(ok, foo(700))}
    ]}.
{code}

it works as expected:
{code}
======================== EUnit ========================
module 'ignored_cancel'
  ignored_cancel:11: step1_test_ (foo3)...*timed out*
  ignored_cancel:12: step1_test_ (foo1)...[0.802 s] ok
  ignored_cancel:13: step1_test_ (foo2)...[0.701 s] ok
  [done in 0.805 s]
=======================================================
  Failed: 0.  Skipped: 0.  Passed: 2.
One or more tests were cancelled.
error
{code}

Similarly, increasing the timeout and time spent in the test over the time spent in other tests makes the whole report correct:
{code}
     {"foo1",                ?_assertEqual(ok, foo(800))},
     {"foo2",                ?_assertEqual(ok, foo(700))},
     {"foo3", {timeout, 0.9, ?_assertEqual(ok, foo(1000))}}
{code}
{code}
======================== EUnit ========================
module 'ignored_cancel'
  ignored_cancel:11: step1_test_ (foo1)...[0.801 s] ok
  ignored_cancel:12: step1_test_ (foo2)...[0.701 s] ok
  ignored_cancel:13: step1_test_ (foo3)...*timed out*
  [done in 0.901 s]
=======================================================
  Failed: 0.  Skipped: 0.  Passed: 2.
One or more tests were cancelled.
error
{code}

[OTP-Maintainer]

JIRAUSER13400 said:

Any tests (failing or passing) added under the test with the timeout are ignored, too.

[OTP-Maintainer]

ingela said:

This is at the moment not a priority for Ericsson, but we will accept PR if some one wants to fix this. 

[choroba]

Note: I am JIRAUSER13400.