Return values of init_per_suite and init_per_group

[ansd]

Hello,

https://www.erlang.org/doc/apps/common_test/ct_suite.html#c:init_per_testcase/2 documents that {fail, Reason} is a valid return value.

https://www.erlang.org/doc/apps/common_test/ct_suite.html#c:init_per_suite/1 and https://www.erlang.org/doc/apps/common_test/ct_suite.html#c:init_per_group/2 do not document this return value.

However, I tested that both init_per_suite and init_per_group seem to allow {fail, Reason} and behave as desired in the sense that test cases are skipped and the process returns a non-zero exit code.

Is there a specific reason that {fail, Reason} is undocumented (and therefore {fail, Reason} shouldn't be returned from init_per_suite and init_per_group), or did you just forget to document this return value?

[essen]

I think it works because CT hooks allow the fail tuple in pre/post_init_per_suite/group. CT tries to call hooks before/after init_per_suite, and post_init_per_suite receives the result of init_per_suite. It can then modify it or fail. But the default is to just use the return value of init_per_suite as the result of post_init_per_suite, so if init_per_suite returned a fail tuple then it'll be handled as if it came from a CT hook.

It probably wasn't intentional, but since it already works and is generally useful I would say it is worth documenting and adding typespecs.

[u3s]

thanks for information, we will plan looking into this.