Report failed Tarantool assertions and crashes

tarantool / test-run

Tarantool functional testing framework

14 stars 15 forks source link

Report failed Tarantool assertions and crashes #430

Open drewdzzz opened 4 months ago

drewdzzz commented 4 months ago

In this workflow (taken from master) the test config-luatest/anonymous_replica_test.lua is marked as flaky. Actually, this flakiness is much more serious than just test fail: here is a fail of internal Tarantool assertion. Such failures are much more serious than flaky tests, so probably we should report assertion fails (and crashes) in a separate section, maybe we should even fail the test.

This is also the case for ASAN fails.

sergos commented 4 months ago

Should be tightly bound with multivac system to enable a separate statistics for tests failed by assertion.