Runs: Failed AfterSuite() causes any scenario failed

[mirao]

Describe the bug

To Reproduce Suppose AfterSuite() in our CodeceptJS's test suite fails (simulated by I.assertEqual(1, 2);)

Expected behavior Other scenarios passed and only AfterSuite() fails

Actual behavior https://app.testomat.io/projects/spork/runs/01d25f66/report/test/21954381 All scenarios failed:

It looks like Testomat computes a result of scenario not only from scenario's result, but also from BeforeSuite() and AfterSuite(). But IMHO it makes no sense, as BeforeSuite() and AfterSuite() are executed only when test suite starts and ends.

This is how it looks in console output of test run on my side:

-- FAILURES:

  1) CoursesManage
       "after all" hook: AfterSuite for "Delete course":

      expected 1 to equal 2
      + expected - actual

      -1
      +2

      at chaiWrapper.assertEqual (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs-chai@2.3.5/node_modules/codeceptjs-chai/index.js:27:53)
      at Step.run (/home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.14_typescript@4.8.2/node_modules/codeceptjs/lib/step.js:123:49)
      at /home/mirao/workspace/codeceptjs/node_modules/.pnpm/codeceptjs@3.5.14_typescript@4.8.2/node_modules/codeceptjs/lib/actor.js:138:23

  Scenario Steps:
  - I.assertEqual(1, 2) at Context.<anonymous> (./CoursesManage_test.ts:221:7)

  FAIL  | 15 passed, 1 failed   // 57s
Run with --verbose flag to see complete NodeJS stacktrace
[TESTOMATIO] 📊 Report Saved. Report URL: https://app.testomat.io/projects/spork/runs/01d25f66/report

Desktop (please complete the following information):

• OS: Ubuntu 22.04
• CodeceptJS 3.5.14, @testomatio/reporter 1.2.0

[bt-jagababu]

Hi @DavertMik @olexandr13 @poliarush @TetianaKhomenko We too facing this and its messing up with stats. Just commenting for better reach.

[DavertMik]

This is CodeceptJS issue, not related to our release cyecle. I'm moving it to next sprint.