Builder style tests don't work well with Vitest Github Actions reporter

[sazzer]

Describe the bug Firstly, I'm not sure if this is a Pactum or Vitest issue but I think it's probably more here.

When writing Pactum tests in the Builder style, if you're using Vitest as the test runner and have the github-actions reporter configured then the test failures aren't correctly reported.

They are correctly reported if you use the BDD style tests

To Reproduce See https://github.com/sazzer/vitest-gha In particular, see https://github.com/sazzer/vitest-gha/actions/runs/10849090321 as a test run with this problem. There are two test failures - one for each of Builder and BDD style - but only one annotation added to the action.

Expected behavior Annotations added for test failures in both the Builder and BDD style tests.

Screenshots

Software (please complete the following information):

• OS: Running on Github Actions ubuntu-latest runner
• NodeJS Version: latest
• Test Runner: vitest

[ASaiAnudeep]

Hello @sazzer , I don't think adding annotations is related with PactumJS. May be Github Actions pick the last failure reported.

[sazzer]

Hello @sazzer , I don't think adding annotations is related with PactumJS. May be Github Actions pick the last failure reported.

Adding annotations is nothing to do with PactumJS. That's handled by the Vitest github-actions reporter.

However, the reason I raised this issue here is that it has something to do with the way that the Builder style tests cause test failures which means the Vitest reporter doesn't see them properly.

You can see from the attached Actions run that there are two test cases - one using Builder style and the other using BDD style - being run in the same action with the same Vitest instance, but only the BDD style one is generating annotations.

[ASaiAnudeep]

The underlying assertion code for both styles have the same logic. Have you tried running just the builder style test?

[sazzer]

This is the result of only having a single Builder style test: https://github.com/sazzer/vitest-gha/actions/runs/10887364402

Conversely, this is the result of only having a single BDD style test: https://github.com/sazzer/vitest-gha/actions/runs/10887376975

If you compare the code for the two runs, you can see that the tests are both doing the exact same thing only in the different styles, and yet the reported output is very different.

Cheers

[ASaiAnudeep]

I'm not sure on how the system/library behaves around annotations. Need more time and help in understanding it.