Vitest - Pact Verifier fails with 3 or more interactions

shishkin commented 3 weeks ago

Software versions

OS: Mac OSX 14.5
Consumer Pact library: Pact JS v12.5.2
Provider Pact library: Pact JS v12.5.2
Node Version: Node v20.11.1

Issue Checklist

Please confirm the following:

[x] I have upgraded to the latest
[x] I have the read the FAQs in the Readme
[x] I have triple checked, that there are no unhandled promises in my code and have read the section on intermittent test failures
[x] I have set my log level to debug and attached a log file showing the complete request/response cycle
[x] For bonus points and virtual high fives, I have created a reproduceable git repository (see below) to illustrate the problem

Expected behaviour

It should either work or explain clearly what user did wrong.

Actual behaviour

I'm running provider tests and as long as I get 3 or more interactions in the pact file, subsequent interactions start failing due to state change handlers. Weirdly, it fails when I duplicate/copy paste interactions in the pact file, so the same interaction that just succeeded will now fail.

Steps to reproduce

Create an interaction and see it succeed.
Duplicate it twice and see the last one fail.
Move interactions in the file around and see a different one of the same interactions fail.

See test and pact JSON.

Relevant log files

See log.

YOU54F commented 3 weeks ago

can you update your packages so pact-js pulls in pact-js-core 14.3.7

i believe this is fixed and your logs show they are running 14.3.6

https://github.com/pact-foundation/pact-js/issues/1216#issuecomment-2137209732

shishkin commented 3 weeks ago

I still observe the same errors with pact-js-core 14.3.7.

YOU54F commented 3 weeks ago

The error is

libc++abi: terminating due to uncaught exception of type Napi::Error

I can resolve with this workaround, tested locally

https://github.com/vitest-dev/vitest/issues/2972#issuecomment-1948012464

mefellows commented 3 weeks ago

Glad you got to the bottom of this Yousaf!

We may need to look into how we can make the core threadsafe, if possible, to enable multi-threaded tests like this (see https://github.com/nodejs/node-addon-api/blob/main/doc/threadsafe.md). For now, that workaround is good to know.

YOU54F commented 3 weeks ago

no worries man, the reproducer was really helpful, so thanks to @shishkin

It might be worth getting a basic vitest example in place, and utilise this test to reproduce our issue, document the current workaround, and it will help us test out any solutions from the linked doc.

💯 enabling multi-threaded tests would be a nice bonus rather than having to forgoe them

shishkin commented 3 weeks ago

Thanks @YOU54F. Indeed, running vitest with --pool=forks resolves the issue. I think I can live with it for now, since provider verifier is only a single test.

mefellows commented 3 weeks ago

Thanks @shishkin. If you do feel inclined to go deep to see if we can make it multi-threaded, we'd be happy to give you some pointers (TL;DR - it'll be in here somewhere: https://github.com/pact-foundation/pact-js-core/blob/master/native/provider.cc).

shishkin commented 2 weeks ago

@mefellows sorry, I'm not competent in native development to dig in.

pact-foundation / pact-js