search

pact-foundation / pact-js

JS version of Pact. Pact is a contract testing framework for HTTP APIs and non-HTTP asynchronous messaging systems.
https://pact.io
Other
1.61k stars 344 forks source link

Execute Pact tests with Vite (vitest) fails consistently #965

Closed crazycabo closed 1 year ago

crazycabo commented 1 year ago

Software versions

Issue Checklist

Please confirm the following:

Expected behaviour

Teardown process should not cause errors.

Actual behaviour

After Pact files are created, an unknown error occurs during teardown process. It's difficult to debug since no output points to a possible cause. I have not found any documentation of Pact using Vite, either.

Tests do execute and Pact files are created but, for some reason a non-zero exit code causes issues.

Steps to reproduce

Execute at least one Pact test with Vite via vitest command.

import path from 'path';
import { Pact, Matchers } from '@pact-foundation/pact';

const fs = require('fs');
const unirest = require('unirest');

const { like } = Matchers;

const provider = new Pact({
  consumer: 'dealer-portal',
  provider: 'mng-service',
  spec: 2,
  log: path.resolve(process.cwd(), 'logs', 'pact.log'),
  logLevel: 'DEBUG',
  dir: path.resolve(process.cwd(), 'pacts'),
  cors: true
});

function getJsonData(filePath) {
  try {
    const jsonString = fs.readFileSync(path.resolve(__dirname, filePath));
    return JSON.parse(jsonString.toString());
  } catch (error) {
    // eslint-disable-next-line no-console
    console.log(`JSON file at ${filePath} could not be loaded. \n error`);
  }
  return null;
}

describe('FloorPlanService API', () => {
  beforeAll(() => provider.setup());
  afterEach(() => provider.verify());
  afterAll(() => provider.finalize());

  describe('Get Floorplan search results', () => {
    const responseBody = getJsonData('./json/floorplan_search.json');
    console.log('ResponseBody:', responseBody);
    const floorplanSearchUrl = '/api/v2/floorplans/search';

    test('Floorplan Keyword Search', async () => {
      await provider.addInteraction({
        state: 'Floorplan keyword search',
        uponReceiving: 'request for search results',
        withRequest: {
          method: 'GET',
          path: floorplanSearchUrl,
          query: { statusId: '123', keyword: 'searchText', page: '0', size: '25' },
          headers: {
            Accept: 'application/json',
            'Content-Type': 'application/json'
          }
        },
        willRespondWith: {
          status: 200,
          headers: {
            'Content-Type': 'application/json'
          },
          body: like(responseBody)
        }
      });

      await unirest
        .get(`${provider.mockService.baseUrl}${floorplanSearchUrl}`)
        .query({ statusId: 123, keyword: 'searchText', page: 0, size: 25 })
        .headers({ Accept: 'application/json', 'Content-Type': 'application/json' })
        .then((response) => {
          expect(response.body).toStrictEqual(responseBody);
        });
    });
  });
});

Relevant log files

[2022-10-26 17:29:37.906 +0000] DEBUG (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: Starting pact binary '/Users/brian.smith/Documents/github_projects/customer-portals/node_modules/.pnpm/@pact-foundation+pact-node@10.17.6/node_modules/@pact-foundation/pact-node/standalone/darwin-1.89.02-rc1/pact/bin/pact-mock-service', with arguments [service --consumer dealer-portal --cors true --pact_dir /Users/brian.smith/Documents/github_projects/customer-portals/apps/dealer-portal/pacts --host 127.0.0.1 --log /Users/brian.smith/Documents/github_projects/customer-portals/apps/dealer-portal/logs/pact.log --log-level DEBUG --provider mng-service --pact_specification_version 2 --ssl false --pact-file-write-mode overwrite]
[2022-10-26 17:29:37.921 +0000] DEBUG (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: Created '/Users/brian.smith/Documents/github_projects/customer-portals/node_modules/.pnpm/@pact-foundation+pact-node@10.17.6/node_modules/@pact-foundation/pact-node/standalone/darwin-1.89.02-rc1/pact/bin/pact-mock-service' process with PID: 34549
[2022-10-26 17:29:38.672 +0000] DEBUG (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: INFO  WEBrick 1.3.1
INFO  ruby 2.4.10 (2020-03-31) [x86_64-darwin19]

[2022-10-26 17:29:38.673 +0000] DEBUG (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: INFO  WEBrick::HTTPServer#start: pid=34549 port=54542

[2022-10-26 17:29:38.673 +0000] INFO (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: Pact running on port 54542
[2022-10-26 17:29:38.955 +0000] INFO (34533 on LNGCmacW15RJGH8): pact@9.18.1: Setting up Pact with Consumer "dealer-portal" and Provider "mng-service"
    using mock service on Port: "54542"
[2022-10-26 17:29:39.027 +0000] INFO (34533 on LNGCmacW15RJGH8): pact@9.18.1: Pact File Written
[2022-10-26 17:29:39.027 +0000] INFO (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: Removing Pact process with PID: 34549
[2022-10-26 17:29:39.029 +0000] DEBUG (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: INFO  going to shutdown ...

[2022-10-26 17:29:39.030 +0000] INFO (34533 on LNGCmacW15RJGH8): pact-node@10.17.6: Deleting Pact Server with options: 
{"timeout":30000,"consumer":"dealer-portal","cors":true,"dir":"/Users/brian.smith/Documents/github_projects/customer-portals/apps/dealer-portal/pacts","host":"127.0.0.1","log":"/Users/brian.smith/Documents/github_projects/customer-portals/apps/dealer-portal/logs/pact.log","logLevel":"DEBUG","pactfileWriteMode":"overwrite","provider":"mng-service","spec":2,"ssl":false,"port":54542,"pactFileWriteMode":"overwrite"}
 ❯ src/pact/floorplan_search.service.pact.test.js  (1 test) 1127ms

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

FAIL  src/pact/floorplan_search.service.pact.test.js > FloorPlanService API
TypeError: hook teardown value must be function, received "object"

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/1]⎯

Test Files  1 failed (1)
     Tests  1 passed (1)
  Start at  17:29:34
  Duration  4.12s (transform 477ms, setup 243ms, collect 863ms, tests 1.13s)
crazycabo commented 1 year ago

Vite has some changes from Jest in its setup and teardown methods. I simply had to follow them, and the process works.

mefellows commented 1 year ago

Ah, thanks!

If you're are able to share the lessons for anyone else that might be interested?

TimothyJones commented 1 year ago

I bet it's because vitest expects an Awaitable<void>. Could you try changing the setup and teardown functions to:

  beforeAll(async () => { await provider.setup() } );  // note the braces

If this is indeed the problem, I kind of think it's a bug in vitest because they say they're Jest compatible.

TimothyJones commented 1 year ago

I raised a PR with vitest that will fix this: https://github.com/vitest-dev/vitest/pull/3147

lafriakh commented 1 year ago

Vite has some changes from Jest in its setup and teardown methods. I simply had to follow them, and the process works.

Can you please share your findings? I tried to change setup and teardown functions to use asynchronous functions but didn't work for me

mefellows commented 1 year ago

Does this not work: https://github.com/pact-foundation/pact-js/issues/965#issuecomment-1348185596 ?

lafriakh commented 1 year ago

Does this not work: #965 (comment) ?

No, not working

mefellows commented 1 year ago

Perhaps you can share the error you're having or the code you have written?

lafriakh commented 1 year ago

Perhaps you can share the error you're having or the code you have written?

To give you more context, I was using Pact with Jest and I'm switching to Vitest. But seems the test's feeling and I believe it's an issue with how Vitest handles teardown hooks.

In Jest I was using it like so:

describe('...', () => {
  const provider = new Pact({
    ...commonPactConfig,
    provider: 'provider-name',
  });

  beforeAll(() => provider.setup());
  afterAll(() => provider.finalize());
  afterEach(() => provider.verify());

  it('should ...', async () => {
    const email = 'john.doe@hse.de';
    const interaction = new Interaction()
      .given('...')
      .uponReceiving('...')
      .withRequest({
        method: 'POST',
        path: '/path',
        body: {
          ...
        },
        headers: {
          'Content-Type': 'application/json',
        },
      })
      .willRespondWith({
        status: 200,
      });

    await provider.addInteraction(interaction);
    const response = await serviceApi(email);
    expect(response.status).toBe(200);
  });
});

Console logs:

Pact verification failed!
Test failed for the following reasons:

  Mock server failed with the following mismatches:

        0) The following request was expected but not received: 
            Method: POST

Error: Pact verification failed - expected interactions did not match actual.
TimothyJones commented 1 year ago

See the linked issue on Vite above - you will need to change your methods to return a void promise instead of an unknown promise.

I fixed this in a PR for them but it has gone into limbo.

TimothyJones commented 1 year ago

Oh wait, looking at your test, I think the problem is in the part you omitted. Can you share your whole test please

lafriakh commented 1 year ago

@TimothyJones I updated the previous comment.

lafriakh commented 1 year ago

I solved my problem by moving from Pact to PactV3 and using --threads=false option when running the tests.

Command:

vitest run --threads=false -c vite.pact.config.ts --mode=pact

Pact example:

describe('...', () => {
  const provider = new PactV3({
    ...commonPactConfig,
    provider: 'provider-service',
  });

  describe('...', () => {
    it('..., () => {
      provider
        .given('...')
        .uponReceiving('...')
        .withRequest({
          method: 'POST',
          path: `/path`,
          body: ...,
        })
        .willRespondWith({
          status: 200,
          body: ...,
        });

      return provider.executeTest(async () => {
        const response = await API.createSomethingAwesome();

        expect(response).toBeDefined();
      });
    });
  });
});
YOU54F commented 10 months ago

I've just tested this out with some simple consumer examples using PactV2 and PactV3 interfaces, with vitest and got them passing successfully and generating pact files

https://github.com/YOU54F/vscode-pact-snippets/pull/2

TimothyJones commented 10 months ago

I think this was to do with https://github.com/vitest-dev/vitest/issues/3146

YOU54F commented 10 months ago

Thanks @TimothyJones, I was reading through that thread and noted the PR had been closed, as there is some confusion between striving to be a drop-in replacement and actually being a drop-in replacement.

I'm not sure if anything has changed in vitest to support this working now. I didn't try this at the time, so thought I would whip something quickly together.

That came about as in a separate project, there was a suggestion to switch from jest -> vitest, so I wanted to check if it would work, which reminded me of this topic.