search

cucumber / cucumber-js

Cucumber for JavaScript
https://cucumber.io
MIT License
5.01k stars 1.09k forks source link

Facing Issue with Parallelism in Jenkins while running Playwright Cucumber tests #2411

Open onFilm opened 3 weeks ago

onFilm commented 3 weeks ago

👓 What did you see?

I've encountered an issue while attempting to run Playwright Cucumber tests in parallel on Jenkins. Locally, specifying parallel: 4 in cucumber.mjs works as expected on Windows. However, when running the tests on Jenkins, it seems to be executing in a single worker instead of utilizing parallelism.

✅ What did you expect to see?

Tests should run in parallel, leveraging the specified number of workers.

📦 Which tool/library version are you using?

No response

🔬 How could we reproduce it?

No response

📚 Any additional context?

No response

davidjgoss commented 3 weeks ago

Are you certain your cucumber.mjs configuration is being picked up when you run in Jenkins?

You could try turning on debug mode which will output the resolved configuration among other things.

Also just to double check, you're not expecting it to spin up 4 executors in Jenkins?

onFilm commented 3 weeks ago

With DEBUG=cucumber I see below

+ docker run --rm --entrypoint sh -e BUILD_NUMBER -e BUILD_URL -e JOB_NAME -e ENV=stable -e DEBUG=cucumber -v /var/lib/jenkins/workspace/testParallelExecution/reports:/tests/reports -w /tests testparallelexecution -c 'npx cucumber-js --tags "@Smoke" --parallel 4'
Configuration will be loaded from "cucumber.mjs"
No profiles specified; using default profile
Resolved configuration: {
  backtrace: false,
  dryRun: false,
  forceExit: false,
  failFast: false,
  format: [
    'json:reports/cucumber-report.json',
    'html:reports/report.html',
    'rerun:reports/@failedrerun.txt',
    '@cucumber/pretty-formatter'
  ],
  formatOptions: { snippetInterface: 'async-await', printAttachments: false },
  import: [],
  language: 'en',
  name: [],
  order: 'defined',
  paths: [],
  parallel: 4,
  publish: false,
  publishQuiet: true,
  require: [ 'step-definitions/*.ts', 'setup/*.ts', 'utils/*.ts' ],
  requireModule: [ 'ts-node/register' ],
  retry: 1,
  retryTagFilter: '@flaky',
  strict: true,
  tags: '(@Smoke)',
  worldParameters: { foo: 'bar' }
}

During execution, I encounter the error below in jenkins, but I don't observe it in my local run, also despite specifying parallel 4, resulting in at least 2 instances of Chrome launching in local.


VError: Unexpected error on worker.receiveMessage: a BeforeAll hook errored on worker 0, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at exit (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:8:38)
    at /tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:22:9
caused by: VError: a BeforeAll hook errored on worker 0, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at Worker.runTestRunHooks (/tests/node_modules/@cucumber/cucumber/src/runtime/run_test_run_hooks.ts:32:19)
    at Worker.initialize (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:93:5)
    at Worker.receiveMessage (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:110:7)
caused by: Error: function timed out, ensure the promise resolves within 60000 milliseconds
    at Timeout.<anonymous> (/tests/node_modules/@cucumber/cucumber/src/time.ts:52:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
VError: Unexpected error on worker.receiveMessage: a BeforeAll hook errored on worker 1, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at exit (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:8:38)
    at /tests/node_modules/@cucumber/cucumber/src/runtime/parallel/run_worker.ts:22:9
caused by: VError: a BeforeAll hook errored on worker 1, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds
    at Worker.runTestRunHooks (/tests/node_modules/@cucumber/cucumber/src/runtime/run_test_run_hooks.ts:32:19)
    at Worker.initialize (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:93:5)
    at Worker.receiveMessage (/tests/node_modules/@cucumber/cucumber/src/runtime/parallel/worker.ts:110:7)
caused by: Error: function timed out, ensure the promise resolves within 60000 milliseconds
    at Timeout.<anonymous> (/tests/node_modules/@cucumber/cucumber/src/time.ts:52:14)
    at listOnTimeout (node:internal/timers:559:17)
    at processTimers (node:internal/timers:502:7)
davidjgoss commented 3 weeks ago

This is the issue you need to deal with first:

a BeforeAll hook errored on worker 0, process exiting: setup/common-hooks.ts:19: function timed out, ensure the promise resolves within 60000 milliseconds

Regardless of parallel settings, this will prevent any scenarios from running. Only you will have the context to understand why your BeforeAll hook might be timing out in CI, but it could be connectivity to an external resource, permissions etc.

onFilm commented 1 week ago

I have BeforeAll as below, and still getting this error, I am not able to figure out what is causing. Can you help me figure out issue and solution, Thanks BeforeAll image Error in console: image

mrsenzy commented 1 week ago

@davidjgoss - Similar / Same issue occurring for me, but different stacktrace based on my custom playwright project with cucumber 10.8.0. we were using 10.3.1 and it was working fine and with 10.8.0 upgrade, it works fine for single thread execution but when parallel count is set to 1 or greater -- we get below error

Error: Unexpected error on worker.receiveMessage at exit (C:\Code\node_modules\@cucumber\cucumber\src\runtime\parallel\run_worker.ts:7:21) at C:\Code\node_modules\@cucumber\cucumber\src\runtime\parallel\run_worker.ts:21:9 at processTicksAndRejections (node:internal/process/task_queues:95:5) { [cause]: Error: Cannot find module './stepDefinitions/*/.ts' Require stack: