codeceptjs / CodeceptJS

Supercharged End 2 End Testing Framework for NodeJS
http://codecept.io
MIT License
4.11k stars 723 forks source link

Using cucumber, scripts exception ends in a passing test #2773

Open npirotte opened 3 years ago

npirotte commented 3 years ago

What are you trying to achieve?

I am using cucumber to write the tests.

A step for some reasons throws a javascript exception.

What do you get instead?

The scenario execution stops without any log, and test is considered as "passing".

This happens whenever I use the UI executor or the headless mode.

Details

require('ts-node/register');
const { setHeadlessWhen, setSharedCookies } = require('@codeceptjs/configure');

// turn on headless mode when running with HEADLESS=true environment variable
// export HEADLESS=true && npx codeceptjs run
setHeadlessWhen(process.env.HEADLESS);
setSharedCookies();

process.env.TEST_URL = process.env.TEST_URL || 'http://localhost:3000';

// eslint-disable-next-line no-undef
exports.config = {
  tests: './test/**/*.test.ts',
  output: './output',
  helpers: {
    Puppeteer: {
      url: process.env.TEST_URL,
      show: true,
      browser: 'chromium',
      chrome: {
        args: ['--no-sandbox'],
      },
    },
    REST: {
      endpoint: `${process.env.TEST_URL}/api`,
    },
  },
  include: {
    I: './steps_file.ts',
  },
  bootstrap: null,
  mocha: {
    reporterOptions: {
      mochaFile: 'reports/e2e/community-management-web.xml',
    },
  },
  name: 'xxx-e2e',
  plugins: {
    pauseOnFail: {},
    retryFailedStep: {
      enabled: true,
    },
    tryTo: {
      enabled: true,
    },
    screenshotOnFail: {
      enabled: true,
    },
    allure: {
      enabled: false,
    },
  },
  gherkin: {
    features: './test/features/*.feature',
    steps: [
      './test/step-definitions/base.steps.ts',
      './test/step-definitions/xxx.steps.ts',
      './test/step-definitions/auth.steps.ts'
    ],
  },
};
VaasRamsay commented 3 years ago

This PR broke it. The fix is in - most probably in the next release. Until then you can go down to 3.0.2 if that's alright with your usecase.