cucumber / cucumber-js

Cucumber for JavaScript
https://cucumber.io
MIT License
5.04k stars 1.09k forks source link

Cucumber 10.0.0 changes execution order of the feature files when `paths` configuration is provided #2343

Closed pk closed 11 months ago

pk commented 11 months ago

👓 What did you see?

We use configuration profiles with paths attribute to enforce execution order in our E2E test suites:

export const ciexcludebugs = {
  ...JSON.parse(JSON.stringify(common)),
  tags: '(not @bug) and (not @ignore)',
  paths: [
    'fixtures/',
    'features/'
  ]
}

In 9.x.x all fixtures (subfolders, and respective *.feature files) have been executed first and the features after that. Since 10.x.x the order is reversed and features are always the first.

We have fixtures and features (both Cucumber feature files). fixtures contains cucumber feature files which execute and each saves a state of our server into a json file - we generate re-usable state & we test our management features.

✅ What did you expect to see?

The Cucumber should execute the feature files in the specified order from the Profile paths (or command line) and then, within the folders alphabetically as in 9.x.x.

📦 Which tool/library version are you using?

Cucumber 10.0.0 Node.js 20.8.1

🔬 How could we reproduce it?

Use profile paths as above or cucumber-js fixtures/ features/

pk commented 11 months ago

I've been going through the open issues and I'm not sure if this one is relevant or not to my use case: #2253

davidjgoss commented 11 months ago

Thanks for raising @pk. I think this has come from the upgrade to the latest glob which we did before the 10.0.0 release and included some changes to ordering which we had to adapt. I'll take a look and see what we can do to fix.

pk commented 11 months ago

@davidjgoss I had to revert to @cucumber/cucumber@9.6.0 to have the (my) expected behaviour. Which I think is the latest 9x release, right?

davidjgoss commented 11 months ago

@pk that's right yes.

davidjgoss commented 11 months ago

Released in https://github.com/cucumber/cucumber-js/releases/tag/v10.0.1

pk commented 11 months ago

Hi @davidjgoss , thank you for the quick fix and release. We confirm the fix works, at least for our scenario.