Support multiple `module` parameters (QUnit.config.module array)

mgol commented 4 months ago

Tell us about your runtime:

QUnit version: 2.20.1
Which environment are you using? (e.g., browser, Node): browser
How are you running QUnit? (e.g., QUnit CLI, Grunt, Karma, manually in browser): manually in browser: currently in Chrome 121.0.6167.184 for macOS or Firefox 123.0 for macOS

What are you trying to do?

I'm trying to run tests for multiple modules by providing a URL directly, without initial UI interactions on my side.

Code that reproduces the problem:

In jQuery, we have multiple test modules. It is possible to choose them from the module dropdown which results in multiple moduleId added to the URL. But you cannot compute those moduleIds in your mind easily since they are hashed.

QUnit supports specifying a module via the module query parameter. In jQuery, we often use it to run tests divided by module on TestSwarm in CI.

I'd like to pass multiple modules via specifying the module field directly multiple times. Example: ?module=support&module=basic.

What did you expect to happen?

I expected ?module=support&module=basic to run all the tests from the support & basic modules.

What actually happened?

Passing the module parameter multiple times results in errors. If you e.g. pass ?module=support&module=basic, you get an error:

qunit.js:3041 Uncaught TypeError: config.module.toLowerCase is not a function
    at Test.valid (qunit.js:3041:59)
    at new Test (qunit.js:2428:19)
    at addTest (qunit.js:3116:19)
    at Object.test (qunit.js:3133:5)
    at ready.js:13:8
    at ready.js:166:4

The error happens in the following line: https://github.com/qunitjs/qunit/blob/8ea67f4448bbbcfca7871fb3e65481da7ba2a3ad/src/test.js#L835.

It happens because while in the ?module=support case config.module is "support", in the ?module=support&module=basic case it's an array: ["support", "basic"].

I think this way of passing multiple modules worked in the past. Can we bring it back?

Krinkle commented 4 months ago

@mgol I'll look into if/when this worked. I'd certainly welcome it as a new feature, even if it wasn't there before.

Today, one way to run multiple modules in the HTML Runner, is to select them from the dropdown menu. Internally, this is powered by the moduleId parameter. For example: ?moduleId=c7776548&moduleId=52aa3c97.

To apply a free-form filter by word match, you can use also use filter with a value like support or /support|basic/.

mgol commented 4 months ago

filter is a nice workaround but it also catches test titles from other modules.

I know about the moduleId way but this is not that useful when you want to schedule a specific run from the command line. @timmywil is just hitting it which made him copy the QUnit hashing algorithm so that running tests in multiple modules from the CLI are possible. It'd be much easier if we could just pass multiple module names.

qunitjs / qunit