search

karma-runner / karma

Spectacular Test Runner for JavaScript
http://karma-runner.github.io
MIT License
11.95k stars 1.71k forks source link

Performance-wise way to run all tests in test file #3664

Open segrey opened 3 years ago

segrey commented 3 years ago

Currently, there is no good way to run all tests in a given test file. The closest thing is to use --grep=<top level suite/test name> option that will filter all available tests by the top level suite/test names. That's how it works for Jasmine and Mocha frameworks. The problem with such an approach is that all tests are evaluated at runtime anyway. This slows things down. For example, running a single fast test in a big project with many test files won't be as fast as it would be in the same project but having a single test file.

It'd be great to extend karma run API to add a new --testFilePath parameter. It could be consumed by IntelliJ/WebStorm. Related issue in IntelliJ/WebStorm: https://youtrack.jetbrains.com/issue/WEB-49930

johnjbarton commented 3 years ago

In small apps, the number of test files does not make a big difference because the whole system is small. In big apps, the app-under-test code dwarfs the test code so again makes a small difference. I suppose that very large tests of a simple app could see an effect, but I wonder if the gain will be as large as you expect.

segrey commented 3 years ago

@johnjbarton Agree, the app-under-test code contributes a significant part here. Can it be reduced if only needed part the app-under-test code is delivered to browser? For example, using require/import?

Other than performance, the current approach (passing --grep=<top level suite/test name>) has the following drawback: suites/tests from other test files will also be run if they match the top level suite/test names in a given test file. Adding an API for filtering tests by test file would help here too.