denoland / deno

A modern runtime for JavaScript and TypeScript.
https://deno.com
MIT License
97.61k stars 5.37k forks source link

deno test --parallel is sequential per file #17511

Open aricart opened 1 year ago

aricart commented 1 year ago

Firstly deno test is awesome, and parallel makes it very awesome. It would be great if the test runner could internally parallelize tests that are in the same file

deno test -A  --parallel tests/authenticator_test.ts
Check file:///Users/aricart/Dropbox/code/src/github.com/nats-io/nats.deno/tests/authenticator_test.ts
./tests/authenticator_test.ts => authenticator - username password fns ... ok (2s)
./tests/authenticator_test.ts => authenticator - username string password fn ... ok (2s)
./tests/authenticator_test.ts => authenticator - username fn password string ... ok (2s)
./tests/authenticator_test.ts => authenticator - token fn ... ok (2s)
./tests/authenticator_test.ts => authenticator - nkey fn ... ok (2s)
./tests/authenticator_test.ts => authenticator - jwt bearer fn ... ok (2s)
./tests/authenticator_test.ts => authenticator - jwt fn ... ok (2s)
./tests/authenticator_test.ts => authenticator - creds fn ... ok (2s)

ok | 8 passed | 0 failed (18s)

This is a small sample, I have other files that take over 2m to run, but as shown in the example above, what should potentially take 3s total, takes over 18, because tests are sequential when coming from the same file.

nayeemrmn commented 1 year ago

Duplicate of #11409, it's recommended to use the test steps API for this

aricart commented 1 year ago

The issue with the steps is that sanitizers must be turned off (or so I thought from the docs when I read them).

aricart commented 1 year ago

The other reason why the step api is sub-optimal is that if your test step is longer than say 10 lines of code, it creates a fairly complex-looking file.

While the idea of grouping tests is great, for things that are more complex it doesn't scale well. For tests where you are not setting up a shared fixture (as are mine), the option to just run the functions in parallel would mean I don't have to change any code, and I can opt-in to do that. Starting up a server and tearing it down for each test, keeps the tests independent and isolated, and that is a good thing.