Open dgp1130 opened 3 years ago
This sounds like a great feature to have added to Jasmine. I would be happy to review a pull request to add configuration in the node.js library as well as here to allow the current setting to be configured from the external environment.
Current Behavior
The only ways to set test timeout currently are to explicitly set
jasmine.DEFAULT_TIMEOUT_INTERVAL = // ...
in JavaScript, or to add a timeout to the specific test callback. When trying to change the timeout for a large number of tests, it requires either modifying every test individually, or having some piece of shared code where the default value can be set. If thejasmine
binary is sharded and executed multiple times, there may not be a single place to set this value.Possible Solution
It should be possible to configure
jasmine.DEFAULT_TIMEOUT_INTERVAL
via an environment variable or command line flag. Runningjasmine --default-timeout 10000 # ...
orJASMINE_DEFAULT_TIMEOUT="10000" jasmine # ...
should use the provided value as the default timeout.If any tests override this value directly in code, those should probably take precedence. This flag merely sets the default timeout value for all tests.
Context
This is a particular challenge with Bazel, where tests are often sharded and executed across multiple
jasmine
executions for higher parallelism and to reduce dependencies on each execution for better cacheability. In this system, a single command likebazel test //...
may run manyjasmine
binaries to run many tests. With this system, it is very difficult to modify the timeout for all the executed tests. Answering a question like "Are my tests running too slowly, or are they stuck altogether?" is much harder without being able to easily increase the test timeout.Another example I'm encountering right now when debugging flaky tests, I would like to re-run them many times over to reproduce rare bugs. Bazel supports this with
--runs_per_test 100
, which will run all tests 100 times, helping to reproduce rare bugs. There is also a--jobs
flag to run many of these targets in parallel. However in practice, running so many tests in parallel means there is more resource competition and the tests run slower as a result. This causes a number of timeout errors that I can't easily ignore. If I could just set the default timeout via environment variable or command line, I would be able to configure Bazel to pass that through and, then these erroneous failures could be easily ignored for local development via some kind of--define JASMINE_TIMEOUT="99999"
. Unfortunately, Jasmine provides no such mechanism, and there is no single location where I could addjasmine.DEFAULT_TIMEOUT_INTERVAL = 99999
, which would apply to all tests.Your Environment
Steps to reproduce:
Some tests fail with, and I want to be able to raise the timeout to allow them to run a little slower: