storybookjs / test-runner

🚕 Turn stories into executable tests
https://storybook.js.org/docs/writing-tests/interaction-testing
MIT License
229 stars 72 forks source link

[Feature request]: Add option to retry failed tests #502

Open ajhenry opened 2 months ago

ajhenry commented 2 months ago

Is your feature request related to a problem? Please describe.

When using chromium, if the browser does not start in time, it can cause the test runner to timeout. In these rare cases we'd ideally like to retry the test (and potentially up the timeout limit).

Describe the solution you'd like

Jest provides this functionality through a config option call retryTimes with the following API:

jest.retryTimes(numRetries, options?)

This option should exist in the storybook base config/CLI options.

The output of all the failed attempts should be condensed in the failure report.

Describe alternatives you've considered

There's a couple unideal options:

  1. You would have to run the whole suite again on fail.
  2. You would grep/fetch which tests failed and run those individually again

Both of these present significant challenges in terms of time and effort.

Are you able to assist to bring the feature to reality?

no

Additional context

No response