microsoft / playwright

Playwright is a framework for Web Testing and Automation. It allows testing Chromium, Firefox and WebKit with a single API.
https://playwright.dev
Apache License 2.0
66.51k stars 3.64k forks source link

[Question] Retry Failed Step(s) #2580

Closed vcamposs closed 4 years ago

vcamposs commented 4 years ago

Hello! How deal with retry failed step(s) with playwright ? There is a lib or workaround ? Tks

arjunattam commented 4 years ago

Thanks @vcamposs. Playwright internally retries in certain circumstances (so as to avoid the user-side burden of managing retries). Can you please elaborate on your scenario – more details the better? We could then come up with a solution.

vcamposs commented 4 years ago

Hello @arjun27!

This is a way to deal with false negativies. For example : I need run some feature (with some steps) but a specific service was out for a seconds. With "retry failed steps", i can re-run this feature and a dont lose my pipeline execution.

Retry sample : -- Execution 1 : Feature checkout --> Fail -- Execution 2 : Feature checkout --> Sucess

-- Result : Feature checkout --> Sucess

Samples with other tools : -- https://github.com/dblock/rspec-rerun -- https://github.com/Bkucera/cypress-plugin-retries

If dont have a native solution, there is a workaround ?

Tks =))

arjunattam commented 4 years ago

Thanks @vcamposs

For now, I would recommend checking out jest-playwright, which enables you to run Playwright with the Jest test framework. With jest-playwright, you can use the jest-circus test runner which supports retries

jest.retryTimes(3);

If you prefer using plain Playwright, you can write JavaScript to put your feature test in a function, and re-run it if the success criteria is not met.

Would love to know how you're thinking about these two options.

pavelfeldman commented 4 years ago

As per discussion above, retry is a test runner feature, not a test driver feature. We are looking into it, but don't have a good place to file this bug yet.