Add a way to mark a test as flaky: - Githubissues

Shopify / deprecation_toolkit

⚒Eliminate deprecations from your codebase ⚒

MIT License

460 stars 40 forks source link

Add a way to mark a test as flaky: #38

Closed Edouard-chin closed 5 years ago

Edouard-chin commented 5 years ago

Add a way to mark a test as flaky:

The deprecations triggered by some test is different (either they get triggered in a different order or they don't trigger the same amount of deprecations between multiple test runs).

By definition the test is flaky. Even if at the end the test passes, the fact that sometime a method run and sometime it doesn't is the sign of flakyness. The deprecation toolkit can't deal with that.

This PR adds a way to mark a test as flaky so that the deprecations triggered by the test will be ignored. The syntax is like that:
```
my_test:
- flaky: true
```