search

HypothesisWorks / hypothesis

Hypothesis is a powerful, flexible, and easy to use library for property-based testing.
https://hypothesis.works
Other
7.58k stars 587 forks source link

Default to CI-appropriate settings when running on CI #4152

Closed DRMacIver closed 2 weeks ago

DRMacIver commented 2 weeks ago

After one too many complaints about Hypothesis being unsuitable for CI usage, I finally got around to making it so that the trivial configuration for making it deterministic on CI happens out of the box for you. Now if you set the CI environment variable it will run deterministically (as long as your test code is deterministic).

DRMacIver commented 2 weeks ago

Phew. Finally a green build. It turns out it's surprisingly hard to make changes to brittle code that affects literally every single test!

DRMacIver commented 2 weeks ago

Note for later: I'd like it if when people define their own CI profile they could reliably expect it to run on a CI server. This is not, I think, currently the case, because you need to load a profile after registering it. I think if you reregister the currently loaded profile it should automatically load the new one, but I don't want to do it in this PR because it's another change to brittle code.