search

HypothesisWorks / hypothesis

Hypothesis is a powerful, flexible, and easy to use library for property-based testing.
https://hypothesis.works
Other
7.51k stars 583 forks source link

Add type annotations to `hypothesis.internal.conjecture` #3074

Open Zac-HD opened 3 years ago

Zac-HD commented 3 years ago

The low-level engine behind Hypothesis is hypothesis.internal.conjecture. While well-commented, it can also be tricky to follow in places and I think that type annotations would make it easier to understand - especially with the variety of things called "data"!

This would be particularly valuable now because we're thinking about refactoring conjecture to add a "mid-level IR", which would improve our ability to detect redundancy (i.e. #1986 but more so) and support better integration with Crosshair.

Using a tool like MonkeyType to derive initial annotations from our tests would be pretty neat, though we'll still need careful review. autotype might be a better starting point, since it's fully static rather than using potentially-incomplete runtime data.

Zac-HD commented 3 years ago

Adding the performance issue label because this unlocks experiments with e.g. the mypyc compiler, which could plausibly be a big win for bytestring-oriented code like conjecture.

https://glyph.twistedmatrix.com/2022/04/you-should-compile-your-python-and-heres-why.html suggests that this might be a big win.

https://ichard26.github.io/blog/2022/05/31/compiling-black-with-mypyc-part-1/ walks through the process of applying mypyc to black.

Zac-HD commented 2 years ago

In #3336, Adrian spells out the precise steps:

  1. Install monkeytype and the pytest plugin: pip install monkeytype pytest-monkeytype.
  2. Export an env var to tell MonkeyType to only look in the module we want: export MONKEYTYPE_TRACE_MODULES=hypothesis.internal.conjecture
  3. Run tests: pytest --monkeytype-output=./monkeytype.sqlite3 hypothesis-python/tests/conjecture
  4. Apply some type annotations: monkeytype apply hypothesis.internal.conjecture.utils
  5. Review and adjust

Once we have all of hypothesis.internal.conjecture annotated, we should also try for a much stricter typechecking config on that section of the code. And once it all works, try compiling!

adriangb commented 2 years ago

A couple notes from the first round of typing. There's a couple instances of things that looks like they should be refactored in the future: