search

proptest-rs / proptest

Hypothesis-like property testing for Rust
Apache License 2.0
1.67k stars 157 forks source link

Support live visualizations with Tyche, via the `OpenPBTStats` log format? #497

Open Zac-HD opened 3 weeks ago

Zac-HD commented 3 weeks ago

Tyche is a VS Code extension designed to help users understand what their property-based tests are doing - with nice features like live-updating graphs, and lots of explorable raw data if you want to dig deeper.

I've been collaborating with the author since late last year (as a Hypothesis maintainer), as described in our paper. Notably, this helped us find several performance problems and outright bugs in Hypothesis itself, as well as being useful for our users.

But the reason I'm opening this issue is that Tyche is not Hypothesis-specific; it's instead based on a simple json-lines format which could be emitted from any PBT library. If you want to support it from proptest, emitting the type, run_start, property, status, and representation is enough to enable a substantial portion of Tyche’s features.

matthew-russo commented 3 weeks ago

This looks pretty cool -- thanks for letting us know. Will look into it