Add features `handle-panics` and `backtrace`

proptest-rs / proptest

Hypothesis-like property testing for Rust

Apache License 2.0

1.63k stars 152 forks source link

Add features `handle-panics` and `backtrace` #421

Open target-san opened 5 months ago

target-san commented 5 months ago

handle-panics attempts to capture panic backtrace when one happens, instead of spitting it to console
backtrace enables actual backtraces using std::backtrace
Short backtrace printed second time after test failure is produced by toplevel test func reporting its failure via panic, can be remedied by rewriting helper macros so that they return failure instead of panicking.
Collected backtrace is still 50+ frames long. Unfortunately it seems to be impossible to even enumerate frames on stable, yet alone use existing short backtrace style. Backtrace::frames() and BacktraceFrame are still experimental.

P.S: Sorry for "PR spam", prev one #419 was closed prematurely due to accident

closes #356

target-san commented 4 months ago

My biggest concern is that I had to change TestCaseError and TestError types to keep backtrace. This effectively breaks public API. Unfortunately making this change cfg-conditional isn't very appealing too, as it introduces conditional matches in unit tests.

rexmas commented 4 months ago

cc @matthew-russo

matthew-russo commented 4 months ago

I had started going through this last night. will finish up later tonight

rexmas commented 4 months ago

This effectively breaks public API.

Ah right, thank you for raising this. I don't think we can merge then until we move to a new major version unfortunately, or without making it a separate feature like you suggested, and yes I can see how that's unappealing.

target-san commented 4 months ago

Ok, then I guess the best course is to take out "backtrace" feature and try introduce it as separate PR. Will do it in a few days.

matthew-russo commented 4 months ago

we're looking at a 2.0 release so we can include this and a couple other things that are pending. it'd probably take 1-2 months to get everything together if those timelines work for you. also happy to accept changes with no backwards compatibility issues in the meantime if you want a faster turnaround