proper-testing / proper

PropEr: a QuickCheck-inspired property-based testing tool for Erlang
http://proper-testing.github.io
GNU General Public License v3.0
879 stars 168 forks source link

run_parallel_commands doesn't handle exceptions in commands the same way run_commands does #224

Open x4lldux opened 4 years ago

x4lldux commented 4 years ago

When a command crashes during run_parallel_commands, the entire test just crashes also, whilerun_commands catches such a situation and returns a exception tuple as a result. Shouldn't they work the same way?

kostis commented 4 years ago

Please submit a minimal test case that illustrates the issue.

x4lldux commented 4 years ago

I can, but it'll be in Elixir using PropCheck library (a thin wrapper around PropEr); my Erlang is not that strong. But I've traced the origin of it:

So when a SUT has a crashes, run_commands will return {exception, _, _, _} tuple as a result, while run_parallel_commands just forwards the crash.

kostis commented 4 years ago

I can, but it'll be in Elixir using PropCheck library (a thin wrapper around PropEr); my Erlang is not that strong.

Well, you've came to the PropEr (TM) repository. Here you have your chance to strengthen your Erlang skills ;-)

x4lldux commented 4 years ago

Well, you've came to the PropEr (TM) repository.

Which runs on a BEAM™ that supports many other languages. Here you have your chance to make us feel included ;)

x4lldux commented 4 years ago

@kostis test case added, see commit x4lldux/proper@c932eef62cfddc8a2726d09a2cb9bf5e81fd6015

x4lldux commented 4 years ago

Also, the docs for run_parallel_commands are missing the exception tuple as a valid result type and that's what is being returned when command raises during the sequential part.