search

proper-testing / proper

PropEr: a QuickCheck-inspired property-based testing tool for Erlang
http://proper-testing.github.io
GNU General Public License v3.0
881 stars 168 forks source link

Execution time varies? #93

Closed okeuday closed 10 years ago

okeuday commented 10 years ago

I am using the repo from tag v1.1 and have noticed that some runs of the test https://github.com/CloudI/CloudI/blob/develop/src/lib/cloudi_service_quorum/test_ct/cloudi_service_quorum_SUITE.erl exceed the timeout (30 minutes with numtests == 1) when ran virtualized in an automated build (at https://travis-ci.org/CloudI/CloudI/). I was wondering if any of the proper source code would be creating potentially large variation in execution time (similar to dialyzer's -Wrace_conditions possibly in this different context due to the PropEr analysis)?

okeuday commented 10 years ago

I have determined this is due to my timeout usage. Sorry for the noise.

kostis commented 10 years ago

Due to some strange cosmic coincidence, I was looking at this issue today (sorry for the delay in reacting) but could not really come up for any reason for this behavior and was about to ask for more details later today. Good that you resolved the issue without our help.

In general, it may help next time, to send us some more focused test that exhibits the issue (and some more info on what is happening on a regular machine vs. a virtualized environment).

Thanks for your interest in and your use of PropEr.