search

dreal / dreal2

Please check dreal4 instead.
https://github.com/dreal/dreal4
GNU General Public License v3.0
13 stars 15 forks source link

Failing Regression Tests #42

Closed danbryce closed 10 years ago

danbryce commented 10 years ago

I've been seeing several failed regression tests for the last couple of months and wasn't sure if you were aware of the status:

 39 - regression_nra_ode_bouncing-ball-alpha-random-no-ode.smt2 (Failed)
 40 - regression_nra_ode_bouncing-ball-alpha-random.smt2 (Failed)
 50 - regression_nra_ode_normal-faulty.smt2 (Failed)
 51 - regression_nra_ode_normal-half.smt2 (Failed)
 52 - regression_nra_ode_normal.smt2 (Failed)
 57 - regression_nra_ode_shortsat_bouncing-ball-alpha-random-no-ode.smt2 (Failed)
 58 - regression_nra_ode_shortsat_bouncing-ball-alpha-random.smt2 (Failed)
 66 - regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2 (Timeout)
 68 - regression_nra_ode_shortsat_normal-faulty.smt2 (Failed)
 69 - regression_nra_ode_shortsat_normal-half.smt2 (Failed)
 70 - regression_nra_ode_shortsat_normal.smt2 (Failed)

Are these known to be failing?

soonhokong commented 10 years ago

@danbryce, I'm cleaning up regression tests. You can ignore most of them, but this one

 66 - regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2 (Timeout)

should be passed. Could you run it on your machine and let me know how long it takes?

danbryce commented 10 years ago

In my latest pull request, I’ve cleaned up a few that I created by merging the flags into the default tests.

I’ll check this one again.

On Jun 10, 2014, at 2:03 PM, Soonho Kong notifications@github.com wrote:

@danbryce, I'm cleaning up regression tests. You can ignore most of them, but this one

66 - regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2 (Timeout)

should be passed. Could you run it on your machine and let me know how long it takes?

— Reply to this email directly or view it on GitHub.

danbryce commented 10 years ago

I ran all my tests again and got the following:

 39 - regression_nra_ode_bouncing-ball-alpha-random-no-ode.smt2 (Failed)
 40 - regression_nra_ode_bouncing-ball-alpha-random.smt2 (Failed)
 44 - regression_nra_ode_cardiac_good_3_0.smt2 (Failed)
 46 - regression_nra_ode_cardiac_to1_good_3_0.smt2 (Failed)
 48 - regression_nra_ode_cardiac_to2_good_3_0.smt2 (Failed)
 50 - regression_nra_ode_normal-faulty.smt2 (Failed)
 51 - regression_nra_ode_normal-half.smt2 (Failed)
 52 - regression_nra_ode_normal.smt2 (Failed)
 57 - regression_nra_ode_shortsat_bouncing-ball-alpha-random-no-ode.smt2 (Failed)
 58 - regression_nra_ode_shortsat_bouncing-ball-alpha-random.smt2 (Failed)
 62 - regression_nra_ode_shortsat_cardiac_good_3_0.smt2 (Failed)
 64 - regression_nra_ode_shortsat_cardiac_to1_good_3_0.smt2 (Failed)
 66 - regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2 (Failed)
 68 - regression_nra_ode_shortsat_normal-faulty.smt2 (Failed)
 69 - regression_nra_ode_shortsat_normal-half.smt2 (Failed)
 70 - regression_nra_ode_shortsat_normal.smt2 (Failed)

running the one you mentioned manually results in a failure.

On Jun 10, 2014, at 2:03 PM, Soonho Kong notifications@github.com wrote:

@danbryce, I'm cleaning up regression tests. You can ignore most of them, but this one

66 - regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2 (Timeout)

should be passed. Could you run it on your machine and let me know how long it takes?

— Reply to this email directly or view it on GitHub.

soonhokong commented 10 years ago

@danbryce, let me check the current blessed/master version.

soonhokong commented 10 years ago

I ran ctest with blessed/master on a linux machine. Everything passed but

regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2
danbryce commented 10 years ago

I must have done something to break it in my local copy. I get the same result as you do on the blessed/master copy.

On Jun 10, 2014, at 3:44 PM, Soonho Kong notifications@github.com wrote:

I ran ctest with blessed/master on a linux machine. Everything passed but

regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2

— Reply to this email directly or view it on GitHub.

soonhokong commented 10 years ago

I see. We can wait the merge until you find the problem.

danbryce commented 10 years ago

I found the problem. However, I’m getting now:

 63 - regression_nra_ode_shortsat_cardiac_to2_good_3_0.smt2 (Timeout)

I’m not sure if you want me to update the pull request or fix this inefficiency. LMK.

On Jun 11, 2014, at 2:55 PM, Soonho Kong notifications@github.com wrote:

I see. We can wait the merge until you find the problem.

— Reply to this email directly or view it on GitHub.

soonhokong commented 10 years ago

I think the 63 was already there and it's just timed out.

I’m not sure if you want me to update the pull request or fix this inefficiency.

Yes, please fix the problem and create another pull-request.

danbryce commented 10 years ago

I wasn’t using the options file in the test script.

On Jun 11, 2014, at 3:32 PM, Soonho Kong notifications@github.com wrote:

I think the 63 was already there and it's just timed out.

I’m not sure if you want me to update the pull request or fix this inefficiency.

Yes, please fix the problem and create another pull-request.

— Reply to this email directly or view it on GitHub.

soonhokong commented 10 years ago

Got it. I think it's OK to close this issue now.