uw-pluverse / perses

language-agnostic program reducer.
GNU General Public License v3.0
168 stars 24 forks source link

Perses may hang due to test scripts with specific loops #20

Closed AIRTEspresso closed 2 years ago

AIRTEspresso commented 2 years ago

Hi! I am now using Perses on my buggy test scripts and I really appreciate your effort because Perses did help me cut down my test scripts and eventually find the bug. However, for several complex scripts I fond Perses may hang, here is an example:

class Test {
static long instanceCount = 218L;
static void vMeth1(byte by, long l, int i4) {
    int i5;
    float f;
    for(i5=14;i5<284;++i5) {
        switch((i5%5)*5+99)
        {
        case 104: {
        f=1;
        while(++f<6) instanceCount+=f;
        }
        }
    }
}
void vMeth(int i2, int i3) {
    byte by2=54;
    vMeth1(by2,instanceCount,i3);
}
void mainTest(String[] strArr1){
    int i,i17=11,i19=-16903,i20=-55683;
    byte by3=23;
    for (i=24;i<390;i++){
        vMeth(i,i);
        i20+=(i*i17+i17)-instanceCount;
    }
    System.out.println("i19 i20 by3 = "+i19+","+i20+","+by3);
}
public static void main(String[] strArr) {
    Test _instance=new Test();
    for(int i=0;i<10;i++)
        _instance.mainTest(strArr);
}
}

Perses may hang with this piece of code, maybe Perses may try to reduce some tokens which make the loop here endless when executing the test script, I guess. I also find it is useful to set a timeout limit in org.perses.reduction.FutureExecutionResultInfo, like this:

class FutureExecutionResultInfo(
  val edit: AbstractSparTreeEdit<*>,
  val program: AbstractCacheRetrievalResult.CacheMiss,
  // TODO: make future private.
  val future: TestScriptExecutorService.FutureTestScriptExecutionTask
) {
  val result: PropertyTestResult
    get() {
      try {
        return future.get(TIME_LIMIT, TimeUnit.SECONDS)
      }catch (e : TimeoutException) {
        return PropertyTestResult.NON_INTERESTING_RESULT
      }
    }

  fun cancel() {
    future.cancel(true)
  }
}

At least with this modification on my local code, Perses no longer hangs. I hope my advice can be helpful.

chengniansun commented 2 years ago

Thank you for your interest in perses and the feedback on the code change.

The common practice is to use the command timeout inside the shell script. One example is here.

You can change your command into this form: timeout -s 9 60 <your command and arguments here>

I think inside Perses, if a property test runs too long, e.g., 10 minutes or 5 minutes, there will be a warning message emitted from Perses. But I forgot the exact threshold.

AIRTEspresso commented 2 years ago

Oh, that's right! Thanks for your answering!

chengniansun commented 2 years ago

Glad to help.

chengniansun commented 2 years ago

This is addressed internally, and will be released in the next version.