search

soot-oss / soot

Soot - A Java optimization framework
GNU Lesser General Public License v2.1
2.85k stars 706 forks source link

Automated testing + regression tests #61

Open malaverdiere opened 11 years ago

malaverdiere commented 11 years ago

A lot of the test cases in Soot are dependent on a human to read their output. They should be converted to use a testing framework such as JUnit of TestNG that can give a clear pass/fail.

Test cases should be available for all the features in soot. Good luck with that! :)

ericbodden commented 11 years ago

Good idea, but where do we get the test cases from?

pcpratts commented 11 years ago

Hey Guys,

We can build them as we go for each problem submitted to the mailing list.

Here is an example of the types of test cases I plan on making:

  1. Take a directory of .class files
  2. Load them into the Scene and convert to Jimple/Shimple
  3. Bring them back down to .class
  4. Execute the original .class and the generated .class
  5. Capture the System.exit value and System.out/System.err of each
  6. Compare the results. If they are equal, the test case passes.
pcpratts commented 11 years ago

This is what I have so far.

https://github.com/pcpratts/soot-testharness

ericbodden commented 11 years ago

Depends on #94

ericbodden commented 11 years ago

One thing we should do is try to find a good integration of the abc test suite which we currently run every night into Jenkins. One way to do so would be to write a wrapper that makes the suite appear as a JUnit test suite to external components like Jenkins.