secure-software-engineering / phasar

A LLVM-based static analysis framework.
Other
942 stars 142 forks source link

Unit Tests compareResults numeric ID based ground truth values are fragile #328

Open blipper opened 3 years ago

blipper commented 3 years ago

Is your feature request related to a problem? Please describe.

A typical test pattern in unittets is TEST_F(IFDSTaintAnalysisTest, TaintTest_04) { initialize({PathToLlFiles + "dummy_source_sink/taint_04_cpp_dbg.ll"}); IFDSSolver_P TaintSolver(*TaintProblem); TaintSolver.solve(); map<int, set> GroundTruth; GroundTruth[19] = set{"18"}; GroundTruth[24] = set{"23"}; compareResults(GroundTruth); }

Specifying individual IDs is very fragile since it depends on how clang compiles and a single change can break everything.

Describe the solution you'd like A more flexible/less fragile pattern

e.g. specify the instruction pattern (with filters/qulifiers) %kIVSize = alloca i32, align 4, !psr.id !35 | ID: 7

Something like IDLookup.Builder.IRpattern("%kIVSize = alloca i32, align 4").inFunction("main").afterIRpattern("%p = alloca %struct.Params").build().getID();

MMory commented 3 years ago

I agree that this is fragile. So far we have not had problems with that, which I don't mean as an excuse not to do it right.

blipper commented 3 years ago

One example is the differences than result between stdlibc++ and libcxx