search

SVF-tools / Test-Suite

PTABen: Micro-benchmark Suite for Pointer Analysis
69 stars 38 forks source link

Fix test invocation #92

Closed MartinNowack closed 1 year ago

MartinNowack commented 1 year ago

Explicitly fail diff_tests if running the cmd failed to detect assertions

Without these changes, tests succeed even though the executed application failed in the first place. Require zero-return for executed applications not 0-pattern matching.

If this is merged, this will fail certain test cases, for example the following assertion is triggered often without failing the test case (https://github.com/SVF-tools/SVF/actions/runs/3538322327/jobs/5939060081#step:11:78):

svf-ex: /home/runner/work/SVF/SVF/include/SVF-LLVM/IRAnnotator.h:108: void SVF::IRAnnotator::readPAGgepNodes(SVF::SVFIR*): Assertion `idEq && "nodeId is not equal to gepnodeId?"' failed.

This should be fixed by somebody more familiar with the code than me.

yuleisui commented 1 year ago

Explicitly fail diff_tests if running the cmd failed to detect assertions

Without these changes, tests succeed even though the executed application failed in the first place. Require zero-return for executed applications not 0-pattern matching.

If this is merged, this will fail certain test cases, for example the following assertion is triggered often without failing the test case (https://github.com/SVF-tools/SVF/actions/runs/3538322327/jobs/5939060081#step:11:78):

svf-ex: /home/runner/work/SVF/SVF/include/SVF-LLVM/IRAnnotator.h:108: void SVF::IRAnnotator::readPAGgepNodes(SVF::SVFIR*): Assertion `idEq && "nodeId is not equal to gepnodeId?"' failed.

This should be fixed by somebody more familiar with the code than me.

we will remove IRAnnotator soon.

MartinNowack commented 1 year ago

Oh great! In this case, this PR is read as well.

MartinNowack commented 1 year ago

Add the small test case for sanitizer performance tests

MartinNowack commented 1 year ago

@yuleisui I updated the PR. For now, it does not fail the svf-ex test case. So it's safe to merge it.

yuleisui commented 1 year ago

@yuleisui I updated the PR. For now, it does not fail the svf-ex test case. So it's safe to merge it.

I am waiting for a pull request for SVF to remove IRAnnotator before I merge yours, will be soon.