Change the stack_pointer expression to relative position in `is_used`

[Tai7sy]

Change the stack_pointer expression to relative position in is_used, so when detecting whether the stack accessing can be discarded in https://github.com/vtil-project/VTIL-Core/blob/6ffa73ac4ac68d4a861060caaa2428145e413a26/VTIL-Compiler/common/auxiliaries.cpp#L156-L167 can get right result when meet muti-block

I dont sure if change these could cause side effect to other optimizations, but write some test cases for deadcode optimizations and seems works fine.

And test in NoVMP project for follow simple instruments:

Before => After

[Tai7sy]

AppVeyor builds fine dont know why Travis failed in the new test case 😂

[can1357]

It seems these two tests fail, I'll check once I get to a PC if you do not have time. I don't think AppVeyor runs the tests.

Seems to be related to stores under $sp+0 not being discarded?

../VTIL-Tests/dummy.cpp:315: ERROR: CHECK( block->size() == 2 ) is NOT correct!
1998  values: CHECK( 4 == 2 )
1999
2000:: Before:
2001 | | Entry point VIP:       0x1337
2002 | | Stack pointer:         -0x10
2003 | | Already visited?:      N
2004 | | ------------------------
2005 | | 0000: [ PSEUDO ]     -0x8     strq     $sp          -0x8         rax          
2006 | | 0001: [ PSEUDO ]     -0x10    strq     $sp          -0x10        rbx          
2007 | | 0002: [ PSEUDO ]     -0x10    jmpq     0x2000       
2008 | | | | | | Entry point VIP:       0x2000
2009 | | | | | | Stack pointer:         0x10
2010 | | | | | | Already visited?:      N
2011 | | | | | | ------------------------
2012 | | | | | | 0000: [ PSEUDO ]     +0x10    movq     rax          0x0          
2013 | | | | | | 0001: [ PSEUDO ]     +0x10    vexitq   0x0          
2014:: After:
2015 | | Entry point VIP:       0x1337
2016 | | Stack pointer:         -0x10
2017 | | Already visited?:      N
2018 | | ------------------------
2019 | | 0000: [ PSEUDO ]     -0x8     strq     $sp          -0x8         rax          
2020 | | 0001: [ PSEUDO ]     -0x10    strq     $sp          -0x10        rbx          
2021 | | 0002: [ PSEUDO ]     -0x10    jmpq     0x2000       
2022 | | | | | | Entry point VIP:       0x2000
2023 | | | | | | Stack pointer:         0x10
2024 | | | | | | Already visited?:      N
2025 | | | | | | ------------------------
2026 | | | | | | 0000: [ PSEUDO ]     +0x10    movq     rax          0x0          
2027 | | | | | | 0001: [ PSEUDO ]     +0x10    vexitq   0x0          
2028../VTIL-Tests/dummy.cpp:353: ERROR: CHECK( block1->size() == 1 ) is NOT correct!
2029  values: CHECK( 3 == 1 )
2030
2031:: Before:
2032 | | Entry point VIP:       0x1337
2033 | | Stack pointer:         -0x10
2034 | | Already visited?:      N
2035 | | ------------------------
2036 | | 0000: [ PSEUDO ]     -0x8     strq     $sp          -0x8         rax          
2037 | | 0001: [ PSEUDO ]     -0x10    strq     $sp          -0x10        rbx          
2038 | | 0002: [ PSEUDO ]     -0x10    teq      rcx          rcx          0xaabb       
2039 | | 0003: [ PSEUDO ]     -0x10    jsq      rcx          0x2000       0x3000       
2040 | | | | | | Entry point VIP:       0x2000
2041 | | | | | | Stack pointer:         0x10
2042 | | | | | | Already visited?:      N
2043 | | | | | | ------------------------
2044 | | | | | | 0000: [ PSEUDO ]     +0x10    addq     rax          0x1          
2045 | | | | | | 0001: [ PSEUDO ]     +0x10    vexitq   0x0          
2046 | | | | | | Entry point VIP:       0x3000
2047 | | | | | | Stack pointer:         0x10
2048 | | | | | | Already visited?:      N
2049 | | | | | | ------------------------
2050 | | | | | | 0000: [ PSEUDO ]     +0x0     lddq     rax          $sp          -0x8         
2051 | | | | | | 0001: [ PSEUDO ]     +0x10    addq     rax          0x1          
2052 | | | | | | 0002: [ PSEUDO ]     +0x10    vexitq   0x0          
2053:: After:
2054 | | Entry point VIP:       0x1337
2055 | | Stack pointer:         -0x10
2056 | | Already visited?:      N
2057 | | ------------------------
2058 | | 0000: [ PSEUDO ]     -0x8     strq     $sp          -0x8         rax          
2059 | | 0001: [ PSEUDO ]     -0x10    strq     $sp          -0x10        rbx          
2060 | | 0002: [ PSEUDO ]     -0x10    teq      rcx          rcx          0xaabb       
2061 | | 0003: [ PSEUDO ]     -0x10    jsq      rcx          0x2000       0x3000       
2062 | | | | | | Entry point VIP:       0x2000
2063 | | | | | | Stack pointer:         0x10
2064 | | | | | | Already visited?:      N
2065 | | | | | | ------------------------
2066 | | | | | | 0000: [ PSEUDO ]     +0x10    addq     rax          0x1          
2067 | | | | | | 0001: [ PSEUDO ]     +0x10    vexitq   0x0          
2068 | | | | | | Entry point VIP:       0x3000
2069 | | | | | | Stack pointer:         0x10
2070 | | | | | | Already visited?:      N
2071 | | | | | | ------------------------
2072 | | | | | | 0000: [ PSEUDO ]     +0x0     lddq     rax          $sp          -0x8         
2073 | | | | | | 0001: [ PSEUDO ]     +0x10    addq     rax          0x1          
2074 | | | | | | 0002: [ PSEUDO ]     +0x10    vexitq   0x0          

[Tai7sy]

see the https://ci.appveyor.com/project/can1357/vtil-core/builds/41179464#L552 appveyor output the right result.

[mrexodia]

I rebased your branch on master that uses GitHub Actions

[mrexodia]

Thank you!

[wallds]

Does not work for this.

000000000008F250.optimized.vtil.zip

[mrexodia]

Is it bugged now or does it need an improvement to work on that @wallds?

[Tai7sy]

@wallds The case your provided seems when exit vm (block3), the stack is not equal with the vm_entry(block1)

could you provide more simple demo to show the problem? for example, https://github.com/vtil-project/VTIL-Core/blob/4215e438767d42dcf8cbf10a640f81963e8da97a/VTIL-Tests/dummy.cpp#L319-L354

[wallds]

Yeah, stack is not equal.

-0x98+0+0xA0 = 8

Because I included the return instruction.