search

trailofbits / manticore

Symbolic execution tool
https://blog.trailofbits.com/2017/04/27/manticore-symbolic-execution-for-humans/
GNU Affero General Public License v3.0
3.67k stars 472 forks source link

Manticore breaks on simple binary analysis #1291

Closed ex0dus-0x closed 5 years ago

ex0dus-0x commented 5 years ago

Using the most recent Manticore (v0.2.3) through the CLI throws DecodeException before start of program execution. Here is the traceback when testing basic.c from the examples/linux folder:

$ manticore -v ./basic
2018-12-06 16:01:06,955: [12609] m.c.manticore:INFO: Verbosity set to 2.
2018-12-06 16:01:07,132: [12609] m.n.manticore:INFO: Loading program ./basic
2018-12-06 16:01:09,001: [12609] m.c.executor:INFO: load state 0
2018-12-06 16:01:09,011: [12609] m.c.executor:ERROR: Exception: Error decoding instruction @ 0x401b00
Traceback (most recent call last):
  File "/home/ex0dus/Code/manticore/manticore/native/cpu/abstractcpu.py", line 786, in decode_instruction
    insn = self.disasm.disassemble_instruction(code, pc)
  File "/home/ex0dus/Code/manticore/manticore/native/cpu/disasm.py", line 77, in disassemble_instruction
    return next(self.disasm.disasm(code, pc))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ex0dus/Code/manticore/manticore/core/executor.py", line 467, in run
    if not current_state.execute():
  File "/home/ex0dus/Code/manticore/manticore/native/state.py", line 11, in execute
    result = self._platform.execute()
  File "/home/ex0dus/Code/manticore/manticore/platforms/linux.py", line 2229, in execute
    self.current.execute()
  File "/home/ex0dus/Code/manticore/manticore/native/cpu/abstractcpu.py", line 826, in execute
    insn = self.decode_instruction(self.PC)
  File "/home/ex0dus/Code/manticore/manticore/native/cpu/abstractcpu.py", line 788, in decode_instruction
    raise DecodeException(pc, code)
manticore.native.cpu.abstractcpu.DecodeException: Error decoding instruction @ 0x401b00

2018-12-06 16:01:09,029: [12609] m.c.manticore:INFO: Results in /home/ex0dus/linux/mcore_2gs5v1qu
2018-12-06 16:01:09,030: [12609] m.c.manticore:INFO: Total time: 0.10796427726745605
2018-12-06 16:01:09,030: [12609] m.c.plugin:INFO: Instructions executed: 0
2018-12-06 16:01:09,031: [12609] m.c.plugin:INFO: Coverage: 0 different instructions executed

This was ran on an x86_64 Arch Linux machine, with Python 3.7.1. The address that threw the exception, 0x401b00, represents the _start entry point

(gdb) disass 0x401b00
Dump of assembler code for function _start:
   0x0000000000401b00 <+0>:     endbr64
   0x0000000000401b04 <+4>:     xor    ebp,ebp
   0x0000000000401b06 <+6>:     mov    r9,rdx
   0x0000000000401b09 <+9>:     pop    rsi
   0x0000000000401b0a <+10>:    mov    rdx,rsp
   0x0000000000401b0d <+13>:    and    rsp,0xfffffffffffffff0
   0x0000000000401b11 <+17>:    push   rax
   0x0000000000401b12 <+18>:    push   rsp
   0x0000000000401b13 <+19>:    mov    r8,0x402990
   0x0000000000401b1a <+26>:    mov    rcx,0x4028f0
   0x0000000000401b21 <+33>:    mov    rdi,0x4015e0
   0x0000000000401b28 <+40>:    addr32 call 0x401e90 <__libc_start_main>
   0x0000000000401b2e <+46>:    hlt
disconnect3d commented 5 years ago

It's probably bcoz of capstone not being able to decode the endbr64 instruction: https://github.com/trailofbits/manticore/issues/1184

We either need to decode it by ourselves :( or try the updated capstone (?).

disconnect3d commented 5 years ago

This will be fixed with https://github.com/trailofbits/manticore/pull/1312.

njfox commented 5 years ago

I believe I am running into this same issue on Arch Linux in a fresh python 3.7.2 venv. I've installed with pip install manticore[native] and I'm getting the following errors:

2019-01-18 17:21:23,297: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000401000:      f3 0f 1e fa     endbr64
2019-01-18 17:21:23,332: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000401bf0:      f3 0f 1e fa     endbr64
2019-01-18 17:21:23,349: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x000000000047c4c0:      f3 0f 1e fa     endbr64
2019-01-18 17:21:23,438: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000401660:      f3 0f 1e fa     endbr64
2019-01-18 17:21:25,247: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x000000000043f4f0:      f3 0f 1e fa     endbr64
2019-01-18 17:21:25,288: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000407660:      f3 0f 1e fa     endbr64
2019-01-18 17:21:25,298: [20714] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x000000000044bfe0:      f3 0f 1e fa     endbr64
2019-01-18 17:21:25,339: [20714] m.c.executor:ERROR: Exception: Error decoding instruction @ 0x44c038
Traceback (most recent call last):
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/native/cpu/abstractcpu.py", line 786, in decode_instruction
    insn = self.disasm.disassemble_instruction(code, pc)
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/native/cpu/disasm.py", line 77, in disassemble_instruction
    return next(self.disasm.disasm(code, pc))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/core/executor.py", line 454, in run
    if not current_state.execute():
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/native/state.py", line 18, in execute
    result = self._platform.execute()
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/platforms/linux.py", line 2233, in execute
    self.current.execute()
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/native/cpu/abstractcpu.py", line 826, in execute
    insn = self.decode_instruction(self.PC)
  File "/home/nick/venvs/manticore-test/lib/python3.7/site-packages/manticore/native/cpu/abstractcpu.py", line 788, in decode_instruction
    raise DecodeException(pc, code)
manticore.native.cpu.abstractcpu.DecodeException: Error decoding instruction @ 0x44c038

I tested in a Python 3.6 venv on a Kali machine and I had no issues--is there a chance there could be an issue with Python 3.7?

disconnect3d commented 5 years ago

@njfox can you confirm that you have Capstone 4.0 or 4.0.1 installed? Can you try import capstone; print(capstone.__version__) or eventually cstool --version (if you have the tool from capstone, idk if it ships with it)?

disconnect3d commented 5 years ago

@njfox oh wait. This Capstone update (https://github.com/trailofbits/manticore/commit/3a0159155bc85d09fb25ed4b9c1c92c98b5097e5) didn't get into the 0.2.4 release. So it should work, but on the master branch.

njfox commented 5 years ago

Thanks for looking into it @disconnect3d. I'm getting the same behavior with capstone 4.0.1 (compiled from master):

Python 3.7.2 (default, Jan 10 2019, 23:51:51) 
[GCC 8.2.1 20181127] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import capstone
>>> print(capstone.__version__)
4.0.1

Weirdly enough, I ran through the testcases @aquynh used in the issue thread over at capstone and the instructions appear to be supported:

[nick@arch linux]$ cstool x32 "f3 0f 1e fa"
 0  f3 0f 1e fa                                      endbr64
[nick@arch linux]$ cstool x32 "f3 0f 1e fb"
 0  f3 0f 1e fb                                      endbr32

Let me know if I can provide any other information to help troubleshoot.

njfox commented 5 years ago

Update: While I am getting spammed about unimplemented endbr64 instructions, it looks like the actual instruction that is causing the crash is rdsspq rax:

2019-01-20 13:42:50,728: [18779] m.c.executor:ERROR: Exception: Error decoding instruction @ 0x44c038
Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/manticore/native/cpu/abstractcpu.py", line 786, in decode_instruction
    insn = self.disasm.disassemble_instruction(code, pc)
  File "/usr/lib/python3.7/site-packages/manticore/native/cpu/disasm.py", line 77, in disassemble_instruction
    return next(self.disasm.disasm(code, pc))
StopIteration

[...]

gdb-peda$ x/10i 0x44c038
   0x44c038 <__sigsetjmp+88>:   rdsspq rax
   0x44c03d <__sigsetjmp+93>:   mov    QWORD PTR [rdi+0x58],rax
   0x44c041 <__sigsetjmp+97>:   jmp    0x44c050 <__sigjmp_save>
   0x44c046:    nop    WORD PTR cs:[rax+rax*1+0x0]
   0x44c050 <__sigjmp_save>:    endbr64 
   0x44c054 <__sigjmp_save+4>:  push   rbx
   0x44c055 <__sigjmp_save+5>:  mov    rbx,rdi
   0x44c058 <__sigjmp_save+8>:  test   esi,esi
   0x44c05a <__sigjmp_save+10>: jne    0x44c068 <__sigjmp_save+24>
   0x44c05c <__sigjmp_save+12>: mov    DWORD PTR [rbx+0x40],esi

I've confirmed I'm using capstone 4.0.1 and the master branch of manticore, so I'm not sure why I'm still getting the warnings about endbr64 not being implemented. But do you think the error decoding rdsspq rax indicative of a different issue in capstone?

Note that cstool can't parse rdsspq rax's opcode either:

$ objdump -D -M intel basic | grep "rdsspq rax"
  44c038:       f3 48 0f 1e c8          rdsspq rax
$ cstool x64 "f3 48 0f 1e c8"
ERROR: invalid assembly code
aquynh commented 5 years ago

Can you provide the hexcode of this "rdsspq rax"?

disconnect3d commented 5 years ago

@aquynh 

$ pwn asm -c64 'rdsspq rax'
f3480f1ec8

(pwn is a CLI tool installed with pwntools, in this case it used the /usr/bin/x86_64-linux-gnu-as program under the hood)

EDIT: Note that there is also e.g. rdsspd [32-bit reg]. It seems that we miss instructions which are related to CET: https://software.intel.com/sites/default/files/managed/4d/2a/control-flow-enforcement-technology-preview.pdf

njfox commented 5 years ago

I believe it’s f3 48 0f 1e c8 based on the output from objdump and keystone-engine

disconnect3d commented 5 years ago

I have added an issue for those instructions in Capstone repo: https://github.com/aquynh/capstone/issues/1346

aquynh commented 5 years ago

Capstone reports these instructions as "invalid", because they are not supported yet. regardless, your code should not crash (Capstone does not crash).

geremycondra commented 3 years ago

Should I still be seeing this on master with capstone 4.0.2?

ekilmer commented 3 years ago

@geremycondra Are you seeing the same error message?

You may also want to try the capstone-5-dev branch for Manticore, from this PR https://github.com/trailofbits/manticore/pull/1701 as it uses a pretty recent commit from the next branch on Capstone.

geremycondra commented 3 years ago

@ekilmer yes. I have pretty much the simplest possible test.

Here's my C code:

int main(void) {
    return 0;
}

Here's how I compile it:

gcc -static -o simple simple.c

Here's my Manticore script, taken from the examples with minor modifications:

#!/usr/bin/env python3

import sys
from manticore.native import Manticore

# This example demonstrates loading a simple binary in Manticore,
# running it to completion without any callbacks or instrumentation
# and producing basic information about the paths explored

if __name__ == "__main__":
    path = "./simple"
    # Create a new Manticore object
    m = Manticore(path)
    m.run()

Here's the error I get:

2020-10-26 14:25:16,996: [1111516] m.c.worker:ERROR: Exception in state 0: DecodeException('Error decoding instruction @ 0x45c109')
Traceback (most recent call last):
  File "/home/gcondra/projects/manticore/manticore/native/cpu/abstractcpu.py", line 937, in decode_instruction
    insn = self.disasm.disassemble_instruction(code, pc)
  File "/home/gcondra/projects/manticore/manticore/native/cpu/disasm.py", line 79, in disassemble_instruction
    return next(self.disasm.disasm(code, pc))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gcondra/projects/manticore/manticore/core/worker.py", line 121, in run
    current_state.execute()
  File "/home/gcondra/projects/manticore/manticore/native/state.py", line 192, in execute
    result = self._platform.execute()
  File "/home/gcondra/projects/manticore/manticore/platforms/linux.py", line 2906, in execute
    self.current.execute()
  File "/home/gcondra/projects/manticore/manticore/native/cpu/abstractcpu.py", line 987, in execute
    insn = self.decode_instruction(curpc)
  File "/home/gcondra/projects/manticore/manticore/native/cpu/abstractcpu.py", line 939, in decode_instruction
    raise DecodeException(pc, code)
manticore.native.cpu.abstractcpu.DecodeException: Error decoding instruction @ 0x45c109

And here's the disassembly around the relevant instruction:

(gdb) x/10i 0x45c109
   0x45c109 <__sigsetjmp+89>:   rdsspq %rax
   0x45c10e <__sigsetjmp+94>:   mov    %rax,0x58(%rdi)
   0x45c112 <__sigsetjmp+98>:   jmpq   0x45c120 <__sigjmp_save>
   0x45c117:    nopw   0x0(%rax,%rax,1)
   0x45c120 <__sigjmp_save>:    endbr64 
   0x45c124 <__sigjmp_save+4>:  push   %rbx
   0x45c125 <__sigjmp_save+5>:  mov    %rdi,%rbx
   0x45c128 <__sigjmp_save+8>:  test   %esi,%esi
   0x45c12a <__sigjmp_save+10>: jne    0x45c138 <__sigjmp_save+24>
   0x45c12c <__sigjmp_save+12>: mov    %esi,0x40(%rbx)

Here's the current state of Manticore I'm installing from:

gcondra@gcondra-ThinkPad-X250:~/projects/manticore$ git log -n 1
commit 2152023f390a7dc45cf16cd5cc283d0cfa93b8ee (HEAD -> master, origin/master, origin/HEAD)
Author: William Woodruff <william.woodruff@trailofbits.com>
Date:   Tue Oct 6 17:07:25 2020 -0400

    native/cpu/x86: Add support for CPUID EAX=80000000h (#1811)

I'm running a new and pretty bog-standard Linux Mint install, with GCC 0.9.3, linux kernel version 5.4.0-26-generic, and glibc 2.31, if that helps.

Capstone is at: 

commit 4457d451aad63ed7ac4ef259200d165d157f1554 (HEAD -> master, origin/master, origin/HEAD)
Author: StalkR <stalkr@stalkr.net>
Date:   Wed Sep 16 09:05:55 2020 +0000

    MCInst: fix uninitialized value in operand value (#1684)

    https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=14912

I will try the new branch, but don't see anything directly referenced there that seems to address this problem. Could just not be obvious to me though.

Edit: I forgot to mention, I dropped a print into disasm.py, so the line number there may be off by one relative to master. I can rerun and repost without the print if that helps.

geremycondra commented 3 years ago

@ekilmer Running with capstone-5-dev still complains about pshufb and crashes at the same place (rdsspq %rax). As you can see, the two places where that occurs here are in sigsetjmp and longjmp.

Borrowing some code from one of the threads about this, you can see how this winds up looking to capstone:

#! /usr/bin/env python3

from capstone import *

print('Capstone', __version__)

rdsspq = b'\x90' + b'\xf3\x48\x0f\x1e\xc8' + b'\x90'

def decode(mode, code, skipdata):
    md = Cs(CS_ARCH_X86, mode)
    md.skipdata = skipdata
    md.detail = True
    for i in md.disasm(code, 0x1000):
        print("0x%x:\t%s\t%s" % (i.address, i.mnemonic, i.op_str))
    print('')

print('>>> x86, noskip, rdsspq %rax')
decode(CS_MODE_32, rdsspq, False)

print('>>> x86-64, noskip, rdsspq %rax')
decode(CS_MODE_64, rdsspq, False)

print('>>> x86, skip, rdsspq %rax')
decode(CS_MODE_32, rdsspq, True)

print('>>> x86-64, skip, rdsspq %rax')
decode(CS_MODE_64, rdsspq, True)

Output on my machine:

Capstone 4.0.2
>>> x86, noskip, rdsspq %rax
0x1000: nop 
0x1001: dec eax

>>> x86-64, noskip, rdsspq %rax
0x1000: nop 

>>> x86, skip, rdsspq %rax
0x1000: nop 
0x1001: dec eax
0x1003: .byte   0x0f
0x1004: push    ds
0x1005: .byte   0xc8
0x1006: nop 

>>> x86-64, skip, rdsspq %rax
0x1000: nop 
0x1001: .byte   0xf3
0x1002: .byte   0x48
0x1003: .byte   0x0f
0x1004: .byte   0x1e
0x1005: .byte   0xc8
0x1006: nop
ekilmer commented 3 years ago

Did you reinstall Manticore while on the capstone-5-dev branch? That branch has upgrade Capstone to the next branch, which should hopefully support that instruction.

This is my output for your Capstone script. Note that I am in a virtual environment (manticore-N8Ra3REL) that I've installed the capstone-5-dev branch of Manticore within.

~/src/manticore capstone-5-dev* 6s
manticore-N8Ra3REL ❯ python test.py
Capstone 5.0.0
>>> x86, noskip, rdsspq %rax
0x1000: nop
0x1001: dec     eax

>>> x86-64, noskip, rdsspq %rax
0x1000: nop
0x1001: rdsspq  rax
0x1006: nop

>>> x86, skip, rdsspq %rax
0x1000: nop
0x1001: dec     eax
0x1003: .byte   0x0f
0x1004: push    ds
0x1005: .byte   0xc8
0x1006: nop

>>> x86-64, skip, rdsspq %rax
0x1000: nop
0x1001: rdsspq  rax
0x1006: nop

I am running an updated Fedora 32 VM with your simple.c and sample Python script (named run.py) on gcc 10.2.1 and Manticore is able to run the binary, albeit with a WARNING message that we don't have symbolic semantics implemented for rdsspq rax, but this can be ignored for all concrete cases:

~/src/manticore capstone-5-dev*
manticore-N8Ra3REL ❯ gcc --version
gcc (GCC) 10.2.1 20201016 (Red Hat 10.2.1-6)
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

~/src/manticore capstone-5-dev*
manticore-N8Ra3REL ❯ ./run.py
WARNING:root:Consider installing rusty-rlp to improve pyrlp performance with a rust based backend
2020-10-27 10:48:51,246: [8044] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x000000000044f309:       f3 48 0f 1e c8  rdsspq  rax
Pl0414141 commented 2 years ago

Hi,

i have the same problem with the 'master' branch of Manticore and capstone 4.0.2 and 5 dev in Ubuntu 20.04TLS

root@ubuntu:/home/ubuntu/Desktop/bof/test# manticore xxx 2021-10-29 23:40:28,402: [2309] m.n.manticore:INFO: Loading program xxx 2021-10-29 23:40:28,743: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb0df0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,095: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffbc0a0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,116: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcb6f0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,216: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc9040: f3 0f 1e fa endbr64 2021-10-29 23:40:29,233: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffccf30: f3 0f 1e fa endbr64 2021-10-29 23:40:29,281: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,295: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,307: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,321: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,334: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,344: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,358: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffccf80: f3 0f 1e fa endbr64 2021-10-29 23:40:29,376: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb15e0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,397: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc3b0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,412: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffbc200: f3 0f 1e fa endbr64 2021-10-29 23:40:29,430: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb00c0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,435: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc5b0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,441: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb00a0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,445: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc490: f3 0f 1e fa endbr64 2021-10-29 23:40:29,471: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2380: f3 0f 1e fa endbr64 2021-10-29 23:40:29,533: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffbc160: f3 0f 1e fa endbr64 2021-10-29 23:40:29,540: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb0150: f3 0f 1e fa endbr64 2021-10-29 23:40:29,556: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb0160: f3 0f 1e fa endbr64 2021-10-29 23:40:29,688: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb286e: 66 0f c6 c3 02 shufpd xmm0, xmm3, 2 2021-10-29 23:40:29,704: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffce5f0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,839: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffbc0a0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,859: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc1d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,867: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffce250: f3 0f 1e fa endbr64 2021-10-29 23:40:29,912: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb9050: f3 0f 1e fa endbr64 2021-10-29 23:40:29,923: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffbf770: f3 0f 1e fa endbr64 2021-10-29 23:40:29,935: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc95d0: f3 0f 1e fa endbr64 2021-10-29 23:40:29,984: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd1e10: f3 0f 1e fa endbr64 2021-10-29 23:40:30,022: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb00a0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,026: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc490: f3 0f 1e fa endbr64 2021-10-29 23:40:30,050: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2370: f3 0f 1e fa endbr64 2021-10-29 23:40:30,062: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2370: f3 0f 1e fa endbr64 2021-10-29 23:40:30,154: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb00a0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,158: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc490: f3 0f 1e fa endbr64 2021-10-29 23:40:30,176: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb00a0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,179: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcc490: f3 0f 1e fa endbr64 2021-10-29 23:40:30,202: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2570: f3 0f 1e fa endbr64 2021-10-29 23:40:30,223: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2570: f3 0f 1e fa endbr64 2021-10-29 23:40:30,238: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2570: f3 0f 1e fa endbr64 2021-10-29 23:40:30,254: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd2570: f3 0f 1e fa endbr64 2021-10-29 23:40:30,278: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc11e0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,312: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb31a4: 66 0f c6 c1 02 shufpd xmm0, xmm1, 2 2021-10-29 23:40:30,321: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc3450: f3 0f 1e fa endbr64 2021-10-29 23:40:30,334: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffc11d0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,345: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffcdde0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,363: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffbe510: f3 0f 1e fa endbr64 2021-10-29 23:40:30,416: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb87a0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,423: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffd1be0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,454: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffb0090: f3 0f 1e fa endbr64 2021-10-29 23:40:30,458: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffccc40: f3 0f 1e fa endbr64 2021-10-29 23:40:30,471: [2309] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x00007ffffffce2e0: f3 0f 1e fa endbr64 2021-10-29 23:40:30,499: [2309] m.c.worker:ERROR: Exception in state 0: DecodeException('Error decoding instruction @ 0x7ffffffce333') Traceback (most recent call last): File "/home/ubuntu/Desktop/bof/manticore/manticore/native/cpu/abstractcpu.py", line 982, in decode_instruction insn = self.disasm.disassemble_instruction(code, pc) File "/home/ubuntu/Desktop/bof/manticore/manticore/native/cpu/disasm.py", line 77, in disassemble_instruction return next(self.disasm.disasm(code, pc))

Anyone can help? Thank you

tr4nc3 commented 2 years ago

Installed manticore 

(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ manticore --version
Manticore 0.3.6
(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ manticore basic
2022-02-17 08:33:47,069: [5564] m.n.manticore:INFO: Loading program basic
2022-02-17 08:33:47,193: [5564] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000401ca0:   f3 0f 1e fa endbr64
2022-02-17 08:33:47,200: [5564] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000402150:   f3 0f 1e fa endbr64
2022-02-17 08:33:47,206: [5564] m.n.c.abstractcpu:WARNING: Unimplemented instruction: 0x0000000000401cd0:   f3 0f 1e fa endbr64`
:
:
:

The version of capstone is 4.0.2. 

(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ python
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import capstone
>>> print(capstone.__version__)
4.0.2

Any advice on how to fix this?

Update: I also tried with capstone v5.0.0 (the next branch).

(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ manticore basic
2022-02-17 09:17:55,144: [8364] m.n.manticore:INFO: Loading program basic
2022-02-17 09:17:56,198: [8364] m.c.worker:ERROR: Exception in state 0: DecodeException('Error decoding instruction @ 0x45c489')
Traceback (most recent call last):
  File "/home/ubuntu/manticore/manticore/native/cpu/abstractcpu.py", line 982, in decode_instruction
    insn = self.disasm.disassemble_instruction(code, pc)
  File "/home/ubuntu/manticore/manticore/native/cpu/disasm.py", line 77, in disassemble_instruction
    return next(self.disasm.disasm(code, pc))
StopIteration

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/ubuntu/manticore/manticore/core/worker.py", line 137, in run
    current_state.execute()
  File "/home/ubuntu/manticore/manticore/native/state.py", line 298, in execute
    result = self._platform.execute()
  File "/home/ubuntu/manticore/manticore/platforms/linux.py", line 2928, in execute
    self.current.execute()
  File "/home/ubuntu/manticore/manticore/native/cpu/abstractcpu.py", line 1026, in execute
    insn = self.decode_instruction(curpc)
  File "/home/ubuntu/manticore/manticore/native/cpu/abstractcpu.py", line 984, in decode_instruction
    raise DecodeException(pc, code)
manticore.native.cpu.abstractcpu.DecodeException: Error decoding instruction @ 0x45c489

2022-02-17 09:17:56,204: [8364] m.c.manticore:INFO: Results in /home/ubuntu/manticore/examples/linux/mcore_o593yq1q
2022-02-17 09:17:56,204: [8364] m.c.manticore:INFO: Total time: 0.9428601264953613

These are some other commands I ran to diagnose the issue:

(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ cstool -v
cstool for Capstone Disassembler, v5.0.0
Capstone build: x86=1 arm=1 arm64=1 mips=1 ppc=1 sparc=1 sysz=1 xcore=1 m68k=1 tms320c64x=1 m680x=1 evm=1 wasm=1 mos65xx=1 bpf=1 riscv=1
(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ cstool x64 "f3 48 0f 1e c8"
 0  f3 48 0f 1e c8                                   rdsspq rax
(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ gdb ./basic
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./basic...
(No debugging symbols found in ./basic)
gdb-peda$ x/10i 0x45c489
   0x45c489 <__sigsetjmp+89>:   rdsspq rax
   0x45c48e <__sigsetjmp+94>:   mov    QWORD PTR [rdi+0x58],rax
   0x45c492 <__sigsetjmp+98>:   jmp    0x45c4a0 <__sigjmp_save>
   0x45c497:    nop    WORD PTR [rax+rax*1+0x0]
   0x45c4a0 <__sigjmp_save>:    endbr64
   0x45c4a4 <__sigjmp_save+4>:  push   rbx
   0x45c4a5 <__sigjmp_save+5>:  mov    rbx,rdi
   0x45c4a8 <__sigjmp_save+8>:  test   esi,esi
   0x45c4aa <__sigjmp_save+10>: jne    0x45c4b8 <__sigjmp_save+24>
   0x45c4ac <__sigjmp_save+12>: mov    DWORD PTR [rbx+0x40],esi
gdb-peda$ quit
(test-py) ubuntu@ip-172-31-68-72:~/manticore/examples/linux$ objdump -D basic |  grep rdsspq
  45c489:   f3 48 0f 1e c8          rdsspq %rax
  47eb49:   f3 48 0f 1e c8          rdsspq %rax
tr4nc3 commented 2 years ago

I fixed this by deleting the the capstone directory in my python site-packages directory. This forced manticore to use capstone 5.0.0 (somehow previously manticore was stuck using 4.0.2). This fixed my issue.