search

utds3lab / multiverse

A static binary rewriter that does not use heuristics
GNU Lesser General Public License v3.0
299 stars 33 forks source link

Error: operand type mismatch for `pop', when trying to rewrite /usr/bin/true (64-bit) #4

Open mewmew opened 6 years ago

mewmew commented 6 years ago

On rev b90ffd6df1d1470bce097e5afb7172ba3cdb3b3a, the following error is output when running python2 ./multiverse.py /usr/bin/true

I've uploaded a copy of the true command to https://ufile.io/nyh9c

EDIT: note, the above download link is only valid for 30 days.

Found .text
binary does not contain plt
Writing as main binary
Base address: 0x0
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
[ERROR] There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-u2XtGd/step2', '/tmp/pwn-asm-u2XtGd/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-u2XtGd/step1: Assembler messages:
    /tmp/pwn-asm-u2XtGd/step1:7: Error: operand type mismatch for `pop'

[ERROR] An error occurred while assembling:
       1: .section .shellcode,"awx"
       2: .global _start
       3: .global __start
       4: _start:
       5: __start:
       6: .intel_syntax noprefix
       7:     pop eax
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 665, in asm
        _run(assembler + ['-o', step2, step1])
      File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 383, in _run
        log.error(msg)
      File "/usr/lib/python2.7/site-packages/pwnlib/log.py", line 417, in error
        raise PwnlibException(message % args)
    PwnlibException: There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-u2XtGd/step2', '/tmp/pwn-asm-u2XtGd/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-u2XtGd/step1: Assembler messages:
    /tmp/pwn-asm-u2XtGd/step1:7: Error: operand type mismatch for `pop'

Traceback (most recent call last):
  File "./multiverse.py", line 317, in <module>
    rewriter.rewrite(args.filename,args.arch)
  File "./multiverse.py", line 191, in rewrite
    mapping = mapper.gen_mapping()
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/brute_force_mapper.py", line 53, in gen_mapping
    newins = self.translator.translate_one(ins,None) #In this pass, the mapping is incomplete
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_translator.py", line 24, in translate_one
    return self.translate_ret(ins,mapping)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_translator.py", line 64, in translate_ret
    code = asm(template_before)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_assembler.py", line 119, in asm
    code+=_asm(line)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_assembler.py", line 28, in _asm
    code = pwn.asm(text)
  File "/usr/lib/python2.7/site-packages/pwnlib/context/__init__.py", line 1392, in setter
    return function(*a)
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 710, in asm
    log.exception("An error occurred while assembling:\n%s" % lines)
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 665, in asm
    _run(assembler + ['-o', step2, step1])
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 383, in _run
    log.error(msg)
  File "/usr/lib/python2.7/site-packages/pwnlib/log.py", line 417, in error
    raise PwnlibException(message % args)
pwnlib.exception.PwnlibException: There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-u2XtGd/step2', '/tmp/pwn-asm-u2XtGd/step1']:
It had the exitcode 1.
It had this on stdout:
/tmp/pwn-asm-u2XtGd/step1: Assembler messages:
/tmp/pwn-asm-u2XtGd/step1:7: Error: operand type mismatch for `pop'
utds3lab commented 6 years ago

This issue is likely something to do with pwntools. It appears that pwntools is attempting to assemble 32-bit assembly as 64-bit assembly. I can rewrite the binary on my test machine without encountering this bug, so there may be some issue with pwntools itself. However, the rewritten binary will not run for a few reasons.

Multiverse by default assumes the architecture is 32-bit x86. If you want to rewrite a 64-bit binary, you need to include the flag --arch x86-64. However, the binary you are attempting to rewrite is a Position-Independent Executable (PIE). We do support Position-Independent Code (PIC), but only in libraries. This isn't an inherent limitation of our approach, but some limitations with the way we generate a final rewritten executable mean that we don't currently support the main executable being position-independent. That will require some extra engineering that hasn't been done yet. I apologize for any inconvenience this may cause.

mewmew commented 6 years ago

Hi @utds3lab,

Thanks for the detailed explanation. Of course, there will always be edge cases, good to know that the PIE related one is not a design issue, but rather based on implementation time.

What version of pwntools do you have? I keep running into the operand type mismatch forpop'` error also for 32-bit binaries.

u@x220 /u/bin> file /usr/bin/{jad,wine}
/usr/bin/jad:  ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for GNU/Linux 2.0.0, stripped, too many notes (256)
/usr/bin/wine: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=a0030d47f13af4b7f865977133c4d1fa94793bb6, stripped

For 32-bit binary wine: https://ufile.io/zyikp

u@x220 ~/D/g/s/g/m/multiverse> python2 ./multiverse.py --arch x86 /usr/bin/wine
Found .text
Writing as main binary
Base address: 0x7c000000
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
[ERROR] There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-MDhWG2/step2', '/tmp/pwn-asm-MDhWG2/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-MDhWG2/step1: Assembler messages:
    /tmp/pwn-asm-MDhWG2/step1:7: Error: operand type mismatch for `pop'

[ERROR] An error occurred while assembling:
       1: .section .shellcode,"awx"
       2: .global _start
       3: .global __start
       4: _start:
       5: __start:
       6: .intel_syntax noprefix
       7:     pop eax
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 665, in asm
        _run(assembler + ['-o', step2, step1])
      File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 383, in _run
        log.error(msg)
      File "/usr/lib/python2.7/site-packages/pwnlib/log.py", line 417, in error
        raise PwnlibException(message % args)
    PwnlibException: There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-MDhWG2/step2', '/tmp/pwn-asm-MDhWG2/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-MDhWG2/step1: Assembler messages:
    /tmp/pwn-asm-MDhWG2/step1:7: Error: operand type mismatch for `pop'

Traceback (most recent call last):
  File "./multiverse.py", line 317, in <module>
    rewriter.rewrite(args.filename,args.arch)
  File "./multiverse.py", line 191, in rewrite
    mapping = mapper.gen_mapping()
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/brute_force_mapper.py", line 53, in gen_mapping
    newins = self.translator.translate_one(ins,None) #In this pass, the mapping is incomplete
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_translator.py", line 24, in translate_one
    return self.translate_ret(ins,mapping)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_translator.py", line 64, in translate_ret
    code = asm(template_before)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_assembler.py", line 119, in asm
    code+=_asm(line)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_assembler.py", line 28, in _asm
    code = pwn.asm(text)
  File "/usr/lib/python2.7/site-packages/pwnlib/context/__init__.py", line 1392, in setter
    return function(*a)
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 710, in asm
    log.exception("An error occurred while assembling:\n%s" % lines)
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 665, in asm
    _run(assembler + ['-o', step2, step1])
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 383, in _run
    log.error(msg)
  File "/usr/lib/python2.7/site-packages/pwnlib/log.py", line 417, in error
    raise PwnlibException(message % args)
pwnlib.exception.PwnlibException: There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-MDhWG2/step2', '/tmp/pwn-asm-MDhWG2/step1']:
It had the exitcode 1.
It had this on stdout:
/tmp/pwn-asm-MDhWG2/step1: Assembler messages:
/tmp/pwn-asm-MDhWG2/step1:7: Error: operand type mismatch for `pop'

And for 32-bit binary jad: https://ufile.io/m8ijp

u@x220 ~/D/g/s/g/m/multiverse> python2 ./multiverse.py --arch x86 /usr/bin/jad
Found .text
binary does not contain plt
Writing as main binary
Base address: 0x8048000
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
[ERROR] There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-lXz0Es/step2', '/tmp/pwn-asm-lXz0Es/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-lXz0Es/step1: Assembler messages:
    /tmp/pwn-asm-lXz0Es/step1:7: Error: operand type mismatch for `pop'

[ERROR] An error occurred while assembling:
       1: .section .shellcode,"awx"
       2: .global _start
       3: .global __start
       4: _start:
       5: __start:
       6: .intel_syntax noprefix
       7:     pop eax
    Traceback (most recent call last):
      File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 665, in asm
        _run(assembler + ['-o', step2, step1])
      File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 383, in _run
        log.error(msg)
      File "/usr/lib/python2.7/site-packages/pwnlib/log.py", line 417, in error
        raise PwnlibException(message % args)
    PwnlibException: There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-lXz0Es/step2', '/tmp/pwn-asm-lXz0Es/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-lXz0Es/step1: Assembler messages:
    /tmp/pwn-asm-lXz0Es/step1:7: Error: operand type mismatch for `pop'

Traceback (most recent call last):
  File "./multiverse.py", line 317, in <module>
    rewriter.rewrite(args.filename,args.arch)
  File "./multiverse.py", line 191, in rewrite
    mapping = mapper.gen_mapping()
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/brute_force_mapper.py", line 53, in gen_mapping
    newins = self.translator.translate_one(ins,None) #In this pass, the mapping is incomplete
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_translator.py", line 24, in translate_one
    return self.translate_ret(ins,mapping)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_translator.py", line 64, in translate_ret
    code = asm(template_before)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_assembler.py", line 119, in asm
    code+=_asm(line)
  File "/home/u/Desktop/go/src/github.com/mewbak/multiverse/x86_assembler.py", line 28, in _asm
    code = pwn.asm(text)
  File "/usr/lib/python2.7/site-packages/pwnlib/context/__init__.py", line 1392, in setter
    return function(*a)
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 710, in asm
    log.exception("An error occurred while assembling:\n%s" % lines)
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 665, in asm
    _run(assembler + ['-o', step2, step1])
  File "/usr/lib/python2.7/site-packages/pwnlib/asm.py", line 383, in _run
    log.error(msg)
  File "/usr/lib/python2.7/site-packages/pwnlib/log.py", line 417, in error
    raise PwnlibException(message % args)
pwnlib.exception.PwnlibException: There was an error running ['/usr/bin/as', '-64', '-o', '/tmp/pwn-asm-lXz0Es/step2', '/tmp/pwn-asm-lXz0Es/step1']:
It had the exitcode 1.
It had this on stdout:
/tmp/pwn-asm-lXz0Es/step1: Assembler messages:
/tmp/pwn-asm-lXz0Es/step1:7: Error: operand type mismatch for `pop'

Cheers, /u

utds3lab commented 6 years ago

I have successfully run it with pwntools versions 2.2 and 3.12.0. Could you try the solution of adding the bits=32 option as was tried by @ehsmeng in this issue? It looks like they had a similar problem that was solved by this. Please let me know if this change works for you, and also what version of pwntools you are currently using.

mewmew commented 6 years ago

For the records, I'm using pwntools version 3.12.0.

Applying the change to x86_assembler.py as suggested by @ehsmeng worked better. Now multiverse was capable of rewriting the 32-bit jad binary:

u@x220 ~/D/g/s/g/m/multiverse> python2 ./multiverse.py --arch x86 ./foo/jad 
Found .text
binary does not contain plt
Writing as main binary
Base address: 0x8048000
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
WARNING: silently converting "mov eax, word ptr [<number>]" to "mov eax, dword ptr [<number>]"
Disassembly 100% complete...
final offset for mapping is: 0x3c8c68
Generating new code...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
WARNING: silently converting "mov eax, word ptr [<number>]" to "mov eax, dword ptr [<number>]"
Disassembly 100% complete...
mapping is being placed at offset: 0x3c8c68
last address in mapping was 0x8119be0
just set global_flag value to 0x-4
66
72
last address in mapping was 0x8119be0
0x8119be0
code increase: 761%
lookup w/unknown mapping 66
lookup w/known mapping 66
new entry point: 0x93c8c13
new _start point: 0x9100c97
global lookup: 0x7000000
local lookup: 0x0
secondary local lookup: 0x8f
mapping offset: 0x3c8c68
length of new contents: 0x4
.text section too small to hold phdrs (or 32-bit binary); using other heuristics to relocate phdrs
WARNING   elfmanip.relocate_phdrs :: All methods failed
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment

However, the rewritten binary does not seem to work.

u@x220 ~/D/g/s/g/m/m/foo> ls -la ./jad-r
-rwxr-xr-x 1 u users 961576 May 17 03:28 ./jad-r*
u@x220 ~/D/g/s/g/m/m/foo> file ./jad-r
./jad-r: ERROR: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for GNU/Linux 2.0.0 error reading (Invalid argument)
u@x220 ~/D/g/s/g/m/m/foo> ./jad-r
fish: “./jad-r” terminated by signal SIGSEGV (Address boundary error)

A copy of jad-r has been uploaded to https://ufile.io/ygbg3

Cheers, /u

utds3lab commented 6 years ago

Please refer to the README for some necessary details about rewriting and running binaries. The default settings require that you rewrite all the libraries of the binary as well. If you want to only rewrite the binary and not the libraries, you need to pass the --execonly flag.

Also, you need to run rewritten binaries with LD_BIND_NOW=1, as that environment variable tells the loader to not do lazy loading while a rewritten binary is running. Since we aren't rewriting the loader, it can redirect control flow to incorrect addresses unless we disable lazy loading.

I suggest getting the simplest64 or simplest32 test program working first, as getting it working is documented in the README.

mewmew commented 6 years ago

Thanks for the info, I should have read up more in advance.

I checked the jad binary, and it seems to be statically linked, so it should work?

I also tried running with LD_BIND_NOW=1 and --execonly, but got the same results:

[u@x220 multiverse]$ LD_BIND_NOW=1 python2 ./multiverse.py --arch=x86 --execonly foo/jad 
Found .text
binary does not contain plt
Writing ONLY main binary, without support for rewritten .so files
Base address: 0x8048000
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
WARNING: silently converting "mov eax, word ptr [<number>]" to "mov eax, dword ptr [<number>]"
Disassembly 100% complete...
final offset for mapping is: 0x3c8c95
Generating new code...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
WARNING: silently converting "mov eax, word ptr [<number>]" to "mov eax, dword ptr [<number>]"
Disassembly 100% complete...
mapping is being placed at offset: 0x3c8c95
last address in mapping was 0x8119be0
just set global_flag value to 0x-4
111
117
last address in mapping was 0x8119be0
0x8119be0
code increase: 761%
lookup w/unknown mapping 62
lookup w/known mapping 62
new entry point: 0x93c8c40
new _start point: 0x9100cc4
global lookup: 0x7000000
local lookup: 0x0
secondary local lookup: 0x3e
mapping offset: 0x3c8c95
length of new contents: 0x4
.text section too small to hold phdrs (or 32-bit binary); using other heuristics to relocate phdrs
WARNING   elfmanip.relocate_phdrs :: All methods failed
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment
[u@x220 foo]$ file jad-r
jad-r: ERROR: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, for GNU/Linux 2.0.0 error reading (Invalid argument)
[u@x220 foo]$ ./jad-r 
Segmentation fault (core dumped)

What am I missing?

EDIT: re-reading your post, I noticed that it is the rewritten binary I need to run with LD_LINK_NOW=1, not multiverse. Must have been an early morning :)

However, running the rewritten jad with LD_LINK_NOW=1 results in a segfault all the same.

u@x220 ~/D/g/s/g/m/m/foo> /bin/bash
[u@x220 foo]$ LD_LINK_NOW=1 ./jad-r 
Segmentation fault (core dumped)

Info from readelf:

[u@x220 foo]$ readelf -a jad-r
ELF Header:
  Magic:   7f 45 4c 46 01 01 01 00 00 00 00 00 00 00 00 00 
  Class:                             ELF32
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Intel 80386
  Version:                           0x1
  Entry point address:               0x93c8c40
  Start of program headers:          52 (bytes into file)
  Start of section headers:          960776 (bytes into file)
  Flags:                             0x0
  Size of this header:               52 (bytes)
  Size of program headers:           32 (bytes)
  Number of program headers:         3
  Size of section headers:           40 (bytes)
  Number of section headers:         24
  Section header string table index: 19
readelf: Error: Reading 960 bytes extends past end of file for section headers
readelf: Error: Section headers are not available!

Program Headers:
  Type           Offset   VirtAddr   PhysAddr   FileSiz MemSiz  Flg Align
  LOAD           0x000000 0x08048000 0x08048000 0xd1be0 0xd1be0 R E 0x1000
  LOAD           0x0d1be0 0x0811abe0 0x0811abe0 0x14c9c 0x1636c RW  0x1000
  NOTE           0x000094 0x08048094 0x08048094 0x00020 0x00020 R   0x4

There is no dynamic section in this file.

Displaying notes found at file offset 0x00000094 with length 0x00000020:
  Owner                 Data size   Description
  GNU                  0x00000010   NT_GNU_ABI_TAG (ABI version tag)

The latest rewritten binary (jad-r) is uploaded to https://ufile.io/6nrwa

utds3lab commented 6 years ago

I apologize for the delay, I have been travelling.

There is a problem with the elfmanip library. As you can tell from the output

ERROR   elfmanip.add_segment :: Cannot add another section. Not enough room in the program headers to add another segment

elfmanip is having trouble inserting the segments that Multiverse needs for the binary to work. Check out this comment about why some binaries may not be successfully rewritten. This specific binary lacks many of the components that elfmanip uses for adding segments, unfortunately. Hopefully we can continue to improve elfmanip's capabilities in the future.

mewmew commented 6 years ago

I apologize for the delay, I have been travelling.

No worries, and thanks for tracking this issue.

wliuxingxiangyu commented 4 years ago

@utds3lab hello, I follow the readme file, but
$ LD_BIND_NOW=1 ./simplest64-r Segmentation fault (core dumped)

could you give me some advice, thx !