My Ubuntu 18 confused about 32" binaries

ehsmeng commented 6 years ago

Would you please consider this?

ehsmeng@dell7520:/mnt/torture/3pp/multiverse$ git diff x86_assembler.py
diff --git a/x86_assembler.py b/x86_assembler.py
index 3571864..63b8471 100644
--- a/x86_assembler.py
+++ b/x86_assembler.py
@@ -1,5 +1,5 @@
 import pwn
-pwn.context(os='linux',arch='i386')
+pwn.context(os='linux',arch='i386',bits=32)
 import re
 import struct

utds3lab commented 6 years ago

Does pwntools not work correctly on Ubuntu 18 without this change? What is the error you get without changing this? From looking at the source code of pwntools, it appears that setting the arch as 'i386' should imply bits=32. If this actually makes a difference, then we may need to make more changes than just the one shown here, depending on pwntool's behavior.

ehsmeng commented 6 years ago

ehsmeng@dell7520:~/t/asm1$ uname -a
Linux dell7520 4.15.0-20-generic #21-Ubuntu SMP Tue Apr 24 06:16:15 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
ehsmeng@dell7520:~/t/asm1$ file ex1
ex1: ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), statically linked, not stripped
ehsmeng@dell7520:~/t/asm1$ cat ex1.asm 
global _start
_start:
    mov eax, 1
    mov ebx, 42
    int 0x80
ehsmeng@dell7520:~/t/asm1$ head /mnt/torture/3pp/multiverse/x86_assembler.py
import pwn
pwn.context(os='linux',arch='i386')
import re
import struct

cache = {}
pat = re.compile('\$\+[-]?0x[0-9a-f]+')
pat2 = re.compile('[ ]*push [0-9]+[ ]*')
pat3 = re.compile('[ ]*mov eax, (d)?word ptr \[0x[0-9a-f]+\][ ]*')
pat4 = re.compile('[ ]*mov eax, (dword ptr )?\[(?P<register>e[a-z][a-z])( )?[+-]( )?(0x)?[0-9a-f]+\][ ]*')
ehsmeng@dell7520:~/t/asm1$ /mnt/torture/3pp/multiverse/multiverse.py ex1
Found .text
binary does not contain plt
Writing as main binary
Base address: 0x8048000
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
Disassembly 100% complete...
[ERROR] There was an error running ['/usr/bin/x86_64-linux-gnu-as', '-64', '-o', '/tmp/pwn-asm-yDMPPQ/step2', '/tmp/pwn-asm-yDMPPQ/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-yDMPPQ/step1: Assembler messages:
    /tmp/pwn-asm-yDMPPQ/step1:7: Error: operand type mismatch for `push'
    /tmp/pwn-asm-yDMPPQ/step1:11: Error: operand type mismatch for `pop'
    /tmp/pwn-asm-yDMPPQ/step1:21: Error: operand type mismatch for `pop'
    /tmp/pwn-asm-yDMPPQ/step1:26: Error: operand type mismatch for `pop'

[ERROR] An error occurred while assembling:
       1: .section .shellcode,"awx"
       2: .global _start
       3: .global __start
       4: _start:
       5: __start:
       6: .intel_syntax noprefix
       7:    push ebx
       8:    mov ebx,eax
       9:    call get_eip
      10:     get_eip:
      11:    pop eax
      12:    sub eax,8
      13:      sub ebx,134512640
      14:    jb outside
      15:    cmp ebx,108
      16:    jae outside
      17:    mov ebx,[eax+ebx*4+143]
      18:    cmp ebx, 0xffffffff
      19:    je failure
      20:    add eax,ebx
      21:    pop ebx
      22:    ret
      23:     outside:
      24:      add ebx,134512640
      25:    mov eax,ebx
      26:    pop ebx
      27:    mov DWORD PTR [esp-32],117440512
      28:      jmp [esp-32]
      29:     failure:
      30:    hlt
    Traceback (most recent call last):
      File "/usr/local/lib/python2.7/dist-packages/pwnlib/asm.py", line 665, in asm
        _run(assembler + ['-o', step2, step1])
      File "/usr/local/lib/python2.7/dist-packages/pwnlib/asm.py", line 383, in _run
        log.error(msg)
      File "/usr/local/lib/python2.7/dist-packages/pwnlib/log.py", line 417, in error
        raise PwnlibException(message % args)
    PwnlibException: There was an error running ['/usr/bin/x86_64-linux-gnu-as', '-64', '-o', '/tmp/pwn-asm-yDMPPQ/step2', '/tmp/pwn-asm-yDMPPQ/step1']:
    It had the exitcode 1.
    It had this on stdout:
    /tmp/pwn-asm-yDMPPQ/step1: Assembler messages:
    /tmp/pwn-asm-yDMPPQ/step1:7: Error: operand type mismatch for `push'
    /tmp/pwn-asm-yDMPPQ/step1:11: Error: operand type mismatch for `pop'
    /tmp/pwn-asm-yDMPPQ/step1:21: Error: operand type mismatch for `pop'
    /tmp/pwn-asm-yDMPPQ/step1:26: Error: operand type mismatch for `pop'

Traceback (most recent call last):
  File "/mnt/torture/3pp/multiverse/multiverse.py", line 317, in <module>
    rewriter.rewrite(args.filename,args.arch)
  File "/mnt/torture/3pp/multiverse/multiverse.py", line 191, in rewrite
    mapping = mapper.gen_mapping()
  File "/mnt/torture/3pp/multiverse/brute_force_mapper.py", line 59, in gen_mapping
    lookup_size = len(self.runtime.get_lookup_code(self.base,len(self.bytes),0,0x8f)) #TODO: Issue with mapping offset & size
  File "/mnt/torture/3pp/multiverse/x86_runtime.py", line 116, in get_lookup_code
    return _asm(lookup_template%(lookup_off+8,exec_code%base,size,mapping_off,exec_restore%base,self.context.global_lookup))
  File "/mnt/torture/3pp/multiverse/x86_assembler.py", line 28, in _asm
    code = pwn.asm(text)
  File "/usr/local/lib/python2.7/dist-packages/pwnlib/context/__init__.py", line 1392, in setter
    return function(*a)
  File "/usr/local/lib/python2.7/dist-packages/pwnlib/asm.py", line 710, in asm
    log.exception("An error occurred while assembling:\n%s" % lines)
  File "/usr/local/lib/python2.7/dist-packages/pwnlib/asm.py", line 665, in asm
    _run(assembler + ['-o', step2, step1])
  File "/usr/local/lib/python2.7/dist-packages/pwnlib/asm.py", line 383, in _run
    log.error(msg)
  File "/usr/local/lib/python2.7/dist-packages/pwnlib/log.py", line 417, in error
    raise PwnlibException(message % args)
pwnlib.exception.PwnlibException: There was an error running ['/usr/bin/x86_64-linux-gnu-as', '-64', '-o', '/tmp/pwn-asm-yDMPPQ/step2', '/tmp/pwn-asm-yDMPPQ/step1']:
It had the exitcode 1.
It had this on stdout:
/tmp/pwn-asm-yDMPPQ/step1: Assembler messages:
/tmp/pwn-asm-yDMPPQ/step1:7: Error: operand type mismatch for `push'
/tmp/pwn-asm-yDMPPQ/step1:11: Error: operand type mismatch for `pop'
/tmp/pwn-asm-yDMPPQ/step1:21: Error: operand type mismatch for `pop'
/tmp/pwn-asm-yDMPPQ/step1:26: Error: operand type mismatch for `pop'

with bits=32

ehsmeng@dell7520:~/t/asm1$ head /mnt/torture/3pp/multiverse/x86_assembler.py
import pwn
pwn.context(os='linux',arch='i386',bits=32)
import re
import struct

cache = {}
pat = re.compile('\$\+[-]?0x[0-9a-f]+')
pat2 = re.compile('[ ]*push [0-9]+[ ]*')
pat3 = re.compile('[ ]*mov eax, (d)?word ptr \[0x[0-9a-f]+\][ ]*')
pat4 = re.compile('[ ]*mov eax, (dword ptr )?\[(?P<register>e[a-z][a-z])( )?[+-]( )?(0x)?[0-9a-f]+\][ ]*')
ehsmeng@dell7520:~/t/asm1$ /mnt/torture/3pp/multiverse/multiverse.py ex1
Found .text
binary does not contain plt
Writing as main binary
Base address: 0x8048000
Generating mapping...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
Disassembly 100% complete...
final offset for mapping is: 0x1f5
Generating new code...
Starting disassembly...
Disassembly 0% complete...
Disassembly 10% complete...
Disassembly 20% complete...
Disassembly 30% complete...
Disassembly 40% complete...
Disassembly 50% complete...
Disassembly 60% complete...
Disassembly 70% complete...
Disassembly 80% complete...
Disassembly 90% complete...
Disassembly 100% complete...
mapping is being placed at offset: 0x1f5
last address in mapping was 0x804806c
just set global_flag value to 0x-4
63
69
last address in mapping was 0x804806c
0x804806c
code increase: 763%
lookup w/unknown mapping 63
lookup w/known mapping 63
new entry point: 0x90001a0
new _start point: 0x900016e
global lookup: 0x7000000
local lookup: 0x0
secondary local lookup: 0x8f
mapping offset: 0x1f5
length of new contents: 0x4
.text section too small to hold phdrs (or 32-bit binary); using other heuristics to relocate phdrs
ERROR   elfmanip._phdr_hack1 :: Cannot relocate the program headers.
Traceback (most recent call last):
  File "/mnt/torture/3pp/multiverse/multiverse.py", line 317, in <module>
    rewriter.rewrite(args.filename,args.arch)
  File "/mnt/torture/3pp/multiverse/multiverse.py", line 254, in rewrite
    bin_write.rewrite(fname,fname+'-r','newbytes',self.context.newbase,mapper.runtime.get_global_mapping_bytes(),self.context.global_lookup,self.context.newbase+self.context.new_entry_off,offs,size,self.context.num_new_segments,arch)
  File "/mnt/torture/3pp/multiverse/bin_write.py", line 96, in rewrite
    elf.relocate_phdrs()
  File "build/bdist.linux-x86_64/egg/elfmanip/elfmanip.py", line 184, in relocate_phdrs
  File "build/bdist.linux-x86_64/egg/elfmanip/elfmanip.py", line 216, in _phdr_hack1
elfmanip.elfmanip.BadELF: ELF has no writable section?

utds3lab commented 6 years ago

Thank you, it looks like this may actually be a more common error than I realized. What is your version of pwntools? The bug may depend on the pwntools version, and this change looks like it should handle versions that might otherwise have problems.

I notice you're still getting some errors with rewriting a binary, even after fixing the pwntools issue. I would like to try to explain this problem, because our solution looks kind of messy, but we came to this solution after a lot of experimentation.

Certain binaries make it difficult-to-impossible to construct a rewritten binary that will be accepted by the loader without moving the text section. This is avoidable---modifying the loader can address this. Since this problem has nothing to do with rewriting or instrumenting instructions, we didn't spend the engineering efforts getting this working for all edge-case binaries.

For example, the binary you are using appears to be a very small example that's directly assembled, and it probably gives us very little flexibility to insert our new segments. Therefore, the ELFManip library is encountering an unusual case it doesn't expect. From the error message, I'm guessing your test binary may not have any sort of .data section.

I tried to make this work for most typical binaries, and I implemented an approach that should work for almost any moderately-sized 64-bit binary (it is not implemented for 32-bit binaries), but binaries with very small text sections may encounter an error if there's no room to insert new segment header entries.

ehsmeng commented 6 years ago

pwntools is just a git clone from the 14th of may. I wanted to be sure I looked at the correct code when debugging.

If I produce file with gcc it works (but generated binary segfaulted but that's another issue). The test file above was a trivial hand compiled bare minimum asm file.

ehsmeng@dell7520:~/t/asm1$ cat ex1.asm 
global _start
_start:
    mov eax, 1
    mov ebx, 42
    int 0x80
ehsmeng@dell7520:~/t/asm1$ cat wot.txt 
# https://www.youtube.com/watch?v=wLXIWKUWpSs&t=621s
nasm -f elf32 ex1.asm -o ex1.o
ld -m elf_i386 ex1.o -o ex1
[ex1.zip](https://github.com/utds3lab/multiverse/files/2012439/ex1.zip)

utds3lab / multiverse

My Ubuntu 18 confused about 32" binaries #2