search

williballenthin / python-idb

Pure Python parser and analyzer for IDA Pro database files (.idb).
Apache License 2.0
457 stars 73 forks source link

Reading segment descriptor #13

Closed ghost closed 7 years ago

ghost commented 7 years ago

Hi!

When I try to read segment descriptor using api.idaapi.getsegs, I got partially invalid result.

Name          Start    End      R W X D L Align  Base Type   Class AD es       ss       ds       fs       gs      
----          -----    ---      - - - - - -----  ---- ----   ----- -- --       --       --       --       --      
.init         080496AC 080496CF R . X . L dword  0001 public CODE  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.plt          080496D0 08049DE0 R . X . L para   0002 public CODE  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.plt.got      08049DE0 08049DE8 R . X . L qword  0003 public CODE  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.text         08049DF0 0805B634 R . X . L para   0004 public CODE  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.fini         0805B634 0805B648 R . X . L dword  0005 public CODE  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.rodata       0805B660 08060C14 R . . . L 32byte 0006 public CONST 32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.eh_frame_hdr 08060C14 08061430 R . . . L dword  0007 public CONST 32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.eh_frame     08061430 080664D8 R . . . L dword  0008 public CONST 32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.init_array   08067F00 08067F04 R W . . L dword  0009 public DATA  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.fini_array   08067F04 08067F08 R W . . L dword  000A public DATA  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.jcr          08067F08 08067F0C R W . . L dword  000B public DATA  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.got          08067FFC 08068000 R W . . L dword  000C public DATA  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.got.plt      08068000 080681CC R W . . L dword  000D public DATA  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.data         080681E0 08068344 R W . . L 32byte 000E public DATA  32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
.bss          08068380 08068FB8 R W . . L 64byte 000F public BSS   32 FFFFFFFF FFFFFFFF 000E     FFFFFFFF FFFFFFFF
extern        08068FB8 08069188 ? ? ? . L para   0010 public       32 FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF FFFFFFFF

With this segments, I printed out segment information using folowing python code.

def get_section_list(db):
    api = idb.IDAPython(db)
    res = []
    for s in api.idautils.Segments():
        seg = api.idaapi.getseg(s)
        print '%s' % api.idc.SegName(s)
        print ' - start address: 0x%x' % seg.startEA
        print ' - sclass: 0x%x' % seg.sclass
        print ' - orgbase: 0x%x' % seg.orgbase
        print ' - align: 0x%x' % seg.align
        print ' - comb: 0x%x' % seg.comb
        print ' - perm: 0x%x' % seg.perm
        print ' - bitness(should perm): 0x%x' % seg.bitness
        print ' - flag(should bitness): 0x%x' % seg.flags
        print ' - sel(should flag?): 0x%x' % seg.sel
        print ' - defsr(should sel): 0x%x' % seg.defsr
        print ' - type(should defsr?): 0x%x' % seg.type
        print ' - color(should type?): 0x%x' % seg.color
        print ' - ???(should color?): 0x%x' % seg.color

And the result is following 

.init
 - start address: 0x80496ac
 - sclass: 0x2
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x5
 - flag(should bitness): 0x1
 - sel(should flag?): 0x2
 - defsr(should sel): 0x1
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.plt
 - start address: 0x80496d0
 - sclass: 0x2
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x3
 - perm: 0x2
 - bitness(should perm): 0x5
 - flag(should bitness): 0x1
 - sel(should flag?): 0x2
 - defsr(should sel): 0x2
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.plt.got
 - start address: 0x8049de0
 - sclass: 0x2
 - orgbase: 0x0
 - align: 0x10
 - comb: 0xa
 - perm: 0x2
 - bitness(should perm): 0x5
 - flag(should bitness): 0x1
 - sel(should flag?): 0x2
 - defsr(should sel): 0x3
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.text
 - start address: 0x8049df0
 - sclass: 0x2
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x3
 - perm: 0x2
 - bitness(should perm): 0x5
 - flag(should bitness): 0x1
 - sel(should flag?): 0x2
 - defsr(should sel): 0x4
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.fini
 - start address: 0x805b634
 - sclass: 0x2
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x5
 - flag(should bitness): 0x1
 - sel(should flag?): 0x2
 - defsr(should sel): 0x5
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.rodata
 - start address: 0x805b660
 - sclass: 0x8
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x8
 - perm: 0x2
 - bitness(should perm): 0x4
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0x6
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.eh_frame_hdr
 - start address: 0x8060c14
 - sclass: 0x8
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x4
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0x7
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.eh_frame
 - start address: 0x8061430
 - sclass: 0x8
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x4
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0x8
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.init_array
 - start address: 0x8067f00
 - sclass: 0xc
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x6
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0x9
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.fini_array
 - start address: 0x8067f04
 - sclass: 0xc
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x6
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0xa
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.jcr
 - start address: 0x8067f08
 - sclass: 0xc
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x6
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0xb
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.got
 - start address: 0x8067ffc
 - sclass: 0xc
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x6
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0xc
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.got.plt
 - start address: 0x8068000
 - sclass: 0xc
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x5
 - perm: 0x2
 - bitness(should perm): 0x6
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0xd
 - type(should defsr?): 0x0
 - color(should type?): 0x0
.data
 - start address: 0x80681e0
 - sclass: 0xc
 - orgbase: 0x0
 - align: 0x10
 - comb: 0x8
 - perm: 0x2
 - bitness(should perm): 0x6
 - flag(should bitness): 0x1
 - sel(should flag?): 0x3
 - defsr(should sel): 0xe
 - type(should defsr?): 0x0
 - color(should type?): 0x0

I think the value of sclass is correct. And value of bitness is actually value of perm. And the vaule of type is always 0. According to the IDA SDK, the value 0 means that unknown type. And one more thing, In the result, there is no .bss section.

Actually, I'm not sure that I'm using python-idb in right way. Can you help me?

I made a following link again. This is a IDB of /bin/ls on Ubuntu OS 32bit.

ls.idb

Thank you for your help!

williballenthin commented 7 years ago

hey @binoopang

You ask a lot of really good questions! Thanks for reporting this issue.

To be honest, I haven't completely recovered the format of these segment blobs. As you can see, the library is able to extract a sequence of integer values, but they may not be mapped correctly. I had reviewed the hex-rays online documentation for hints as to what these fields may be, but haven't verified all of them.

If you can correct the ordering of these fields, I'd be extremely grateful! The relevant code is here. I'll also take a look over the next few days and see what I can add.

williballenthin commented 7 years ago

Actually, I'm not sure that I'm using python-idb in right way. Can you help me?

What makes you think you are using it incorrectly? From the snippets you've sent me, I think you've found a way to use the library well.

I've tried to set up a python-idb interface so it emulates the IDA Pro one. This means it should be possible to run scripts both in IDA Pro and under python-idb. Take a look at scripts/run_ida_script.py. This might allow you to run the same script in different environments and compare the results.

While the IDA Pro scripting interface can be inconsistent and confusing, IDAPython users should be able to use python-idb without learning many new concepts.

williballenthin commented 7 years ago

here's a script that runs in IDA Pro and python-idb:

#!/usr/bin/env python

import idc
import idaapi
import idautils

def print_section_list():
    for s in idautils.Segments():
        seg = idaapi.getseg(s)
        print('%s' % idc.SegName(s))
        print(' - start address: 0x%x' % seg.startEA)
        print(' - sclass: 0x%x' % seg.sclass)
        print(' - orgbase: 0x%x' % seg.orgbase)
        print(' - align: 0x%x' % seg.align)
        print(' - comb: 0x%x' % seg.comb)
        print(' - perm: 0x%x' % seg.perm)
        print(' - bitness: 0x%x' % seg.bitness)
        print(' - flag: 0x%x' % seg.flags)
        print(' - sel: 0x%x' % seg.sel)
        print(' - defsr: 0x%x' % seg.defsr)
        print(' - type: 0x%x' % seg.type)
        print(' - color: 0x%x' % seg.color)

print_section_list()
williballenthin commented 7 years ago

and here are the results, side by side:

IDA                                 python-idb
===                                 ==========

.init                   .init
 - start address: 0x80496ac      - start address: 0x80496ac
 - sclass: 0x2               - sclass: 0x2
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x5                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x5
 - flag: 0x10                - flag: 0x1
 - sel: 0x1              - sel: 0x2
 - defsr: 0x156653e          - defsr: 0x1
 - type: 0x2                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.plt                    .plt
 - start address: 0x80496d0      - start address: 0x80496d0
 - sclass: 0x2               - sclass: 0x2
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x3                - align: 0x10
 - comb: 0x2                 - comb: 0x3
 - perm: 0x5                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x5
 - flag: 0x10                - flag: 0x1
 - sel: 0x2              - sel: 0x2
 - defsr: 0x15665ae          - defsr: 0x2
 - type: 0x2                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.plt.got                .plt.got
 - start address: 0x8049de0      - start address: 0x8049de0
 - sclass: 0x2               - sclass: 0x2
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0xa                - align: 0x10
 - comb: 0x2                 - comb: 0xa
 - perm: 0x5                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x5
 - flag: 0x10                - flag: 0x1
 - sel: 0x3              - sel: 0x2
 - defsr: 0x1566656          - defsr: 0x3
 - type: 0x2                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.text                   .text
 - start address: 0x8049df0      - start address: 0x8049df0
 - sclass: 0x2               - sclass: 0x2
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x3                - align: 0x10
 - comb: 0x2                 - comb: 0x3
 - perm: 0x5                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x5
 - flag: 0x10                - flag: 0x1
 - sel: 0x4              - sel: 0x2
 - defsr: 0x15666c6          - defsr: 0x4
 - type: 0x2                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.fini                   .fini
 - start address: 0x805b634      - start address: 0x805b634
 - sclass: 0x2               - sclass: 0x2
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x5                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x5
 - flag: 0x10                - flag: 0x1
 - sel: 0x5              - sel: 0x2
 - defsr: 0x156679e          - defsr: 0x5
 - type: 0x2                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.rodata                 .rodata
 - start address: 0x805b660      - start address: 0x805b660
 - sclass: 0x8               - sclass: 0x8
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x8                - align: 0x10
 - comb: 0x2                 - comb: 0x8
 - perm: 0x4                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x4
 - flag: 0x10                - flag: 0x1
 - sel: 0x6              - sel: 0x3
 - defsr: 0x156680e          - defsr: 0x6
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.eh_frame_hdr               .eh_frame_hdr
 - start address: 0x8060c14      - start address: 0x8060c14
 - sclass: 0x8               - sclass: 0x8
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x4                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x4
 - flag: 0x10                - flag: 0x1
 - sel: 0x7              - sel: 0x3
 - defsr: 0x156687e          - defsr: 0x7
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.eh_frame               .eh_frame
 - start address: 0x8061430      - start address: 0x8061430
 - sclass: 0x8               - sclass: 0x8
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x4                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x4
 - flag: 0x10                - flag: 0x1
 - sel: 0x8              - sel: 0x3
 - defsr: 0x15668ee          - defsr: 0x8
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.init_array             .init_array
 - start address: 0x8067f00      - start address: 0x8067f00
 - sclass: 0xc               - sclass: 0xc
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x6                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x6
 - flag: 0x10                - flag: 0x1
 - sel: 0x9              - sel: 0x3
 - defsr: 0x1566a26          - defsr: 0x9
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.fini_array             .fini_array
 - start address: 0x8067f04      - start address: 0x8067f04
 - sclass: 0xc               - sclass: 0xc
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x6                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x6
 - flag: 0x10                - flag: 0x1
 - sel: 0xa              - sel: 0x3
 - defsr: 0x1566a96          - defsr: 0xa
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.jcr                    .jcr
 - start address: 0x8067f08      - start address: 0x8067f08
 - sclass: 0xc               - sclass: 0xc
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x6                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x6
 - flag: 0x10                - flag: 0x1
 - sel: 0xb              - sel: 0x3
 - defsr: 0x1566b06          - defsr: 0xb
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.got                    .got
 - start address: 0x8067ffc      - start address: 0x8067ffc
 - sclass: 0xc               - sclass: 0xc
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x6                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x6
 - flag: 0x10                - flag: 0x1
 - sel: 0xc              - sel: 0x3
 - defsr: 0x1566b76          - defsr: 0xc
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.got.plt                .got.plt
 - start address: 0x8068000      - start address: 0x8068000
 - sclass: 0xc               - sclass: 0xc
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x5                - align: 0x10
 - comb: 0x2                 - comb: 0x5
 - perm: 0x6                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x6
 - flag: 0x10                - flag: 0x1
 - sel: 0xd              - sel: 0x3
 - defsr: 0x1566be6          - defsr: 0xd
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.data                   .data
 - start address: 0x80681e0      - start address: 0x80681e0
 - sclass: 0xc               - sclass: 0xc
 - orgbase: 0x0              - orgbase: 0x0
 - align: 0x8                - align: 0x10
 - comb: 0x2                 - comb: 0x8
 - perm: 0x6                 - perm: 0x2
 - bitness: 0x1              - bitness: 0x6
 - flag: 0x10                - flag: 0x1
 - sel: 0xe              - sel: 0x3
 - defsr: 0x1566c56          - defsr: 0xe
 - type: 0x3                 - type: 0x0
 - color: 0xffffffff             - color: 0x0
.bss                    
 - start address: 0x8068380     
 - sclass: 0x13             
 - orgbase: 0x0             
 - align: 0x9               
 - comb: 0x2                
 - perm: 0x6                
 - bitness: 0x1             
 - flag: 0x10               
 - sel: 0xf             
 - defsr: 0x1566cc6         
 - type: 0x9                
 - color: 0xffffffff            
extern                  
 - start address: 0x8068fb8     
 - sclass: 0x0              
 - orgbase: 0x0             
 - align: 0x3               
 - comb: 0x2                
 - perm: 0x0                
 - bitness: 0x1             
 - flag: 0x10               
 - sel: 0x10                
 - defsr: 0x1566d36         
 - type: 0x1                
 - color: 0xffffffff
williballenthin commented 7 years ago

python-idb is missing segments:

williballenthin commented 7 years ago

fixed segment parsing in c633faa3f2e1f574b791cda6d7346e888b8e7d96.

this should address the issue of parsing segment fields above. it doesn't address where the extern data is...

ghost commented 7 years ago

Hi @williballenthin ! Thank you for your kind response. I'm using python-idb for my personal project. And it is really useful! Thank you again!