derekbruening
commented
9 years ago From bruen...@google.com on June 12, 2012 10:36:48
pasting in some notes which are mostly me talking to myself but hopefully these are readable:
\ TODO impl notes *\ TODO 64-bit DR or 64-bit capabilities in 32-bit DR?
simplest and cleanest to use 64-bit DR for decode/encode
*\ TODO linux or windows? windows is easier
much easier to have 64-bit lib loaded in windows than to go implement our own linux loader (ld.so won't load our 64-bit lib into 32-bit app)
*\ TODO transitions between modes
on windows can use the already-set-up 64-bit code segment
kernel exposes GDT and LDT slots so should be able to create a descriptor on linux but more work
*\ TODO existing mixed-mode support
xref already-existing support for mixing 32-bit and 64-bit:
- per-thread mode for decoding: set_x86_mode()
- per-instr flag for encoding: instr_set_x86_mode()
- whole fragment must be same mode: FRAG_32_BIT (decode_fragment relies on this. it's used for trace building to add each block to trace ilist, as well as for state recreation of coarse-grain)
- no 32<->64 links
- non-module-bitwidth is fine-grained
- 2 versions of gencode for wow64 today
- debugging support: xref windbg wow64exts "!sw" to swap modes.
for this we don't need to support app w/ mixed code (that's issue #49 ).
*\ TODO x64-incompatible segment operations
push/pop of cs/ds/es/ss segments segments in general are all flat: most apps don't care. ones that do: could just bail on this xformation? if hit unsupported, leave existing frags as x64 and bail on rest? leave some as x86 and try others later that don't have segment ops in them?
*\ TODO injection
don't start from 32-bit DR in process: just have 64-bit DR and add support for 32-bit app (xref issue #49 )
just use x64 drinject and give it a 32-bit process? follow children: treat like 64-to-64, and have DR in child do delay hook on 32-bit ntdll.
or, use 32-bit drinject and solve cross-arch inject => issue #803 though follow children will be 64-to-64
*\ TODO once 64-bit DR loaded
ntdll64 already in process, so DR will initialize normally.
*\ TODO mcontext
64-bit data struct but put in 32-bit values
*\ TODO wow64 layer
don't try to solve issue #49 : leave wow64 native
*\ TODO syscalls: just skip the wow64 far call so may actually be faster!
*\ TODO Ki: need to hook 32-bit ntdll Ki
I'm assuming kernel talks to 64-bit wow64 layer, always, regardless of the current processor mode. one question is: will the wow64 layer, on a fault while in 64-bit mode imposed by DR, still go to 32-bit ntdll Ki? and if so, will DR have any situations where it can't recover or handle its own deliberate faults or something b/c it doesn't have 64-bit state?
*\ TODO gencode
generate 32-bit ibl and cxt switch. add far jmp to cxt sw to support 32-bit for incremental work or if want to bail on translating certain app instructions.
From bruen...@google.com on April 24, 2012 11:22:34
I've long wanted to work on various projects involving switching modes, whether for the app's benefit or the tool's, but so far have not had time to work on any of them.
the idea here is that, when running a 32-bit application on a 64-bit kernel under a DynamoRIO tool, we can switch to 64-bit mode and use the extra registers as scratch space to reduce spills and improve performance.
we should consider using the registers for core DR for ibl, and also make them available to the tool
we'd have to mangle instructions that are not legal in x64 mode. some just need a re-encoding (e.g., 1-byte inc/dec) while others will be more complex (pusha, BCD instrs, lds, etc.) and it may be simpler to swap back to x86 mode rather than try to emulate some of them.
also have to be careful of instrs whose default operand size changes based on mode. most problematic and common will be push/pop which will likely have to all be converted to store/load (multiple if mem arg).
need to ensure fault handling is done properly regardless of mode
xref issue #49 : simultaneous 32-bit and 64-bit app code support
xref my prior proposals about 32-to-64 for app code for supporting 32-bit legacy plugins in 64-bit apps
xref "Dynamic Register Promotion of Stack Variables" in CGO 2011 32-to-64 to optimize app by getting app stack refs into regs
Original issue: http://code.google.com/p/dynamorio/issues/detail?id=751