sysenter emulation broken?

[gdbinit]

I'm trying to emulate a binary that uses sysenter for executing syscalls. A UC_HOOK_INTR doesn't trigger (I assume it's only for syscalls via int 80) so I set a SYSENTER instruction hook. There seems to be a bug inside Unicorn SYSENTER emulation because the following code infinite loops: EADER:0000D708 sub_D708 proc near ; CODE XREF: sub_D70E+4p HEADER:0000D708 ; sub_D70E+13p HEADER:0000D708 0F B6 C0 movzx eax, al HEADER:0000D70B 5A pop edx HEADER:0000D70C 0F 34 sysenter HEADER:0000D70C sub_D708 endp ; sp-analysis failed HEADER:0000D70C HEADER:0000D70E HEADER:0000D70E ; =============== S U B R O U T I N E ======================================= HEADER:0000D70E HEADER:0000D70E HEADER:0000D70E sub_D70E proc near ; CODE XREF: start+3Bp HEADER:0000D70E HEADER:0000D70E arg_0 = dword ptr 4 HEADER:0000D70E HEADER:0000D70E B0 C5 mov al, 0C5h HEADER:0000D710 8B CC mov ecx, esp HEADER:0000D712 E8 F1 FF FF FF call sub_D708 HEADER:0000D717 73 0D jnb short locret_D726

What happens is that after SYSENTER returns from the instruction hook, the EIP will be set to 0xD70E instead of 0xD717, so it's looping forever and calling sysenter forever. Looking at SYSENTER Qemu implementation we have the following code at qemu/target-i386/seg_helper.c: void helper_sysenter(CPUX86State env, int next_eip_addend) { // Unicorn: call registered SYSENTER hooks struct hook hook; HOOK_FOREACH(env->uc, hook, UC_HOOK_INSN) { if (!HOOK_BOUND_CHECK(hook, env->eip)) continue; if (hook->insn == UC_X86_INS_SYSENTER) ((uc_cb_insn_syscall_t)hook->callback)(env->uc, hook->user_data); }

env->eip += next_eip_addend;
return;

(...) }

This means that Unicorn cut the original function halfway. The problem is on env->eip which is pointing to the next instruction as observed in the above loop. I fixed this by replacing it with env->eip = env->regs[R_EDX]; Which sets the EIP to the correct address.

This fix made my SYSENTER hook and syscall emulation work as I wanted but I have no idea about side effects of it. At least everything looks as expected in general registers. SYSEXIT has no Unicorn hooks from what I can see so I am not sure if there's any other recommended way to emulate syscalls via SYSENTER.

GDT and everything else necessary for 32 bits emulation is set via Chris Eagle's sample_x86_32_gdt_and_seg_regs.c code.

Best, fG!

[lunixbochs]

My previous comment was wrong. OSDev Wiki Reference

These must be set by the application, or the C library wrapper
ECX: Ring 3 Stack pointer for SYSEXIT
EDX: Ring 3 Return address

Looking at QEMU upstream, the correct code should probably be this:

env->regs[R_ESP] = env->sysenter_esp;
env->eip = env->sysenter_eip;

I guess the syscall helper code for env->eip += next_eip_addend; was copied.

There's also the question of what happens when someone wants to actually run an OS in Unicorn with hooks enabled. Which will totally not work right now due to some of these hooks taking shortcuts or the emulator trapping out.

[gdbinit]

Yes, EDX/RDX is coming from SYSEXIT documentation at Intel Manuals.

the env->sysenter_* probably point to same RDX and ECX values (I think I verified this), and they are the correct way to do it, since it's how the original code that was cut short was doing.

Not sure what's the best decision but current code is at least incorrect and useless for hooking SYSENTER without ugly start/stop code.

[lunixbochs]

Send a pull request with the original two lines and it will be merged. I guess there's the indirection to account for 32 vs 64 bit.

On Dec 21, 2016, at 12:22 PM, fG! notifications@github.com wrote:

Yes, EDX/RDX is coming from SYSEXIT documentation at Intel Manuals.

the env->sysenter_* probably point to same RDX and ECX values (I think I verified this), and they are the correct way to do it, since it's how the original code that was cut short was doing.

Not sure what's the best decision but current code is at least incorrect and useless for hooking SYSENTER without ugly start/stop code.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[cseagle]

There may be some confusion in the provided example. My understanding wrt sysexit is that while it does do:

eip/rip <- edx/rdx esp/rsp <- ecx/rcx

The values in rcx and rdx are dictated by the kernel's sysexit handler and not necessarily dependent on the user space values of rcx/rdx at the time of the sysenter unless the sysenter handler chooses to make that happen.

I haven't looked at how qemu handles sysenter (hopefully correctly) but a unicorn unicorn user should need to configure the appropriate MSRs so that control transfers to the appropriate sysenter handler. If a unicorn user wants to hook sysenter to skip it they would need to know where to transfer control in user space to resume from the sysenter. This is not necessarily the instruction following the sysenter since sysenter does not save user space eip/rip info.

[lunixbochs]

Okay, so we need to come up with a good default behavior. I think there are multiple cases to consider. Someone could be emulating ring 3 code with no kernel. Someone could be emulating ring 3 code with a kernel. And in both of these cases they can have the SYSENTER hook installed or not.

It would be simple enough to restore the original QEMU behavior and document that if there isn't actually a kernel in the emulator, the hook will need to manually set the return value. We can additionally pass the next EIP value as an argument to the hook to make this case easier.

[cseagle]

I speculate that a lot of people that spend all their time in user space have no idea what an msr is and expecting them to set them up to make things work in a real world manner is probably asking a lot. Allowing them to call something like sysenter_returns_to(address) would allow them to hook sysenter and then allow the internal sysenter code to set eip/rip according to the user's desires so that the trap returns immediately to the specified user space address.

[gdbinit]

I think a decision must be made about what Unicorn is and what is its main goal/purpose. It's definitely a very interesting tool to emulate user space code and with that in mind I think it should establish some assumptions about how users will want to run it and simplify things such as this. It should be easy to emulate the interface between user space and kernel, as easy as possible, because those will be things Unicorn users will face when trying to emulate full binaries or shellcode pieces. Full binary emulation doesn't make it a worthy goal (for that full QEMU or alternatives are more fit) but partial binary emulation to automate unpacking and so on is definitely interesting - doesn't require full emulation but most of the time it requires syscall emulation for example.