lunixbochs / usercorn

dynamic binary analysis via platform emulation
MIT License
878 stars 100 forks source link

Set/Unset Carry flag after syscalls for BSD based systems #212

Open MMMMasterM opened 7 years ago

MMMMasterM commented 7 years ago

BSD based systems change the carry flag to be set iff the result of a syscall is an error. This is often used by following a "syscall" instruction with jnb or jge (e.g. darwin's dyld does this).

If we add a second return value "error" to the syscalls handlers, the CF (un-)set step could be done in a single place (right after the syscall handler was called)

(TODO: check if this behaviour is also intended for syscalls using "int 80" instruction instead of "syscall" instruction)

lunixbochs commented 7 years ago

If it's used in x86_64 syscall, it might be part of the ABI. Worth looking up.

I like the idea of adding an error return because it also solves #96 - basically we need an errno translation layer so I can return EINVAL from a posix syscall handler and have it be translated to whatever the x86/darwin EINVAL enum actually is

and doing that in-band with the existing return values is more confusing.