DRASan fails to identify a thread

[GoogleCodeExporter]

This manifests in reports like
  READ of size 1 at 0x1234 thread T16777215
    [stack]
ans also requires a hack in the ASan RTL to print the stack at all
(see http://llvm.org/viewvc/llvm-project?view=revision&revision=195956)

After some debugging, I think the GetCurrentThread() call returns 0 because it 
fails to read the proper value from TLS.
Presumably, our "push, push, call __asan_report_blah" DR instrumentation is 
TLS-unfriendly.

Original issue reported on code.google.com by timurrrr@google.com on 29 Nov 2013 at 2:58

[GoogleCodeExporter]

Can you suggest a correct way to call the app's exported function from the 
instrumented code?

Current code looks like this:
32-bit:
  PRE(i, push_imm(drcontext, OPND_CREATE_INT32(instr_get_app_pc(i))));
  PRE(i, jmp(drcontext, opnd_create_pc((byte*)*on_error)));

64-bit:
  // Align the stack by 16 bytes.
  PRE(i, and(drcontext, opnd_create_reg(DR_REG_XSP), OPND_CREATE_INT8(-16)));
  app_pc pc = instr_get_app_pc(i);
  int lo = (int)(ptr_int_t)pc;
  int hi = (int)((ptr_int_t)pc >> 32);
  PRE(i, push_imm(drcontext, OPND_CREATE_INT32(lo)));
  PRE(i, mov_st(drcontext, OPND_CREATE_MEM32(DR_REG_XSP, 4),
                OPND_CREATE_INT32(hi)));
  PRE(i, jmp_ind(drcontext, opnd_create_rel_addr((byte*)on_error, OPSZ_PTR)));

(https://code.google.com/p/address-sanitizer/source/browse/trunk/dynamorio/dr_as
an.cpp?r=1969#424)

Basically what I want is (under certain pre-conditions that are already 
implemented)
make the instrumented code call the ASan RTL linked into the running executable.

The ASan RTL then calls pthread_getspecific() and currently gets zero 
[expected: nonzero].

Original comment by timurrrr@google.com on 29 Nov 2013 at 3:05

[GoogleCodeExporter]

A small repro was added in r1970.

Original comment by timurrrr@google.com on 29 Nov 2013 at 3:06

[GoogleCodeExporter]

Key point to understand the code:
#define PRE(at, what) instrlist_meta_preinsert(bb, at, INSTR_CREATE_##what);

So these are meta ctis, and thus native transitions.  Presumably you're running 
with the private loader, and thus with -mangle_app_seg.  In that (default) 
mode, the actual segment setup is for DR, and any app segment refs must be 
transformed by DR.  Thus your native call to pthread_getspecific() is expected 
to fail.

Perhaps a (hackish?) fix is to call dr_switch_to_app_state() first.

Aside: you can use instrlist_insert_push_immed_ptrsz() for your 64-bit push 
immed.

Original comment by bruen...@google.com on 30 Nov 2013 at 8:39

[GoogleCodeExporter]

Thanks Derek!
I've tried both PRE(i, call(...(dr_switch_to_app_state))) and a client error 
handler thunk (as we've discussed over chat), the patches are attached.

With each of these patches I'm able to get the thread number in the report 
correctly (i.e. pthread_getspecific() works as expected), though now I only see 
the top frame of the stack in the report.
Any tips how to get both backtrace() and pthread_getspecific() working well?

Original comment by timurrrr@google.com on 1 Dec 2013 at 1:11

Attachments:

• clean_call_thunk.patch
• switch_to_app_state.patch

[GoogleCodeExporter]

(FTR, these patch only work on 32-bit builds indeed)

Original comment by timurrrr@google.com on 1 Dec 2013 at 1:11

[GoogleCodeExporter]

FTR, both patches also require running drrun with -sysenter_is_int80, otherwise 
I get
"Usage error: Is your client invoking raw system calls via vdso sysenter? While 
such behavior is not recommended and can create problems, it may work with the 
-sysenter_is_int80 runtime option. (core/dispatch.c, line 116)"
errors.

Original comment by timurrrr@google.com on 1 Dec 2013 at 1:15

[GoogleCodeExporter]

hm, wait - I think the ToTT code only shows one frame as well, though no idea 
what's wrong..

Original comment by timurrrr@google.com on 1 Dec 2013 at 1:19

[GoogleCodeExporter]

Probably an unrelated 32-bit specific issue.  Will try to 
call(dr_switch_to_app_state) on x64 then.

Original comment by timurrrr@google.com on 1 Dec 2013 at 1:29

[GoogleCodeExporter]

> now I only see the top frame of the stack in the report.

If you do a clean call, the stack is switched to DR's stack.  If you then run 
your callstack walker without switching it back, it's not going to see the app 
stack.  Probably the top frame is coming from some param you're passing and not 
the actual stack.

> "Usage error: Is your client invoking raw system calls...

Since sysenter does not return to user mode in a normal manner, DR needs a hook 
on the vdso.  It expects to be in control of all app code.  You're violating 
DR's invariants by sneakily going and running some code natively, which 
doubtless makes a system call that hits the vdso hook.  (Although this can also 
happen with a call to a library in tool mode due to imperfect isolation.)

Original comment by bruen...@google.com on 1 Dec 2013 at 3:38