Init-order checking is dlopen-hostile

[GoogleCodeExporter]

Currently, when we call dlopen(), __asan_before_dynamic_init()
poisons all the dynamically initialized globals (except those in the current 
module). Some of them may be concurrently accessed by other threads in the 
program. Thus, we have data races on shadow memory and false positives:

$ cat tmp/init-order/dlopen/a.cc
#include <dlfcn.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>

int foo() {
  return 42;
}
int global = foo();

void *global_poller(void *arg) {
  fprintf(stderr, "global poller started!\n");
  while (true) {
    if (global != 42) {
      break;
    }
  }
  fprintf(stderr, "global poller exiting!\n");
  return 0;
}

int main() {
  pthread_t p;
  pthread_create(&p, 0, global_poller, 0);
  fprintf(stderr, "global poller created!\n");
  if (0 == dlopen("b.so", RTLD_NOW)) {
    fprintf(stderr, "%s\n", dlerror());
  }
  pthread_join(p, 0);
  return 0;
}

$ cat tmp/init-order/dlopen/b.cc 
#include <stdio.h>
#include <unistd.h>

int slow_init() {
  fprintf(stderr, "slow init starting!\n");
  sleep(1);
  fprintf(stderr, "slow init exiting!\n");
  return 42;
}

int slowly_init_glob = slow_init();

$ ./bin/clang++ -fsanitize=address tmp/init-order/dlopen/b.cc -fPIC -shared -o 
b.so
$ ./bin/clang++ -fsanitize=address tmp/init-order/dlopen/a.cc
$ ASAN_OPTIONS=check_initialization_order=true LD_LIBRARY_PATH=. ./a.out 
global poller created!
slow init starting!
global poller started!
=================================================================
==12341==ERROR: AddressSanitizer: initialization-order-fiasco
<....>
0x00000105f500 is located 0 bytes inside of global variable 'global' from 
'tmp/init-order/dlopen/a.cc'

Original issue reported on code.google.com by samso...@google.com on 15 Apr 2013 at 12:49

[GoogleCodeExporter]

LLVM revision r179843 should fix this testcase. It changes the default 
init-order checker mode to the one that _accepts_ access to already initialized 
globals. Note, that this change is required for, e.g. testing LLVM itself: when 
a certain global is initialized, it adds itself to a global registry, assuming 
it can be safely accessed during the following initialization. I've seen at 
least one other project that uses the same pattern.

Still, we may lose a lot of potential bugs in this way :(
My suggestion is to use "strict-init-order" flag for codebases that are known 
to not use such machinery, and fight with described dlopen() issue by some of:
1) intercept pthread_create() and/or dlopen() and stop init-order checking when 
they are called.
2) add an interface function __asan_stop_init_order_checking(). It can be 
called at the beginning of main().
3) use nifty hacks: if we've dlopened() a .so, find out where its globals are 
located, and poison only them during module initialization (ew-w-w).

Original comment by samso...@google.com on 19 Apr 2013 at 8:56

[GoogleCodeExporter]

#1.a is probably the simplest (stop once we have more than 1 thread). Although 
I can imagine a situation where a thread is started from a module initializer. 
Grrr. 

Original comment by konstant...@gmail.com on 19 Apr 2013 at 2:08

[GoogleCodeExporter]

I've implemented #1.a in LLVM r180106.

Original comment by samso...@google.com on 23 Apr 2013 at 1:59

[GoogleCodeExporter]

can we close this? 

Original comment by konstant...@gmail.com on 29 Jan 2014 at 1:24

[GoogleCodeExporter]

Original comment by samso...@google.com on 29 Jan 2014 at 2:54

• Changed state: Fixed

[GoogleCodeExporter]

FTR the problem description provided here is a bit misleading. strict 
init-order checker is *both* multithreading- and dlopen()-hostile: we can end 
up with false positives both if use dlopen() in a single-threaded program, and 
if we call spawn new threads during initialization, but don't call dlopen(). 
That is, we need to disable init-order checker both in pthread_create 
interceptor (was done in r180106) and in dlopen interceptor (this is addressed 
in r230288).

Original comment by samso...@google.com on 24 Feb 2015 at 12:41