[Windows] Revisit how the RTL handles multi-module apps

[GoogleCodeExporter]

We should probably revisit how the RTL handles multi-module apps.

Currently, the main module (.EXE) statically links in the RTL and all the other 
modules (.DLLs, basically) statically link with a simple thunk .LIB that 
delegates all the ASan interface calls to the RTL in the main module.
[See issue 209 for more background]

This works mostly well in practice, but makes deployment slightly more complex.
Namely, one has to use clang as a linker for DLLs or pass the path to the thunk 
.LIB to the link.exe invocations.
COFF .OBJ files have a directive that allows us to specify implicit .LIB 
dependencies, but unfortunately the same .OBJ file might be used in an .EXE and 
a .DLL, thus we can't use the directive with the current structure of the RTL.

Reid has suggested me to restructure the RTL like this:
- put the RTL into a .DLL
- all the modules (.EXE, .DLL, etc) should statically link with a thunk .LIB 
that forwards all the interface calls to the RTL DLL.
- put a directive to implicitly link the thunk .LIB into all the 
address-sanitized .OBJ files

This sounds mostly good, but I wonder how would it affect usability?
Basically, which process should put the ASan RTL DLL next to the .EXE?
Is there a directive for that?

Original issue reported on code.google.com by timurrrr@google.com on 8 Aug 2014 at 1:58

[GoogleCodeExporter]

I think this is a fantastic idea!

The location at which DLLs are looked up on Windows is documented here: 
<http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.a
spx>.  The only way that I'm aware of that you could alter that search order by 
embedding something in the binaries is through embedding a manifest as a 
resource, but that gets a bit tricky if the application already has a manifest. 
 There is a tool called mt.exe 
<http://msdn.microsoft.com/en-us/library/aa375649%28v=vs.85%29.aspx> which is 
apparently capable of extracting a manifest from one binary, merge another 
manifest inside it, and put it back into the binary (see the -updateresource 
and -manifest arguments).

Honestly I don't think this trouble is worth it, because it will only work for 
the cases where you invoke the linker through the compiler (which some code 
bases such as Mozilla don't) and Reid's suggestion basically means that you can 
get a working asan build no matter what your binaries setup looks like by 
adding -fsanitize=address and copying one DLL alongside the app, and that is a 
*huge* improvement over the existing setup.  If that is the best we can 
achieve, I would be happy!

Original comment by ehsan.ak...@gmail.com on 8 Aug 2014 at 2:35

[GoogleCodeExporter]

Do you mean that the user has to do the copying manually?

Original comment by timurrrr@google.com on 8 Aug 2014 at 2:48

[GoogleCodeExporter]

Well hopefully by modifying their build system, but yes.  Typically it's very 
easy to copy additional resources next to the binary in various build systems.

Original comment by ehsan.ak...@gmail.com on 8 Aug 2014 at 2:50

[GoogleCodeExporter]

I assume if a user wants to compile something simple (like "hello, world") as a 
simple clang-cl command (clang-cl -fsanitize=address hello.c && hello.exe), 
clang-cl will copy the RTL?

Original comment by timurrrr@google.com on 8 Aug 2014 at 3:16

[GoogleCodeExporter]

Sure, we can definitely try copying the DLL from the driver driver if we choose 
to invoke link.exe.  That sounds fine to me.

Original comment by ehsan.ak...@gmail.com on 8 Aug 2014 at 3:29

[GoogleCodeExporter]

Reid, any opinions?

Original comment by timurrrr@google.com on 8 Aug 2014 at 4:58

[GoogleCodeExporter]

Original comment by ramosian.glider@gmail.com on 30 Jul 2015 at 9:05

• Added labels: ProjectAddressSanitizer

[GoogleCodeExporter]

Adding Project:AddressSanitizer as part of GitHub migration.

Original comment by ramosian.glider@gmail.com on 30 Jul 2015 at 9:06