google / sanitizers

AddressSanitizer, ThreadSanitizer, MemorySanitizer
Other
11.52k stars 1.04k forks source link

[Windows] Revisit how the RTL handles multi-module apps #332

Open ramosian-glider opened 9 years ago

ramosian-glider commented 9 years ago

Originally reported on Google Code with ID 332

We should probably revisit how the RTL handles multi-module apps.

Currently, the main module (.EXE) statically links in the RTL and all the other modules
(.DLLs, basically) statically link with a simple thunk .LIB that delegates all the
ASan interface calls to the RTL in the main module.
[See issue 209 for more background]

This works mostly well in practice, but makes deployment slightly more complex.
Namely, one has to use clang as a linker for DLLs or pass the path to the thunk .LIB
to the link.exe invocations.
COFF .OBJ files have a directive that allows us to specify implicit .LIB dependencies,
but unfortunately the same .OBJ file might be used in an .EXE and a .DLL, thus we can't
use the directive with the current structure of the RTL.

Reid has suggested me to restructure the RTL like this:
- put the RTL into a .DLL
- all the modules (.EXE, .DLL, etc) should statically link with a thunk .LIB that forwards
all the interface calls to the RTL DLL.
- put a directive to implicitly link the thunk .LIB into all the address-sanitized
.OBJ files

This sounds mostly good, but I wonder how would it affect usability?
Basically, which process should put the ASan RTL DLL next to the .EXE?
Is there a directive for that?

Reported by timurrrr@google.com on 2014-08-08 13:58:38

ramosian-glider commented 9 years ago
I think this is a fantastic idea!

The location at which DLLs are looked up on Windows is documented here: <http://msdn.microsoft.com/en-us/library/windows/desktop/ms682586%28v=vs.85%29.aspx>.
 The only way that I'm aware of that you could alter that search order by embedding
something in the binaries is through embedding a manifest as a resource, but that gets
a bit tricky if the application already has a manifest.  There is a tool called mt.exe
<http://msdn.microsoft.com/en-us/library/aa375649%28v=vs.85%29.aspx> which is apparently
capable of extracting a manifest from one binary, merge another manifest inside it,
and put it back into the binary (see the -updateresource and -manifest arguments).

Honestly I don't think this trouble is worth it, because it will only work for the
cases where you invoke the linker through the compiler (which some code bases such
as Mozilla don't) and Reid's suggestion basically means that you can get a working
asan build no matter what your binaries setup looks like by adding -fsanitize=address
and copying one DLL alongside the app, and that is a *huge* improvement over the existing
setup.  If that is the best we can achieve, I would be happy!

Reported by ehsan.akhgari on 2014-08-08 14:35:40

ramosian-glider commented 9 years ago
Do you mean that the user has to do the copying manually?

Reported by timurrrr@google.com on 2014-08-08 14:48:21

ramosian-glider commented 9 years ago
Well hopefully by modifying their build system, but yes.  Typically it's very easy to
copy additional resources next to the binary in various build systems.

Reported by ehsan.akhgari on 2014-08-08 14:50:04

ramosian-glider commented 9 years ago
I assume if a user wants to compile something simple (like "hello, world") as a simple
clang-cl command (clang-cl -fsanitize=address hello.c && hello.exe), clang-cl will
copy the RTL?

Reported by timurrrr@google.com on 2014-08-08 15:16:19

ramosian-glider commented 9 years ago
Sure, we can definitely try copying the DLL from the driver driver if we choose to invoke
link.exe.  That sounds fine to me.

Reported by ehsan.akhgari on 2014-08-08 15:29:38

ramosian-glider commented 9 years ago
Reid, any opinions?

Reported by timurrrr@google.com on 2014-08-08 16:58:57

ramosian-glider commented 9 years ago

Reported by ramosian.glider on 2015-07-30 09:05:33

ramosian-glider commented 9 years ago
Adding Project:AddressSanitizer as part of GitHub migration.

Reported by ramosian.glider on 2015-07-30 09:06:57

rnk commented 8 years ago

I would still really like to do this, it would drastically simplify deployment if users just have one .lib file to link against instead of one for exes and one for dlls.

This would make the ASan RTL into a DLL. Every DLL comes with a .lib import library, and that import library can actually contain real code. We should be able to jam the interceptors into the import library so that users only have one linker flag to think about.