search

google / sanitizers

AddressSanitizer, ThreadSanitizer, MemorySanitizer
Other
11.36k stars 1.02k forks source link

Questions on linking with -fsanitize=address #1086

Open dilyanpalauzov opened 5 years ago

dilyanpalauzov commented 5 years ago

My understanding is, that -fsanitize=address during compilation instrumentalizes the code and -fsanitize=address during linking loads the ASAN-runtime (libclang_rt.asan-x86_64).

However, when linking shared libraries (DSO) under Linux, additional -shared-libsan is necessary.

chefmax commented 5 years ago

My understanding is, that -fsanitize=address during compilation instrumentalizes the code and -fsanitize=address during linking loads the ASAN-runtime (libclang_rt.asan-x86_64).

However, when linking shared libraries (DSO) under Linux, additional -shared-libsan is necessary.

Not exactly. You can link DSO agains ASan without -shared-libasan, but you need to make sure that your EXE binary will be linked against ASan runtime as well.

  • What does -fsanitize=address do during linking, when no -shared-libsan is provided?

All _asan* symbols are left undefined in instrumented DSO.

  • What does -fsanitize=address do during linking, when -shared-libsan is provided during linking?

The libclang_rt.asan-x86_64.so record is inserted into DT_NEEDED entry of instrumented DSO by linker. All _asan* symbols are left undefined in instrumented DSO.

  • What does -shared-libsan do during linking, when no -fsanitize=address is provided?

Nothing happens, the linkage proceeds as if -shared-libsan was not provided.

  • For a binary linked with -fsanitize=address, that loads DSOs compiled with -fsanitize=address, do the DSOs have to be linked with -fsanitize=address?

Yes, if DSO was compiled with -fsanitize=address, it should be also linked with -fsanitize=address.

  • For a binary linked with -fsanitize=address, that loads DSOs compiled with -fsanitize=address, do the DSOs have to be linked with -shared-libsan?

Same as above. -shared-libasan is not required.

  • For a binary linked without -fsanitize=address, that is started with LD_PRELOAD, and DSOs compiled with -fsanitize=address, do the DSOs have to be linked with -fsanitize=address?

Yes.

  • What happens, if a binary linked with -fsanitize=address, loads a DSO linked with -fsanitize-address -shared-libsan? Is the ASAN runtime loaded then twice?

Error will occur during libclang_rt.asan-x86_64.so load time because it will conflict with ASan runtime statically linked into binary.

  • What happens, if a binary linked with -fsanitize=address, loads ASAN once more time with LD_PRELOAD?

Nothing happens, the binary will be executed as if LD_PRELOAD is not provided.

  • If the same ASAN runtime is loaded twice, does it emit Your application is linked against incompatible ASan runtimes.?

Yes.

  • For a binary linked without -fsanitize=address, that loads DSOs compiled with -fsanitize=address, do the DSOs have to be linked with -shared-libsan?

If you use LD_PRELOAD hack, then you need to use -shared-libasan. Otherwise (w/o LD_PRELOAD), this won't work regardless of -shared-libasan present or not.

chefmax commented 5 years ago

TL;DR: here is the short summary:

  1. If you want to build an EXE with ASan you just need to compile/link it with -fsanitize=address and everything should work.

  2. If you want to build a DSO with ASan and you can relink EXE that depends on that DSO you need to:

    • compile/link DSO with -fsanitize=address
    • relink your EXE with new DSO and -fsanitize=address in link flags

  3. If you want to build a DSO with ASan and you cannot relink EXE that depends on that DSO you need to:

    • compile/link DSO with -fsanitize=address (and -shared-libasan in case of Clang, GCC doesn't need it)
    • when running your not-sanitized EXE with sanitized DSO, use LD_PRELOAD hack to preload ASa runtime

These cases should work. And IMHO other combinations are just different ways to shoot yourself in the foot (they may/may not work depending on some side factors).

@kcc @eugenis @yugr please correct me if I'm wrong/missing something.

dilyanpalauzov commented 5 years ago

What is the difference when linking DSO, if -fsanitize=address is provided or not provided? (Asuming -shared/static-libsan was omitted in both cases.)

What means "instrumented DSO"? Is this instrumented calls to malloc/realloc/free, or is this something more?

If a binary is linked without -fsanitize=address, it does not statically link the ASAN runtime. If a shared library is linked with -fsanitize=address -shared-libsan, it loads dynamically the asan runtime. Why do I have to start the binary then with LD_PRELOAD in order to load the dynamic asan runtime, if the latter is anyway loaded, being a DT_NEEDED of the DSO?

How is linking a binary with -fsanitize=address different from starting the binary under LD_PRELOAD? Does the former link the runtime statically, and the latter dynamically? Will linking of binary with -fsanitize=address -shared-libsan link the asan runtime in the binary (no DSO) dynamically?

Do I understand correctly, that for a process the asan runtime must be loaded only once:

chefmax commented 5 years ago

What is the difference when linking DSO, if -fsanitize=address is provided or not provided? (Asuming -shared/static-libsan was omitted in both cases.)

No difference in case of Clang. In case of GCC DT_NEEDED would or would not contain corresponding shlib entry.

What means "instrumented DSO"? Is this instrumented calls to malloc/realloc/free, or is this something more?

This is a DSO compiled and linked with ASan.

If a binary is linked without -fsanitize=address, it does not statically link the ASAN runtime. If a shared library is linked with -fsanitize=address -shared-libsan, it loads dynamically the asan runtime. Why do I have to start the binary then with LD_PRELOAD in order to load the dynamic asan runtime, if the latter is anyway loaded, being a DT_NEEDED of the DSO?

Because this would be the only way to guarantee that ASan interceptors work (ASan's malloc/free/... should be first in ELF lookup search sequence).

How is linking a binary with -fsanitize=address different from starting the binary under LD_PRELOAD? Does the former link the runtime statically, and the latter dynamically? Will linking of binary with -fsanitize=address -shared-libsan link the asan runtime in the binary (no DSO) dynamically?

Do I understand correctly, that for a process the asan runtime must be loaded only once:

  • Either once statically (with -fsanitize=address during linking the binary) and then it shall not appear as DT_NEEDED and no library can be linked with -static-asan or -shared-asan and no LD_PRELOAD, or
  • the runtime can be loaded dynamically, by mentioning it at one or more places as DT_NEEDED or with LD_PRELOAD or mentining it both in DT_NEEDED and LD_PRELOAD?

Yes.

dilyanpalauzov commented 5 years ago

Speaking about clang, when a binary, that is not instrumented, loads an instrumented DSO (compiled with -fsanitize=address), the process must be started with LD_PRELOAD=asan-runtime, in order to ensure, that ASan interceptors work.

But in this case it does not matter, if the DSO has the ASan runtime in its DT_NEEDED, and therefore linking the DSO with -fsanitize=address -shared-libsan does not have added value.

Linking the DSO with -fsanitize=address -shared-libsan also does not work, if the DSO is loaded by an instumeted binary.

When does linking under clang a DSO with -fsanitize=address -shared-libsan have added value?

yugr commented 5 years ago

Because this would be the only way to guarantee that ASan interceptors work (ASan's malloc/free/... should be first in ELF lookup search sequence).

To expand on this, I believe allocator interception is required for instrumented code to work correctly.

dilyanpalauzov commented 5 years ago

How does linking DSOs with -fsanitize=address -static-libsan work?

Does this export any symbols (visibility=default)?

Is all the instumented code inlined, so that no additional symbols are involved and it does not interefere with possibly instrumented executable or other DSOs?

eugenis commented 5 years ago

With -static-libsan, the runtime library is linked into the executable and the interface is re-exported. Libraries don't declare any dependency and simply expect the symbols to be there at runtime. Technically, all such libraries have undefined, unresolved symbols. This is incompatible with -Wl,-z,defs. It is also easy to break by building the executable with a version script (with wildcards).

On Tue, May 21, 2019 at 9:12 AM Дилян Палаузов notifications@github.com wrote:

How does linking DSOs with -fsanitize=address -static-libsan work?

Does this export any symbols (visibility=default)?

Is all the instumented code inlined, so that no additional symbols are involved and it does not interefere with possibly instrumented executable or other DSOs?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/google/sanitizers/issues/1086?email_source=notifications&email_token=AADG4SSMMJ5DORJUXNHGC3TPWQNPRA5CNFSM4HNDOWPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODV4NFOY#issuecomment-494457531, or mute the thread https://github.com/notifications/unsubscribe-auth/AADG4SUMVBKWRTJZ63UOTX3PWQNPRANCNFSM4HNDOWPA .

dilyanpalauzov commented 5 years ago

I do not get the last comment. The executables (with main() ) get the run-tume library integrated, when they are linked with -fsanitize=address.

About DSOs (shared libraries), re-exporting the symbols might not be good. It does not allow using two DSOs that both use static linking. Also run time instrumentation needs loading special code at the very beginning, how is this achieved or circumvented, with DSOs?

yugr commented 5 years ago

I do not get the last comment. The executables (with main() ) get the run-tume library integrated, when they are linked with -fsanitize=address.

Sanitized executable exports various libasan's symbols for sanitized DSOs. Version scripts will not be aware of exported symbols and will normally prevent them from exporting (thus causing runtime symbol lookup fails when sanitized shlibs try to call them).

About DSOs (shared libraries), re-exporting the symbols might not be good.

Note that reexporting happens only in main executable. Sanitized shlibs do not link against libasan (and thus do not reexport any of it's symbols).

dilyanpalauzov commented 5 years ago

I am asking about the impact on linking DSOs with -fsanitize=address -static-libsan with clang, please elaborate on this.

Linking executables with -fsanitize=address, implies -static-libsan.

Are -static-libsan and -shared-libsan ignored, when linking executables?

dilyanpalauzov commented 5 years ago

Can we make progress on this?

I filled also https://bugs.llvm.org/show_bug.cgi?id=42177 towards LLVM/Clang on the same matter.

kcc commented 5 years ago

With clang, -static-libsan is the default. -shared-libsan forces the linker to use the shared asan run-time. 

% clang++ -fsanitize=address ~/misc-c/use-after-free.cc -shared-libsan  && ldd a.out  2>&1 | grep libclang
        libclang_rt.asan-x86_64.so => not found
% clang++ -fsanitize=address ~/misc-c/use-after-free.cc   && ldd a.out  2>&1 | grep libclang
% clang++ -fsanitize=address ~/misc-c/use-after-free.cc -static-libsan  && ldd a.out  2>&1 | grep libclang
%
dilyanpalauzov commented 5 years ago

https://github.com/google/sanitizers/issues/1086#issuecomment-492693405 says:

For a binary linked without -fsanitize=address, that loads DSOs compiled with -fsanitize=address, do the DSOs have to be linked with -shared-libsan?

If you use LD_PRELOAD hack, then you need to use -shared-libasan. Otherwise (w/o LD_PRELOAD), this won't work regardless of -shared-libasan present or not.

I conclude, that linking DSO with -fsanitize=address does not work, and having implicit -static-libsan does not change anything. Moreover, as far as I remember, linking DSOs with -fsanitize=address resulted undefined symbols at compile time. It does not make sense if during linking DSOs a -static-libsan is implicit, but it does not provide the necessary instumented funtcions.

https://github.com/google/sanitizers/issues/1086#issuecomment-492952798 says:

If you want to build a DSO with ASan and you cannot relink EXE that depends on that DSO you need to:

  • compile/link DSO with -fsanitize=address (and -shared-libasan in case of Clang, GCC doesn't need it)
  • when running your not-sanitized EXE with sanitized DSO, use LD_PRELOAD hack to preload ASa runtime

So why do I have to pass -shared-libsan when linking DSOs with -fsanitize=address, when the default is static-libsan?

https://github.com/google/sanitizers/issues/1086#issuecomment-493211565 states:

What is the difference when linking DSO, if -fsanitize=address is provided or not provided? (Asuming -shared/static-libsan was omitted in both cases.) No difference in case of Clang. In case of GCC DT_NEEDED would or would not contain corresponding shlib entry.

Here is stated, that having -fsanitize=address and having no -fsanitize=address is the same, but the last comment says that having -fsanitize=address implies -static-libsan.

The last example does not state, whether the binaries produced by -fsanitize=address -static-libsan and only -fsanitize=address differ. Neither is it for a DSO. And if the binaries differ, how (e.g. different set of exported symbols).

Please concentrate on the discussion. Having big time gaps is counter productive and so far nobody answered, what does -static-libsan do when linking DSOs with -fsanitize=address.

kcc commented 5 years ago

linking DSO with -fsanitize=address does not work

-fsanitize=address does not affect linking DSOs at all.

linking DSOs with -fsanitize=address resulted undefined symbols at compile time.

You mean, at link time? Yes, it may happen if you are using -Wl,-z,defs or something similar. Remove that. See https://clang.llvm.org/docs/AddressSanitizer.html#id3

So why do I have to pass -shared-libsan

You don't have to.

RightEasyLeftHard commented 1 week ago

Yes, it may happen if you are using -Wl,-z,defs or something similar. Remove that.

I do not think this is true. The undefined symbol happens even without -Wl,-z,defs what are other similar flags?