JuliaPackaging / BinaryBuilderBase.jl

https://juliapackaging.github.io/BinaryBuilderBase.jl/stable
MIT License
11 stars 31 forks source link

Add support for building sanitizer-enabled jlls #255

Closed Keno closed 2 years ago

Keno commented 2 years ago

This adds support for automatically adding the appropriate -fsanitize= flags when the platform includes a sanitizer tag. This is particularly intended for msan, which requires that all .so's in the system are built using it, but could be useful for other sanitizers also.

There's a couple annoyances. One is that we don't currently actually ship the sanitizer runtime libraries in our rootfs. Thus, if we want to start using this, we'd have to add a BuildDependency on LLVMCompilerRT_jll and manually copy the runtime libraries into place. Not the end of the world, but certainly a wart.

The other issue is that platforms_match (which is defined in Base) of course currently ignores the sanitizer tag. In theory it is possible to add a custom compare strategy, but since we're specifying the target by triplet, we can't really add that. Easy enough to add manually here, but does make me wonder whether the custom compare strategies in Base actually fit the use case.

Keno commented 2 years ago

You mean for CI? Maybe just check that something like:

int main() {
   return *(int*)1;
}

triggers an asan/msan report?

Keno commented 2 years ago

We should probably also teach the auditor to check for the presence of the asan/msan symbols in the executable to make sure that the thing that got compiled actually had the options set.

Keno commented 2 years ago

Renamed this to sanitize and tested by building SuiteSparse and running the SparseArray tests on https://github.com/JuliaLang/julia/pull/46336, so I think this is ok for an initial cut.

giordano commented 2 years ago

What's the benefit of having this as a platform tag? Is a library built with sanitizer incompatible with another library built without it? This is probably what you meant with

This is particularly intended for msan, which requires that all .so's in the system are built using it

?

we'd have to add a BuildDependency on LLVMCompilerRT_jll and manually copy the runtime libraries into place.

Would https://github.com/JuliaPackaging/Yggdrasil/pull/1681 be a better long-term solution?

Keno commented 2 years ago

Is a library built with sanitizer incompatible with another library built without it?

Yes

Keno commented 2 years ago

Would JuliaPackaging/Yggdrasil#1681 be a better long-term solution?

Yes - I haven't looked at that PR in detail, but presumably.

Keno commented 2 years ago

Is a library built with sanitizer incompatible with another library built without it?

Yes

In particular, julia would detect that it's built with sanitizers and append the platform tag itself.

Keno commented 2 years ago

We should probably split out memory into memory and memory_origins, since the origin tracking is another 2-3 perf overhead and incompatible. If you're using rr anyway, the origin tracking doesn't really buy you anything.