Support leak detection and race condition analysis

[isaachier]

llvm/clang provide many useful sanitizers for debugging low-level code. Here's a few popular ones: AddressSanitizer, LeakSanitizer, MemorySanitizer, ThreadSanitizer. Details about them can be found here: https://github.com/google/sanitizers. They catch a lot of runtime bugs, similar to Valgrind. Anyway, the headers that provide interfaces/hooks for these tools can be found (on my Mac) in /usr/local/Cellar/llvm/6.0.0/lib/clang/6.0.0/include/sanitizer. It would be awesome if Zig supported these tools out of the box, just like clang does (I think Go uses at least ThreadSanitizer too).

[isaachier]

Also, seems LLVM IR already has function attributes relating to these sanitizers:

sanitize_address This attribute indicates that AddressSanitizer checks (dynamic address safety analysis) are enabled for this function. sanitize_memory This attribute indicates that MemorySanitizer checks (dynamic detection of accesses to uninitialized memory) are enabled for this function. sanitize_thread This attribute indicates that ThreadSanitizer checks (dynamic thread safety analysis) are enabled for this function. sanitize_hwaddress This attribute indicates that HWAddressSanitizer checks (dynamic address safety analysis based on tagged pointers) are enabled for this function.

[andrewrk]

This is in line with zig's goals. Essentially, we want debug builds to have as many of these sanitizers enabled at once as we can, as long as we can retain performance within 2 orders of magnitude of release-fast.

You can think of zig's safety checks as sanitizers.

Some of these sanitizers are not compatible with each other, so it is a bit of a puzzle. In some cases it may be worth zig doing its own sanitizer checks and ignoring LLVM's, or sometimes it may be worth using LLVM's sanitizers.

One thing we must not break, however, is zig's ability to cross-compile on any target for any target. Which means that if a sanitizer requires a runtime support library, zig must be able to build it from source for every supported target.

[isaachier]

OK that is harder than I thought. I agree the address sanitizer is redundant for Zig given that you are already checking for undefined values in debug mode etc. The checks I'd love to see are leak/double-free detection and race condition detection.

[andrewrk]

For leak/double-free detection, one way we can do that is to have a general-purpose allocator in the standard library which is considered "for debugging" and then either:

• in debug mode default the standard general-purpose allocator to this one
• you could manually enable these kind of sanitation by choosing the SanitizerAllocator in your main entry function rather than GeneralPurposeAllocator

For race condition detection, I want to investigate language-level support for that. I think there is a lot we can do.

[PavelVozenilek]

Very thorough, even excessive, memory leak detection could be implemented by the user. I did it in a C project: see #480. I enjoyed making it to fit my needs, current or envisioned.

If a functionality is hardcoded into the language/compiler/library ecosystem, it may limit freedom of a developer to make better fitting tools. Will I be able to use my own allocator (with somehow different API) and still use significant parts of Zig's stdlib? C++ decided to force one design on everyone, and it is not a success.

Example of a specific need: test may not leak. Test runner knows about all allocators (they register themselves when created, unregister on destruction), will record their current state at the beginning, and at the end of a test it will check no leak occured. Implemented fast enough to be usable.

[isaachier]

C++ didn't really force a design on anyone. You can use your own allocator type, and even maintain state in those allocators if using C++11.

Adding a leak allocator may be the way to go. Essentially, track the chunks of memory allocated to different objects. If any allocations remain when deinit is called, log an error. A good C++ implementation I have seen is here (courtesy of my last employer): https://github.com/bloomberg/bde/blob/ba252cb776f92fae082d5d422aa2852a9be46849/groups/bal/balst/balst_stacktracetestallocator.h. They have a few useful allocators in that library. BDE contains an STL rewrite that allows for standard containers with specific instances of allocators passed in, similar to Zig.

[PavelVozenilek]

@isaachier: I mean, large part of C++ STL (as of C++98) is married with their allocator design. If you do not want it, you have almost empty hands. Go and develop standard library replacement, like EASTL.

What I fear is the general tendency to design grand systems, where everything fits nicely together, and no part can be removed or changed w/o grave consequences.

[andrewrk]

Related: #2301