kubkon
commented
3 months ago Whilst looking into porting the UBSan runtime to Zig, I recalled a conversation with Andrew about the fact that
@panicought to only result in a single panic handler across all compilation units in a binary. Ideally, it should be possible to both reference the handler from C code, and define it in C code.The self-hosted compiler does not yet implement this behavior: uses of
@panicjust emit calls tostd.builtin.panic. This issue details a proposal for how to implement this behavior.The panic handler is provided by a function with the following signature (note that this depends on #17969):
/// The panic cause is unpacked into `cause`, `data0`, and `data1`; depending on `cause`, the data parameters may be ignored. /// If there is no error return trace, then `error_return_trace.index == std.math.maxInt(usize)`. /// If there is no return address, then `ret_addr == 0`. extern fn __zig_panic(cause: @typeInfo(PanicCause).Union.tag_type.?, data0: usize, data1: usize, error_return_trace: CompletedStackTrace, ret_addr: u64); const CompletedStackTrace = extern struct { index: usize, instruction_addresses: [*]usize, };Calls to
@panicare translated into calls to this function, converting thePanicCause,?StackTraceand?u64to the parameter types according to the doc comments.The more difficult question to answer is which compilation unit provides this handler, and how. I propose the following rules.
* If `-fpanic-handler` is passed to a compilation, the panic handler is exported from the ZCU if Zig source files are provided. Otherwise, a single compilation unit containing only the panic handler is implicitly added to the compilation. (The resulting object can be cached globally.) * If `-fno-panic-handler` is passed to a compilation, no panic handler is exported. * Otherwise, we export the panic handler according to the logic from the `-fpanic-handler` case, but: * If we are building an executable, the panic handler is exported with strong linkage. * If we are building an object or library (shared or static), the panic handler is exported with weak linkage.This system allows the user to set a preference using
-f[no-]panic-handlerif desired, but otherwise provides reasonable defaults. Let's analyze the possible failure cases:* a binary has no panic handler * a binary has multiple panic handlers, all with weak linkage * a binary has multiple panic handlers, at least two with strong linkageA binary having no panic handler is not possible under the default settings (it can only happen if
-fno-panic-handleris passed to every compilation). In this case, a link error occurs if any CU references the panic handler, but it is due to explicit user overrides preventing successful compilation, which seems like a non-issue.A binary having multiple panic handlers, all with weak linkage, can occur by default if the final executable is not built with Zig - for instance, if a library (static or shared) is built with Zig and linked into a C program which is compiled with
clangorgcc. In this case, the linker will functionally choose a random implementation (I think it's technically the first one?). This isn't ideal, but isn't easily resolvable, because no compilation unit is more authoritative than any other in this context. In practice, it will be rare for libraries to override the panic handler, so all will probably have the default, meaning the "randomness" here is typically unobservable. If any CU provides a custom panic handler, then the user can set-fpanic-handleron that one in their build script to give it strong linkage and hence make it override the others.A binary having multiple panic handlers, at least two of which have strong linkage, can never occur by default. It only happens when
-fpanic-handleris passed to at least one compilation. The main case I see being confusing here is passing-fpanic-handlerto a library, and linking it to a full executable, all with Zig: both will provide panic handlers with strong linkage, at which point we again defer to functionally random selection. This case is unfortunate: the user presumably expected the override to cause the panic handler to deterministically come from the library. For this reason, we augment the executable rule a little:
Unfortunately, this is not how it works. If the linker pulls in two input relocatable object files each exporting a symbol with strong linkage, it will generate a "duplicate symbol definition" error by default (please note this can be turned off for ELF, but this is always a hard-error for MachO, and in general, if happens, means there is a developer error that has to be corrected and not worked around). If however, there exists a relocatable object file that exports a colliding symbol with strong linkage but it is part of a static library, then it will only be an issue if it is pulled by the linker when resolving missing symbol references from other input relocatable object files that were specified explicitly on the linker link.
* If we are building an executable, the panic handler is exported with strong linkage, **unless another link object already provides it with strong linkage.**This extra case removes this unintuitive behavior, whilst still making sure to provide a panic handler in all cases. It makes sure that doing everything with the Zig compiler is a happy case, where everything "just works".
Let us now look at the impact of this design in practice. I'll list a few scenarios, and explain how the logic would play out (assuming no
-f[no-]panic-handleroverrides).Scenario 1: building an executable, from Zig code, which links to a static library, also written in Zig. In this case, the library exports a panic handler with weak linkage, and the executable overrides it with strong linkage. Thus, as is intuitive, the Zig code providing the entrypoint also provides the panic handler.
Why not simply always export the panic handler with weak linkage? Why bother with strong linkage at all? If a user wants to override this symbol they can set the export to strong themselves in their source code can they not? Having the panic handler always exported as weak would avoid a lot of issues I believe and would simply the design by reducing the number of rules required.
Scenario 2: building a shared library written in Zig, which is used by C code. Here, the shared library exports a weak symbol, and the C binary which uses it naturally exports nothing, so the weak symbol from the shared library is used. The C code could provide a strong
__zig_panicsymbol if it wants; I'm unsure if this would be used in this case (how does the dynamic linker handle weak symbols?).
Exported (weak) symbol from a shared library only impacts the image if this one is importing a symbol at runtime. Since in this proposal you said it is a hard-error to never have a panic handler available at link time, this is a non-issue - we never defer to runtime binding of a panic handler.
Scenario 3: building a static library written in Zig, which itself links another static library, also written in Zig. In this case, both handlers have weak linkage, so we get the "random" behavior discussed above. This, of course, assumes that the consumer of the library is compiled with a C compiler; if the final executable is built with Zig, then the executable overrides both panic handlers.
Combining two static libraries together does not perform any symbol resolution at all. If however you want to combine multiple input relocatable object files into a single relocatable object file (the infamous -r option), then symbol resolution will take place. In build.zig this would usually look as follows which to the best of my knowledge is not something folks use, well, except for me when testing the linkers:
const out = b.addObject(...);
out.addObject(a_o);
out.addObject(b_o);Scenario 4: building an executable written in Zig, with a C compilation unit which provides
__zig_panic. In this case, the "unless another link object already provides it with strong linkage" rule kicks in, and the C definition overrides the Zig one. I think this is the intuitive behavior in this case.
Again, if we always export as weak by default would make this situation crystal clear - unless the user forced exported their panic handler as strong, we pick any implementation unless __zig_panic is exported as strong in the C translation unit.
The main thing I'm not certain of is how all of this interacts with shared libraries, since I don't know how they interact with weak linkage in general (are the links resolved at link time or load time?). @kubkon, could you shed some light on this?
I'm also unsure if weak linkage is an issue on any target. From what I can see, Windows is a little shaky on it, but does seem to support it in at least some sense -- this will be another @kubkon question I suppose!
Windows doesn't have the concept of weak linkage. It has the concept of /alternatename:from=to however which can be used as poor man's alternative as far as I know 1. What happens then is that every translation unit we generate makes the panic handler symbol undefined but provides a fallback symbol in case the former is not found. This then allows the user to force the linker to never look for the alternative by simply exporting the panic handler symbol.
mlugg
Whilst looking into porting the UBSan runtime to Zig, I recalled a conversation with Andrew about the fact that
@panicought to only result in a single panic handler across all compilation units in a binary. Ideally, it should be possible to both reference the handler from C code, and define it in C code.The self-hosted compiler does not yet implement this behavior: uses of
@panicjust emit calls tostd.builtin.panic. This issue details a proposal for how to implement this behavior.The panic handler is provided by a function with the following signature (note that this depends on #17969):
Calls to
@panicare translated into calls to this function, converting thePanicCause,?StackTraceand?u64to the parameter types according to the doc comments.The more difficult question to answer is which compilation unit provides this handler, and how. I propose the following rules.
-fpanic-handleris passed to a compilation, the panic handler is exported from the ZCU if Zig source files are provided. Otherwise, a single compilation unit containing only the panic handler is implicitly added to the compilation. (The resulting object can be cached globally.)-fno-panic-handleris passed to a compilation, no panic handler is exported.-fpanic-handlercase, but:This system allows the user to set a preference using
-f[no-]panic-handlerif desired, but otherwise provides reasonable defaults. Let's analyze the possible failure cases:A binary having no panic handler is not possible under the default settings (it can only happen if
-fno-panic-handleris passed to every compilation). In this case, a link error occurs if any CU references the panic handler, but it is due to explicit user overrides preventing successful compilation, which seems like a non-issue.A binary having multiple panic handlers, all with weak linkage, can occur by default if the final executable is not built with Zig - for instance, if a library (static or shared) is built with Zig and linked into a C program which is compiled with
clangorgcc. In this case, the linker will functionally choose a random implementation (I think it's technically the first one?). This isn't ideal, but isn't easily resolvable, because no compilation unit is more authoritative than any other in this context. In practice, it will be rare for libraries to override the panic handler, so all will probably have the default, meaning the "randomness" here is typically unobservable. If any CU provides a custom panic handler, then the user can set-fpanic-handleron that one in their build script to give it strong linkage and hence make it override the others.A binary having multiple panic handlers, at least two of which have strong linkage, can never occur by default. It only happens when
-fpanic-handleris passed to at least one compilation. The main case I see being confusing here is passing-fpanic-handlerto a library, and linking it to a full executable, all with Zig: both will provide panic handlers with strong linkage, at which point we again defer to functionally random selection. This case is unfortunate: the user presumably expected the override to cause the panic handler to deterministically come from the library. For this reason, we augment the executable rule a little:This extra case removes this unintuitive behavior, whilst still making sure to provide a panic handler in all cases. It makes sure that doing everything with the Zig compiler is a happy case, where everything "just works".
Let us now look at the impact of this design in practice. I'll list a few scenarios, and explain how the logic would play out (assuming no
-f[no-]panic-handleroverrides).Scenario 1: building an executable, from Zig code, which links to a static library, also written in Zig. In this case, the library exports a panic handler with weak linkage, and the executable overrides it with strong linkage. Thus, as is intuitive, the Zig code providing the entrypoint also provides the panic handler.
Scenario 2: building a shared library written in Zig, which is used by C code. Here, the shared library exports a weak symbol, and the C binary which uses it naturally exports nothing, so the weak symbol from the shared library is used. The C code could provide a strong
__zig_panicsymbol if it wants; I'm unsure if this would be used in this case (how does the dynamic linker handle weak symbols?).Scenario 3: building a static library written in Zig, which itself links another static library, also written in Zig. In this case, both handlers have weak linkage, so we get the "random" behavior discussed above. This, of course, assumes that the consumer of the library is compiled with a C compiler; if the final executable is built with Zig, then the executable overrides both panic handlers.
Scenario 4: building an executable written in Zig, with a C compilation unit which provides
__zig_panic. In this case, the "unless another link object already provides it with strong linkage" rule kicks in, and the C definition overrides the Zig one. I think this is the intuitive behavior in this case.The main thing I'm not certain of is how all of this interacts with shared libraries, since I don't know how they interact with weak linkage in general (are the links resolved at link time or load time?). @kubkon, could you shed some light on this?
I'm also unsure if weak linkage is an issue on any target. From what I can see, Windows is a little shaky on it, but does seem to support it in at least some sense -- this will be another @kubkon question I suppose!