Exceptions vs. non-local control constructs vs. unwinding

[RossTate]

Exceptions, non-local control constructs, and unwinding are all related but different entities. It is important to understand the distinction between these concepts, and to see that distinction consider the following C++ program:

#include <csetjmp>
int main() {
    jmp_buf env;
    int val = setjmp(env); // val is 0 on first call
    if (val==0) {
        try {
            longjmp(env, 5); // makes setjmp return 5 "instead"
        } catch (...) {
            return 1; // never executed
        }
    } else {
        return 2; // executed
    }
}

According to the C++ spec, this program returns 2, not 1. This is because although longjmp is a non-local control construct, it is not an exception. (And to clarify, the behavior of this program does not depend on how catch (...) is specified to interact with foreign exceptions because longjmp is not considered a foreign exception either.) So exceptions are distinct (but closely related to) from non-local control constructs.

I was careful to make sure this example involves no unwinding. How longjmp interacts with unwinding is not defined by the C++ spec, intentionally deferring it to the platform. (Similarly, the C++ spec does not specify how unwinding interacts with uncaught exceptions, intentionally deferring it to the platform because the behavior of single-phase vs. two-phase EH implementations differ here.) The GNU compilers do not have longjmp cause unwinding, whereas Visual Studio does by default (though you can turn it off). (Visual Studio also lets you configure whether foreign/system exceptions should cause unwinding and discusses why you would want unwinding for some circumstances and why you would not want unwinding for other circumstances.) So non-local control constructs are distinct (but closely related to) from unwinding. (To clarify, I am not advocating to add non-unwinding non-local control constructs in this proposal.)

---

Okay, so why do these distinctions matter? Well, just as many languages compiling to C use setjmp/longjmp to implement their own non-local control constructs (as it is the only non-local option), many languages compiling to WebAssembly will use throw/catch to implement their own non-local control constructs (again, as it is the only non-local option). We should anticipate this. Similarly, WebAssembly eventually add other non-local control constructs. We should leave room for this. unwind does both by providing a way to specifying unwinding code with no assumptions about why the stack is being unwound. It is closely related to unwinding clauses in other systems—fault, (part of) finally, unwind-protect, and (part of) dynamic-wind—all of which similarly specify/treat an unwinder as a block/function of type [] -> [].

Now, one particular non-local control construct that will need to be emulated with throw/catch is setjmp/longjmp. Because the spec gives us the option to have longjmp cause unwinding, this is mostly straightforward to do using some $longjmp exception event. But there's a problem if one translates catch (...) to catch_all: the catch_all will mistake the $longjmp event for an exception. That would make our example C++ program above incorrectly return 1. And while yes, you could hack the compilation of catch (...) to exclude the $longjmp event, that only excludes your own long jumps, failing to exclude other C/C++-as-wasm program's long jumps as well as other languages' non-local control constructs, which catch (...) seems to specifically not be intended to catch.

Hopefully this illustrates part of the rationale behind unwind, and hopefully this more concrete example better illustrates the concern about compositionality of catch_all that I had expressed more abstractly in #128.

[rossberg]

I draw the exact opposite conclusion: given the wealth of arcane control features and semantics across languages, it would be a losing game to try supporting them all natively. Such an additive approach would produce a monster of a language. Instead, we need to enable all their implementations by a combination of a general and composable base mechanism for control transfer and the ability to code up as much of the specifics as possible in user space.

As for cross-language control, I had assumed that their is agreement that it falls under the same "no seamless interop" caveat that Wasm already applies for cross-language data. All an exception/control mechanism for Wasm can hope to achieve is enabling interop. It cannot magically provide it, as there is no universal solution. Multiple interacting languages will either have to agree on a common control ABI, or avoid cross-language control transfer. Maybe interface types can one day be enriched with a control dimension.

[RossTate]

given the wealth of arcane control features and semantics across languages, it would be a losing game to try supporting them all natively

This is a strawman argument. Leaving room to add more control constructs is not the same as adding all the control constructs.

It cannot magically provide it, as there is no universal solution.

This, too, is a misrepresentation of what I was advocating for. I was advocating for a solution that avoids unintentional interference of control constructs. I illustrated that C++ avoids such unintentional interference, as do many other languages.

On that note, I pointed out that unwind-like constructs are widespread, but you have yet to point me to a reference for catch_all/rethrow.

[aheejin]

@RossTate

Whether we should compile catch (...) as catch_all or just catch $cpp_exception_tag is a matter of the tool convention, not the spec. Tool conventions are discussed in the tool-convention repo. (By the way the EH scheme there has not been updated for a long time, so it's not up-to-date)

I'm not sure why is C++'s catch (...) compilation scheme related to necessity for unwind.

[RossTate]

Whether we should compile catch (...) as catch_all or just catch $cpp_exception_tag is a matter of the tool convention, not the spec.

The example above shows that catch_all would be a semantically incorrect way to compile catch (...).

I'm not sure why is C++'s catch (...) compilation scheme related to necessity for unwind.

I'm using the fact that catch (...) is not supposed to catch longjmp to concretize that there are non-local control constructs that are not exceptions. For these, it does not make sense to catch_all and rethrow them—many of them already have a predetermined location in mind and are just piggybacking throw and catch to get to that location. (Hence you don't see this catch_all/rethrow pattern in other systems with non-exceptional non-local control.) Programs just need the portion of the stack that was skipped over by the non-local control transfer to be unwound, and unwind provides the means for precisely that functionality.

[aheejin]

I'm using the fact that catch (...) is not supposed to catch longjmp to concretize that there are non-local control constructs that are not exceptions.

As I said, we can compile catch (...) to catch $cpp_tag. Actually, we are doing it even now (using br_on_exn for the same effect). I'm not sure why you are assuming otherwise.

Programs just need the portion of the stack that was skipped over by the non-local control transfer to be unwound, and unwind provides the means for precisely that functionality.

Still don't understand what this has to do with unwind. First, we haven't defined catch_all should catch non-exception control flows yet in this MVP proposal. But it is assumed that unwind runs for any kinds of control flow construct, right? And longjmp does not cause destructors to run. So according to your argument, implementing longjmp with throw and using unwind with it is actually a problem, not a solution.

[RossTate]

I'm not sure why you are assuming otherwise.

Because you said in our meetings that the reason you wanted catch_all was to support catch (...), and multiple discussions reference this expectation. If that is not the case, then what is your intended purpose for catch_all?

And longjmp does not cause destructors to run.

That spec is poorly worded, as the second sentence there contradicts the first. The second sentence states that the behavior is undefined if replacing the given setjmp/longjmp with a catch/throw would cause non-trivial destructors to fire. This spec is more clear about the undefined behavior, and this spec goes further to clarify that the choice tends to be compiler-specific (and discusses flags for configuring this behavior).

But it is assumed that unwind runs for any kinds of control flow construct, right?

No, because br and the like do not cause unwinding. And if you wanted a version of longjmp that does not unwind, then that would not be another counterexample. But at present, all local control flow does not unwind whereas all non-local control flow does unwind. unwind makes it easy to maintain that property in future extensions if we want, and I don't think there's any reason to consider breaking that property in this MVP proposal.

First, we haven't defined catch_all should catch non-exception control flows yet in this MVP proposal.

What I was trying to point out is that in the MVP all non-local control flow will likely be encoded with catch/throw for quite some time, which means that catch_all will effectively catch all non-local control flow for quite some time.

[rossberg]

given the wealth of arcane control features and semantics across languages, it would be a losing game to try supporting them all natively

This is a strawman argument. Leaving room to add more control constructs is not the same as adding all the control constructs.

It's not the same yet, but it is the end of the path that such an approach inevitably puts you on. (Either that, or you intend to carve language privilege into stone forever.)

It cannot magically provide it, as there is no universal solution.

This, too, is a misrepresentation of what I was advocating for. I was advocating for a solution that avoids unintentional interference of control constructs. I illustrated that C++ avoids such unintentional interference, as do many other languages.

You can program up this solution by using two different Wasm-level exception tags in the C++ runtime, as @aheejin said. I don't see how unwind helps here, unless you are suggesting that we should also make longjmp a Wasm primitive? For that, see above.

On that note, I pointed out that unwind-like constructs are widespread, but you have yet to point me to a reference for catch_all/rethrow.

Catch-all/rethrow is what e.g. JS engines do at the lower level to actually implement something like finally/unwind. Both features are also present in popular languages like C++ or C#.

[RossTate]

Catch-all/rethrow is what e.g. JS engines do at the lower level to actually implement something like finally/unwind. Both features are also present in popular languages like C++ or C#.

Engines are free to implement these constructs however they want. JS engines always "rethrow" an unwinding exception in the same dynamic context it was intercepted in. unwind ensures precisely that. Neither C++ throw; nor C# throw; compile to rethrow. Many implementations of C++ do not compile destructors to catch_all/rethrow both because of longjmp and to interact well with Win32 SEH. The .NET CIL has a rethrow instruction (which neither C# throw; nor finally compile to), but it explicitly says that correct CIL does not use rethrow inside any exception handlers. That is, correct CIL requires the exception to be rethrown in the same dynamic context it was caught in, which is what unwind ensures but catch_all/rethrow does not.

So even these systems seem to be restricted to unwind, which again is in line with other systems that have more advanced notions of control and have investigated compositionality with respect to control more thoroughly.

[RossTate]

@aheejin Given your clarification that catch_all is not planned to be used for catch (...) (avoiding the issue with interacting longjmp), can you clarify what catch_all is planned to be used for?

[rossberg]

Engines are free to implement these constructs however they want. JS engines always "rethrow" an unwinding exception in the same dynamic context it was intercepted in.

Engines do things like compiling finally by having a shared piece of code that is entered with a flag marking its continuation mode. Compilers targeting the EH might want to use a similar technique, but that does not seem to be possible anymore under the current proposal (but was before!).

I don't understand how the current proposal allows a compiler to implement finally at all in a way that consistently uses unwind without requiring to duplicate the entire, arbitrarily large handler code (which would lead to a code size blow-up exponential in the nesting depth of finally handlers). You cannot easily factor it out into a function like in a high-level language, since the handler almost always needs access to locals.

[RossTate]

I don't understand how the current proposal allows a compiler to implement finally at all in a way that consistently uses unwind without requiring to duplicate the entire, arbitrarily large handler code (which would lead to a code size blow-up exponential in the nesting depth of finally handlers).

It doesn't allow this.

[aheejin]

@rossberg

Engines do things like compiling finally by having a shared piece of code that is entered with a flag marking its continuation mode. Compilers targeting the EH might want to use a similar technique, but that does not seem to be possible anymore under the current proposal (but was before!).

How did the previous proposal allow this? You mean the one with exnref, because we were able to factor out the code?

I don't understand how the current proposal allows a compiler to implement finally at all in a way that consistently uses unwind without requiring to duplicate the entire, arbitrarily large handler code (which would lead to a code size blow-up exponential in the nesting depth of finally handlers). You cannot easily factor it out into a function like in a high-level language, since the handler almost always needs access to locals.

This doesn't currently allow this. What do you suggest as an alternative? Re-add exnref? Or add finally?

I'm not against to adding finally as a separate primitive. It was discussed in #11 and people (including you) said we didn't need it as a primitive because it could be compiled away.

[tlively]

It would be good not to get too caught up in the problem of supporting finally without code duplication. The JVM also requires code duplication to implement finally and apparently it hasn't been a problem in practice.

In more detail, the JVM specification documents a compilation scheme for finally that actually does deduplicate code using the jsr instruction, which essentially calls a block of code in the current function as if it were a nested function. However, the docs for jsr note that it hasn't been used by Oracle's Java compiler since Java SE 6. In other words, Java has been duplicating the handler code for finallys with no problems in practice since the end of 2006.

[aheejin]

@RossTate

I'm using the fact that catch (...) is not supposed to catch longjmp to concretize that there are non-local control constructs that are not exceptions. For these, it does not make sense to catch_all and rethrow them—many of them already have a predetermined location in mind and are just piggybacking throw and catch to get to that location. (Hence you don't see this catch_all/rethrow pattern in other systems with non-exceptional non-local control.)

I don't understand why catch_all and rethrow will get in the way of non-exceptional non-local control flow, such as longjmp. You said it has a predetermined location, which is true, but that's something our C++ toolchain has to ensure that it matches. longjmp is not a wasm primitive, and this is a toolchian (or C compiler) correctness problem and not a spec problem.

@aheejin Given your clarification that catch_all is not planned to be used for catch (...) (avoiding the issue with interacting longjmp), can you clarify what catch_all is planned to be used for?

It was originally added to give wasm a way to do custom tasks for all non-local control flows. For example, wasm wants to print some message for all exceptional (or non-local) control flows.

[RossTate]

It was originally added to give wasm a way to do custom tasks for all non-local control flows. For example, wasm wants to print some message for all exceptional (or non-local) control flows.

unwind seems to serve this purpose now. Can you clarify what catch_all is planned to be used for now?

[aheejin]

unwind is intended for cleanups and not user handlers. In the current proposal unwind and catch_all are virtually the same, so you may use unwind for that purpose, but not in the follow-on proposal where unwind's semantics will be different from that of catch/catch_all.

[RossTate]

Can you provide a concrete example of how catch_all is planned to be used in the current proposal?

[ioannad]

I think catch_all (which btw is also part of the already implemented 1st proposal) is intended for catching exceptions unknown to the module.

[RossTate]

I understand the intent, but intents do not always match up with actual usage. I am looking for an actual usage that someone plans to generate to support some aspect of their language.

[aheejin]

I already answered about the usage: wasm needs a way to handle unknown non-local control flows, such as printing a message. unwind is added as a preparation for the future 2PEH proposal, and we can't use it as user handlers there. C++ may not use catch_all for catch (...), but I don't understand why that is the reason we should remove the functionality.

[rossberg]

@aheejin:

How did the previous proposal allow this? You mean the one with exnref, because we were able to factor out the code?

Yes. Here is the situation as I perceive it.

Before, we had a fairly canonical proposal with one universal try-construct that could express everything we needed it to express right now.

Now, we have an ad-hoc proposal that already has a zoo of 4 different try-constructs in order to accommodate some future use cases, but cannot even efficiently express everything we need right now. So we will likely end up with an even larger zoo.

In a low-level VM, I could see the need for perhaps two variants of try. But with 4+, I think we have taken a wrong turn. And to be clear, I don't think that's anybody's fault, and I especially sympathise with you trying to accommodate all the competing requests and making progress -- navigating the incompatible world views that we see on the CG these days has become almost impossible. We simply have run into a serious case of design-by-committee with too many hypotheticals.

It was discussed in #11 and people (including you) said we didn't need it as a primitive because it could be compiled away.

Yes, but that was when we still had a coherent design that allowed doing that. It would be great if it still did. That's the problem: this is no longer possible, at least not without unbounded code duplication.

What do you suggest as an alternative? Re-add exnref? Or add finally?

I'm not against to adding finally as a separate primitive.

I certainly would prefer to not add yet another try-construct. One suggestion I briefly made earlier was to generalise unwind to a finally that receives a Boolean parameter allowing to distinguish regular from exceptional entry.

But ultimately, that would merely be patching around the corners. The deeper problem is that we lack a principled overall design. Unfortunately, I don't have a constructive suggestion at this point other than going back to the drawing board (which is super frustrating, and I don't wanna be that guy).

@tlively, I'd argue that enforcing unbounded code duplication is poor design, the JVM notwithstanding -- which can hardly be seen as a pinnacle of good and forward-looking design. As a counter point, .NET, which had the luxury to learn from some of the JVM's mistakes, supports efficient finally (albeit by introducing its own additive approach).

[rossberg]

@RossTate, I think the need for a catch-all is self-evident in a low-level VM, even if it's just to provide a means to implement robustness and diagnostics against uncaught exceptions on some level of a software stack.

[aheejin]

@rossberg

When we were discussing #11, we didn't have exnref, so I think the necessity for code duplication when compiling away finally is not different from the current situation. Then we had the first version of the proposal, with try, catch $tag, and catch_all.

I share your concerns on proliferation of different try variants. I think we can remove unwind at least in the current version of the proposal; we can add it later if we need it in the 2PEH proposal later, if we get to make it. I understand why @RossTate wants to have it, but what I think is that it doesn't have to be added in this version of the proposal.

delegate is a tricky one to remove, because this was added in order to reduce code duplication, now that we don't have exnref.

If we remove unwind, we are basically back to the first version of the proposal, with only one addition of delegate, which I think is not too bad. (Generalizing unwind to finally with a boolean parameter also sounds OK to me.)

As you said, being able to unifying catch and catch_all in a single catch was a good part of the previous proposal, but it also added the dependency for the reference types proposal, so I think it's some trade-off. Also one of the reasons we added exnref was that we thought we could extend it to your typed continuation proposal, but your proposal didn't end up using it after all.

I appreciate your sympathy for this tumultuous CG process. As you've probably seen, the discussions in the repo in recent months have not been easy. I wish you weighed in more and shared your concerns you are currently sharing in the discussions before we passed the new proposal though, so that we were able to take into account your concerns more.

[RossTate]

I think the need for a catch-all is self-evident in a low-level VM, even if it's just to provide a means to implement robustness and diagnostics against uncaught exceptions on some level of a software stack.

catch_all has no way to distinguish between caught and uncaught exceptions. There are also known applications (see #101) that need uncatchable non-local control transfer (that does not run unwinders). So while I agree with the high-level concern here, I do not believe catch_all properly addresses that concern.

It is also worth noting that even the few other systems with something like a catch_all also have something like unwind/finally and also specifically restrict rethrow (if they have it at all) to disallow it from throwing the caught exception in a different context.

---

From what I can tell, the prevailing misunderstanding seems to be that unwinding is done if and only if one is searching for an exception catch-point. Neither of these directions are true. Neither direction of this if and only if holds. To counter the "if" direction, we already have that a trap ignores unwinders, and #101 gives an application for an exception that ignores unwinders. But that direction is less pressing, so I only mention it to illustrate the disconnect between these constructs. More importantly, to counter the "only if" direction, the unwinding phase of two-phase exception handling has no search for an exception catch-point as that catch-point was already identified in the first phase. Another example is Common Lisp's return-from, which does not search the stack for a catch-point and only searches for unwinders.

As for finally, @aheejin requested that we not worry about directly supporting it now because it was not helpful for the pressing customer needs and can be indirectly supported through code duplication.

[rossberg]

@aheejin:

When we were discussing #11, we didn't have exnref, so I think the necessity for code duplication when compiling away finally is not different from the current situation.

Ah, fair point. I probably did not notice this limitation at the time.

(For completeness, there is a way to compile finally without code duplication in the current proposal, by introducing an auxiliary one-off exception:

try A finally B  ~~>  (try (do (try A (throw $Aux) unwind B)) (catch $Aux))

where the type of $Aux matches the result type of the try block A. But obviously, this translation would be pretty expensive on the regular path, so isn't attractive.)

delegate is a tricky one to remove, because this was added in order to reduce code duplication, now that we don't have exnref.

Thinking out loud, there are a couple of options, but they all have a price:

• Unify try-delegate and try-catch: instead of a catch block, each catch clause would just have a regular label to branch to, receiving the exception arguments. Then we don't need a separate delegate, because multiple handlers can target the same label. Downside: to support rethrow, you'd need some simple form of exnref.

• Unify try-delegate and rethrow: under 1PEH, delegate is just a variation of rethrow. In principle it would be possible to generalise rethrow with a second immediate that determines where to rethrow. Then we can express try-delegate as try-catch_all-rethrow. But I'm not sure if that's the best way to go about things. At least it doesn't fit with the approach to 2PEH as currently imagined.

Maybe structured EH is just too high-level for Wasm. Not that I have a better suggestion...

As you said, being able to unifying catch and catch_all in a single catch was a good part of the previous proposal, but it also added the dependency for the reference types proposal, so I think it's some trade-off. Also one of the reasons we added exnref was that we thought we could extend it to your typed continuation proposal, but your proposal didn't end up using it after all.

FWIW, reference types as such should not be an issue, since they're at phase 4 and about to land Real Soon Now(tm). But in any case, catch vs catch-all is the thing I worry about least.

I appreciate your sympathy for this tumultuous CG process. As you've probably seen, the discussions in the repo in recent months have not been easy. I wish you weighed in more and shared your concerns you are currently sharing in the discussions before we passed the new proposal though, so that we were able to take into account your concerns more.

Yeah, I'm sorry, I think I tried. But simultaneously being stuck in multiple other CG discussions that are even more tumultuous and time-consuming doesn't help. :(

[rossberg]

@RossTate:

catch_all has no way to distinguish between caught and uncaught exceptions.

In a scenario where it is used to gracefully handle uncaught exceptions in a given component, by construction, any exception that ever reaches it is otherwise uncaught, relative to that component.

There are also known applications (see #101) that need uncatchable non-local control transfer (that does not run unwinders).

That may be so, but has nothing to do with exceptions, uncaught or otherwise.

[RossTate]

In what you described, more accurately you want to intercept all non-local control transfers out of the component. unwind does that.

[ioannad]

In what you described, more accurately you want to intercept all non-local control transfers out of the component. unwind does that.

How does it do that when it's equivalent to catch_all ... rethrow? Or are you referring to a different unwind?

And in general I'm confused about terms being used by different people with different meanings, in this proposal's issue discussions. It's not always clear from the context which meaning is meant.

[RossTate]

@ioannad unwind gets executed for anything that unwinds the stack, including single-phase exceptions. For the described scenario, if one doesn't want an exception from the component to propagate further up the stack, one can br or return from within the unwind.

[aheejin]

@RossTate

I don't think people, including me here, are confused about something. What I (and some of other people, I believe) was talking about was, unwind can have its uses, but in the MVP proposal it is the same as catch_all, so we can keep the MVP simple and add unwind later when necessary. That's all. Not having unwind now does not hinder compatibility with the future 2PEH, no matter what that will look like.

Also, when you argue to remove catch_all, your argument is "C++'s catch (...) will not use it". But when you argue to keep unwind, you bring not only future 2PEH but also Common Lisp, which we don't even have a remote plan to support, or someone's question from February (#101), or other hypotheticals. #101 was just a passing question from someone, and we concluded it was not a problem or security threat as you suggested after all, and I woudn't want to relitigate all that again.

[RossTate]

At present, no one has provided a purpose for catch_all besides the role unwind would play. All related systems serve this purpose through an unwind clause or the like. unwind is particularly useful for interoping with foreign systems as it gives them a way to cooperatively unwind your stack without you making assumptions about their non-local control internals.

There is a well-known example of this in the C/C++ community, which is Windows SEH. Windows OS provides a service that helps C/C++ programs interop with OS exceptions. One particular use case is unwinding. And it's known in the Windows C/C++ community that, even if your own program doesn't make direct use of SEH, if it uses a library that does then it is important that it still be compiled to make use of SEH specifically so that destructors get run by SEH exceptions. Someone compiling WebAssembly into this ecosystem will need to do the same thing. The unwind clause has a straightforward compilation. However, there is no analog to rethrow in SEH except if you are rethrowing into the same handler stack the exception was caught in, e.g. rethrow only at the end of a catch_all block. I suspect this contributed to why .NET went out of its way to disallow its rethrow instruction from being used within a different handler stack.

So if we were to pick between catch_all/rethrow and unwind, I would pick unwind. It is the simpler of the two, it is known to work well with many extensions, it is known to work well with foreign systems (e.g. Windows SEH), and it can be found in other systems. On the other hand, the only other system I can find with the ability to catch and rethrow foreign exceptions specifically went out of its way to disallow the expressiveness that catch_all/rethrow adds, and we ourselves have no applications for that additional expressiveness.

[tlively]

@RossTate, can you point to docs about using SEH analogously to unwind? The docs I found only discuss try-except and try-finally, but __finally is different from unwind because it executes in non-exceptional cases as well.

[dschuff]

I have to admit, I don't really understand why we are arguing so much about this. Given that the MVP proposal has 2 primitives that are identical except for the behavior at the end, I think it does make sense to consolidate them. It seems we have disagreements about what the behavior of this primitive should be when it interacts with future 2PEH or other post-MVP proposals. But I don't think that changes what the current primitive should do in the MVP (so I don't see a need to change the currently-proposed behavior beyond picking one of our variants). Additionally (and more importantly) if and when we write a proposal for 2PEH or whatever, we can decide at that time what the behavior of the MVP primitive will be in that context. We can decide that the MVP primitive should act like unwind (i.e. not terminate the 1st phase and always run in the 2nd phase), or we can decide that it should act like catch_all and terminate the 1st phase. Or whatever other behavior we like. And of course we can add an additional primitive at that time that complements whatever we decide the MVP primitive's behavior is. What we choose to call the MVP primitive does not limit those choices at all.

Obviously picking one name or the other signals some intent (and is the source of the present disagreement I think). But regardless of what we call it, users of the MVP are going to use it for whatever different purposes they want; and some of those uses will be in line with whatever intent we have now, and some of them won't. (And consequently some of those users will need to recompile all their programs if they want to have nice interaction with the new primitives). But we can't control that regardless.

Of course we still have to pick a name, but the stakes here seem pretty low to me.

[RossTate]

One of the items of this issue was to determine the intended purpose of catch_all. In our video chats and in various issues, one of the purposes that had been suggested was to use catch_all for compiling catch (...). It sounds like that's not the case, which is very useful to know because then the question seems to become: do we use unwind or do we use catch_all. For that question, it is important to note that the two are not equivalent: whereas unwind can be translated into catch_all, the reverse is not true. That is, catch_all is more expressive than unwind, so the two relevant questions are:

1. Is that additional expressiveness useful?
2. Is that additional expressiveness problematic?

For 1, at present no examples of how the additional expressiveness would be useful have been provided. I have also been going through the effort of seeing how various languages would translate their own control constructs and have yet to find a case where the additional expressiveness would be helpful.

For 2, I am concerned about how it would interact with other control-flow constructs and with foreign systems. Other systems with two-phase exceptions or with non-exceptional forms of non-local control flow all use an unwind or finally construct (or both). Windows SEH is a concrete example of a foreign system designed to facilitate exception-handling interop between the application and the OS, and it supports unwinding but not rethrowing during the unwind phase.

Per @tlively's request, I'll go into Windows SEH a bit. The documents referred to are the C/C++ language-level extensions for SEH. But SEH itself is an OS interface, one that is notoriously poorly documents, with this as the standard reference. Conceptually, the OS uses SEH to find handlers for OS exceptions and the like and to unwind the stack between the throw-point and the handler. It works through a data structure (generally nested directly into the program's stack) of callbacks. These callbacks are responsible for indicating both how to handle the exception (used in the search phase) and how to unwind the stack (used in the second phase if the exception is "caught"). To "catch" an exception, a handler informs the OS library to unwind the stack up to the current handler, at which point the OS library runs the callbacks in the data structure from the throw-point up to but excluding the handler and then returns control to the handler. C++ programs can therefore compile to use SEH (the referenced documents mention the flags to use) even if they do not use _finally so that, in particular, if libraries they link with catch SEH exceptions then those libraries will run the callbacks firing the C++ destructors (and so that C++ exceptions run SEH destructors in those libraries). Note that these handlers are only run when an exception occurs, so _finally has to go out of its way to run the relevant code for local exits. It is straightforward to compile unwind to work with SEH (you just register an unwind callback), but not so straightforward to compile unwinding with catch_all/rethrow as you can't implement rethrow by "rethrowing the current exception" since there's no current exception to rethrow in the second phase of SEH.

The standard is that unwinders are represented by [] -> [] blocks/functions/callbacks. This standard is seen across many languages and even APIs like Windows SEH, and from what I can tell it is universal across languages/VMs/runtimes that directly support at least one of two-phase exception handling, lexical non-local control, or first-class stacks. This standard has been tried and tested in many production systems for decades. So I don't understand why there is so much push for using a non-standard construct like catch_all/rethrow to represent unwinders when no related system has needed the additional expressiveness and when it has never been tried and tested in a production system with direct support for any one of two-phase exception handling, lexical non-local control, or first-class stacks.

[tlively]

Thanks for the SEH explanation, @RossTate!

An important point of concern for you is that catch_all is more expressive than unwind, but as far as I can tell, they're identical in expressiveness. Here's how I would implement them in terms of each other:

try A* catch_all B* end is equivalent to try A* unwind B* (br 0) end

try A* unwind B* end is equivalent to try A* catch_all B* (rethrow) end

Does that line up with your understanding of their semantics?

[RossTate]

While that equivalence was a significant basis for the decision in #126, it did not hold in the semantics that @ioannad formalized in #87. When I pointed this out, @aheejin and @rossberg said they preferred the semantics in #87, and so we went with that. I suggested that we should then revisit #126 as a major premise was invalidated, but that suggestion was ignored. So another source of confusion is that discussions have been conducted with different understandings of the semantics. My concerns above are with respect to the formalized semantics, in which catch_all/rethrow is more expressive than unwind.

[tlively]

Ok, I'll go revisit the formal semantics in #87 and see where I went wrong with my translations 😕

Edit: got it, the formalized difference is that you can't rethrow in an unwind. I'll have to think more about the ramifications of that.

[RossTate]

Oh, sorry, I misread something. Your first equivalence doesn't address rethrow, but that's where the difference in expressiveness lies.

[dschuff]

I actually think the difference of whether rethrow is allowed in unwind is still not relevant to my previous point, which is that in the MVP, both constructs can either propagate the exception or not (since all rethrow does is continue the single-phase unwinding). I said before that when we have a followup proposal we can define what our catching primitive does wrt the new proposal, but that is also true of rethrow.

Windows SEH is a concrete example of a foreign system designed to facilitate exception-handling interop between the application and the OS, and it supports unwinding but not rethrowing during the unwind phase.

Here I think you are assuming that in some followup spec, our rethrow will mean "start a new first phase" rather than "continue unwinding". But there's nothing in the current spec that requires that. Our current rethrow can remain a shorthand for "continue unwinding" when used that way. If we did that, we'd essentially be turning our existing catch_all into the primitive you're calling unwind and that would be OK! It would just mean that when we wrote the new proposal, we concluded that we did want the semantics of unwind (instead of a surface-level catch_all). And that's an option that we aren't closing off now.

[RossTate]

If I understand you correctly, that is not the semantics specified in #87. @ioannad described the formalized semantics as throwing the caught exception from the point of rethrow, and @rossberg and @aheejin confirmed that they wanted the rethrown exception to be caught by try/catch blocks within the catch_all. The reasoning there clearly was that catch_all was only for exceptions, not for general unwinding, which is contradictory to the statements above. So the semantics of rethrow is not simply to resume unwinding, and it is this aspect of the semantics that has not been motivated, that makes it distinct from related systems' representation of unwinding, and that is the source of my concerns above.

[aheejin]

@RossTate I don't think we need to revisit all my or @rossberg's comments in #87. @dschuff's point is, whatever the presumed intention for rethrow there was, we don't encode that in the current proposal; it'd be more precise to say we can't encode that in the current proposal, because there is only single phase unwinding. What we've been saying is, we don't need a separate unwind in this MVP propsal.

Also while catch_all and rethrow can be more expressive than unwind, but I honestly don't understand why being more expressive, not less, is a problem. (Also it should be more expressive to be used for user handlers.)

For usages of catch_all, you keep saying we haven't provided any, but people, including I, mentioned they are to be used for user handlers for custom actions multiple times. You also keep talking only about catch (...), but I don't think we said it was a sole important reason for catch_all's existence. And I also said it was a matter of tool convention and not the spec. As I said, you use "C++ will not use it" as an argument for something, while you bring many other things (including Common Lisp, SEH, someone's random question, your hypothetical proposal, ...) when you argue about other things.

The point is, in the current proposal with single phase unwinding, the functionality of unwind can be expressed by the mostly same catch_all and rethrow. In other posts, you argued we should remove some other instructions saying they can be expressed in terms of other instructions if I did some (probably nontrivial and expensive) code transformation and duplication. In the current proposal, unwind can be expressed perfectly using catch_all and rethrow without any code transformation or cost.

I think it is a sad state that the proposal has two different instructions that mostly do the same thing, with respect to the MVP's single phase unwinding. We can maybe keep both, as a compromise, if both sides can't agree until the end, but I don't think the toolchain will need unwind at least until we proceed to the future 2PEH proposal (if we get to make it).

[RossTate]

What I have been saying is that we don't need catch_all in this proposal. I have given a number of reasons why I prefer unwind over catch_all. Can you explain why you prefer catch_all over unwind? (I already illustrated above how unwind can easily be used for the application of catch_all that you just provided.)

[ioannad]

While that equivalence was a significant basis for the decision in #126, it did not hold in the semantics that @ioannad formalized in #87. When I pointed this out, @aheejin and @rossberg said they preferred the semantics in #87, and so we went with that. I suggested that we should then revisit #126 as a major premise was invalidated, but that suggestion was ignored. So another source of confusion is that discussions have been conducted with different understandings of the semantics.

@RossTate in fact I didn't formalise that part. rethrow's behaviour has always been like that, since the 1st version of the exception handling proposal. @aheejin has replied to you in several points explaining this misunderstanding about rethrow.

[ioannad]

Ok, I'll go revisit the formal semantics in #87 and see where I went wrong with my translations confused

Edit: got it, the formalized difference is that you can't rethrow in an unwind. I'll have to think more about the ramifications of that.

I have not updated the formal semantics yet, since I was waiting for a decision wrt unwind. I'll write down the updated semantics asap and update the comment with the draft spec in #87 .

[tlively]

@ioannad It would be great if you could extract those semantics out into a PR that adds a new document to the repo. That way it would be much easier to comment on it and see its history :)

[ioannad]

@tlively done: #143 Btw the equivalence you described between catch_all and unwind is now hardcoded in that spec, following @aheejin's current spec overview in #137

@RossTate because rethrow has now an immediate, as per #137, the reduction we were discussing in #87 would be now written:

try (throw $exn) catch $exn (try (rethrow 1) catch $exn instr* end) end

and it still reduces to instr* (as it does in the other two proposals as well). I can't think how this could reduce to throw $exn as you suggested. I don't think I understand what problem you see with this reduction.

[rossberg]

@ioannad, sorry, I'm a bit lost on the context of the reduction you are referring to (#87 is a very long thread :) ). Can you restate what would be reduced to the above?

[ioannad]

@rossberg I am referring to this, this, and this comment by @RossTate, to which I replied with the reductions according to the 2nd proposal in this comment, asking whether the semantics of rethrow have changed. Your reply at the time was that indeed

The inner catch around the rethrow should handle it.

IIUC, rethrow has always created a new exception, is that correct?

Can you restate what would be reduced to the above?

The question was whether try (throw $exn) catch $exn (try (rethrow) catch $exn instr* end) end should reduce to instr* or to (throw $exn).

[RossTate]

To be concrete, try (throw $exn) catch $exn (try (rethrow 1) catch $exn instr* end) end reduces to instr*, as does try (throw $exn) catch_all (try (rethrow 1) catch $exn instr* end) end. The latter is not expressible with unwind.

---

To get back to this point, if we are making a decision between two constructs, it would be good to discuss a comparison between the two. Above I have given a number of reasons why to prefer unwind over catch_all/rethrow. It would help advance the conversation to be given reasons why to prefer catch_all/rethrow over unwind.

[ioannad]

@RossTate:

To be concrete, try (throw $exn) catch $exn (try (rethrow 1) catch $exn instr* end) end reduces to instr*, as does try (throw $exn) catch_all (try (rethrow 1) catch $exn instr* end) end. The latter is not expressible with unwind.

According to the formalisation in #143, and the equivalence @tlively noted, both try (throw $exn) catch_all (try (rethrow 1) catch $exn instr* end) end and try (throw $exn) unwind (try (rethrow 1) catch $exn instr* end) (br 0) end reduce to instr* (or are equivalent to instr*, depending on whether instr* throws or not). This is because try bt instr1* unwind instr2* end reduces to try bt instr1* catch_all instr2* (rethrow 0) end.

To get back to this point, if we are making a decision between two constructs, it would be good to discuss a comparison between the two. Above I have given a number of reasons why to prefer unwind over catch_all/rethrow. It would help advance the conversation to be given reasons why to prefer catch_all/rethrow over unwind.

Reasons to keep catch_all have be given in several comments. Perhaps you meant "reasons that you find valid".

AFAICT it's not a matter of preference, but a matter of when unwind should be added (now or in a future 2PEH extension). I don't think I understand what is the argument against the reasons given in this comment by @rossberg :

My take on try-unwind as described right now (please correct me if I'm wrong) is the following:

1. (try A unwind B) is equivalent to (try A catch_all B rethrow) (and probably implemented like that).
2. If this equivalence holds now, then we cannot break it later.
3. Consequently, try-unwind is operationally redundant right now, and will remain so in the future.
4. Consequently, it can neither be useful nor required for later extensions.