Reviving proper tail calls?

[pygy]

Hi all,

as much as some people love to hate Safari ("the new IE", yadda, yadda), I've yet to see a complain about their support for (implicit) proper tail calls in the wild. The debugging problems mentioned in this article seem to be non-issues in practice.

Maybe it is time to revisit the situation?

[Pauan]

@concavelenz PTC, however, can be dynamic you have to record something to create a stack trace and that is the problem.

Yes, that's called a "shadow stack" (the link describes Safari/Webkit's implementation of PTC):

It turns out that there is at least one tail call implementation that doesn't suffer from this problem. It implements proper tail calls in the sense that you won't run out of memory by tail-looping. It also has the power to show you tail-deleted frames in a backtrace, so long as you haven't yet run out of memory.

[...]

But we can do something almost like CHICKEN on a shadow stack. It's a common trick to have a VM maintain two stacks - the actual execution stack plus a shadow stack that has some extra information. The shadow stack can be reshaped, moved, etc, since the VM tightly controls its layout. The main stack can then continue to obey ABI rules.

It's what @getify was talking about. This is a common technique which has been used in many VMs for decades. There are well-known solutions. This is not some new or experimental thing.

[pygy]

Reading the ChakraCore issue, the problem does look intricate. One of the issues comes from the fact that they use C++ exceptions to propagate JS errors, which binds them to the Windows ABI. On x64, it prevents one from messing with the stack pointer.

"use strict"
function foo()
{ 
  bar(1);
}
function bar()
{
  return foobar(1,2,3,4); //tail call
}

IIUC, in the case above, with the current ChakraCore impl, the bar() frame can't be grown to accomodate the extra parameters passed to foobar. A new stack frame must be pushed.

It wouldn't matter for most real world code (the stack would stop growing once the function with the greater number of params in the set of mutually recursive function is called and the amortized cost would still be O(1) relative to the number of tail calls), but one could craft pathological cases using eval-like constructs, or even apply() (not sure how PTC and apply() are meant to interact) where the number of params keep on increasing and the stack grows in a O(n) fashion where n is the number of tail calls.

[concavelenz]

@Pauan The "shadow stack" is only used in Safari's devtools (please correct me if I wrong about this) because it has significant performance impact to maintaining otherwise. You can get an accurate-ish stack or good performance with PTC but not both. Without PTC, you get good performance and a good stack.

[Pauan]

@concavelenz I'm obviously not an expert on the Webkit code, but I took a glance over at their shadow stack implementation, and as far as I can see it is always enabled. However, that is an old patch, so maybe things have changed since then.

But I suspect they do have it always enabled, even with the devtools closed, otherwise Error.prototype.stack would break (which means we would have heard complaints about bad debugging on Safari).

It would be good to have some Safari/Webkit devs confirm whether it's always enabled or only selectively enabled.

As for the performance of shadow stacks, any claims about significant slowdown should be backed up by benchmarks proving that statement. Clearly Safari/Webkit isn't suffering from poor performance.

[PinkaminaDianePie]

According to http://gs.statcounter.com Safari market share is about 15%. Have you seen a lot of bug reports from lib/framework developers? Website developers who collect stack traces? Do 15% of users have "broken web" experience? I can't actually decide for myself if it's funny or annoying to hear lots of times that PTC will break the web - 15% of users have PTC for months (or even years, I don't remember when it was shipped) and everything works for them. There is a way to implement PTC without performance loss and without "breaking the web", so it's not an excuse to refuse the standard. If browser vendors will do what they want and refuse standard even when it's clear that it can be implemented, we will return to IE times, when the web was broken just because every browser had its own set of features.

[concavelenz]

(1) To be clear PTC only applies to strict mode code, it would break arguments.caller in non-strict code. A lot of the web today doesn't run 'strict'. So saying 15% is an great overestimate. PTC doesn't "break the web" (pages still render) but it does make maintaining complex application harder and for folks running servers on Node having lousy stack traces is a real concern. I personally remember when it was impossible to get stack traces and it wasn't pleasant.

(2) There is no reason to guess about the Safari behavior, it is simple to try out. If you do, it is easy to see the broken stacks with this snippet in Safari:

function f(a) {
  'use strict'
  if (a > 10) throw new Error();
  return f(a + 1);
}

try {
  f(0);
} catch (e) {
  alert('value = ' + e.stack)
}

You only see only the last call to f. If you remove 'use strict' and are not in a strict context (not in an ES6 class or es6 module or otherwise in a 'use strict' file, you will see the entire chain of calls leading to the exception.

(3) There were many changes to the ES2015 spec after it was released based on the feedback from the browser vendors and other implementors. The initial version had many problems, and they had to be resolved to create a compatible web. Saying "it is in the spec, you must do it" seems a weak argument to me when there are clear and reasonable objections.

(4) The spec is silent about the effects on the Error 'stack' property because it has never been standardized but it is critical to the code health of the ecosystem that it exists. This to me is the key problem with how PTC was added.

[elibarzilay]

@concavelenz: Your two point in (1) are greatly exaggerated. First, it is the market share of Safari (15% or whatever it is): if only 50% of "the web" is in non-strict mode, then it's half that, but it's also half the problem in Chrome. (In other words, the distribution of non-strict sites does not depend on the agent used.)

Second, I keep being amazed at these claims in the spirit of "lousy stack traces". This is really nowhere near the actual damage to developing that you get from no stack traces at all.

I'm probably repeating myself way too much, but here are a few points:

1. You only lose tail calls that were eliminated --- these are in most cases part of some kind of a loop, and therefore having one frame is as good as having many. Your example is an instance of that.

2. You currently lose the whole stack whenever you delay some code using a timer or something similar, yet this doesn't seem to be a deterrent for doing so. (E.g., I've seen many intro JS texts advocating setTimeout and friends, and none of them had some big warning saying "note that by doing so you lose stack information so development will be harder".)

3. Another thing to consider here are profiler "flame graphs". I argue that these graphs will become better, since you will see more information immediately instead of indirectly. For example, consider A (non-tail) calling B, which does some work and (tail) calls itself 3 times. You will see one of these:

          {--B--}        
      {----B----}        

   {------B------} {-B-}{-B-}{-B-} {------A------} {------A------}

   I argue that the right visualization is more helpful. If A tail calls B, then it will be on the same level as the Bs and its length would be proportional to the actual work done in A --- which is, again, more useful.

4. Practically the main objection is the Guido-style rejection based on losing information, to translate that example:

   if (somethingHoldsFor(x)) return some_call(z); else return 42;

   But the flip side of this is that x is expensive (lots of memory, open file handle, etc): this is exactly the kind of things that tail call elimination gets you. (I have actually seen at least one place in Real Code where a similar problem lead to a weird-looking x = null before the call.)

5. Finally, and most importantly, JS developers have already "discovered" the functional style, and they are using it heavily. I have seen multiple places where there's some use of a modern promise-based API that implements a "loop" which would have been a proper loop if only tail calls were mandated. Instead, such code should be converted to a form that uses some while loop and that is far from a natural or easy thing to do, and it is definitely something that makes "maintaining complex application harder".

[kaizhu256]

@elibarzilay, asynchronous recursion using callbacks/promises/generators/etc is O(1) with the [single] caller-frame discarded after the process-tick. PTC is irrelevant in this case.

[kaizhu256]

tbh, i'm slightly anti-PTC (but not strongly), because when troubleshooting hard-to-diagnose integration-level bugs, i'm paranoid about magic messing with stack-traces and screwing-up telemetry (regardless of assurances otherwise).

PTC doesn't add value to my personal use-cases for recursion which are exclusively either O(1) (async), or O(log(n)) (depth-first tree-traversals).

for O(n**constant), my personal preference is to use traditional loops.

for O(2**n), most use-cases are not proper-tail-calls anyway, so PTC doesn't apply.

i find jslint's PTC-based lexer slightly harder to reason with and debug, than if it had stayed with more traditional looping. its not a deal-breaker, and i would be happy with either PTC or non-PTC resolution to its current issues.

[concavelenz]

@elibarzilay RE: Strict mode My point is that measuring the impact, you need to be aware of whether the code is strict and you should expect that the percentage of impacted code will only increase over time as folks ship more and more ES2015 code native (without transpiling ES6 class and ES6 module). If you don't know how much of your code is strict, you don't know how much this could impact you. Saying Safari has 15% of the market isn't useful for measuring impact.

RE: 1. You don't only lose frames that were part of a loop. You lose all calls that are in the return position. That is the difference between PTC and TCO. PTC says all such call frame must be eliminated.

RE: 2. Timers etc. It has always been this way, and there are ways to record context for these, so those that need this information have built solutions. PTC eliminates frames for existing code, with no way to distinguish between code that requires PTC to operator properly and code that does not, so there is no trivial way to transform code to preserve stack traces. (var a = f(); return a;)

RE: Profiler Generally, sampling profilers rely on accurate call stacks. With PTC that information is destroyed, you must create a shadow stack in order to get accurate profiling information. That changes the profiler from being purely "sampling" to something that introduces additional overhead (the general drawback of an instrumented profile). Here don't get the benefit you describe. If you did you wouldn't be able to distinguish these call chains (assuming all calls were in the tail position):

f -> g -> h -> i f -> x -> y -> i

RE: value lifetime The VM doesn't need to preserve anything it isn't going to use after the call. PTC isn't required for this. In most cases this is handle without intervention by the VMs register allocator. In my experience, you would only need to set "x = null" due to closures and PTC doesn't help you at all there as the context still need to be preserved even if the stack frame is eliminated.

Lastly, for my information, what is what is "Guido-style rejection" ?

[getify]

PTC says all such call frame must be eliminated.

That is not actually what's required. Here's the relevant part of the spec:

A tail position call must either release any transient internal resources associated with the currently executing function execution context before invoking the target function or reuse those resources in support of the target function.

First, this doesn't require elimination of a frame. It only talks about freeing up resources. That's a subtle but important difference, which implementations like Safari seem to have taken to allow things like the Shadow Stack.

But further, I maintain that rather than being concerned about the letter of the law, the spirit of the law is the most important thing here. What the spec is getting at is less that there must strictly not be any extra resource allocation, but rather that the allocation growth must be O(1) instead of O(n). I would like TC39 to reword this section in this respect, as it would clear the way for lots of other creative ways of accomplishing the bigger goal of tail-calls, which is that I could make any depth of call stack in my program without fear of running my device out of memory.

That's why people want tail calls, not that they explicitly want every single non-essential stack frame thrown away.

[kaizhu256]

Lastly, for my information, what is what is "Guido-style rejection" ?

python's creator [in]famously rejected TCO in a 2006 blog-post [1]:

"This is also the reason why Python will never have continuations, and even why I'm uninterested in optimizing tail recursion."

it caused some brouhaha back in the day (similar to what's going on in this thread) [2].

[1] original-quote from python-creator's 2006 blog-post "Language Design Is Not Just Solving Puzzles" https://www.artima.com/weblogs/viewpost.jsp?thread=147358

[2] backlash to [1] on [Python-Dev] mailing-list https://mail.python.org/pipermail/python-dev/2006-May/064817.html

[3] python-creator's response to backlash (with further community "feedback" in the comments-section) http://neopythonic.blogspot.com/2009/04/tail-recursion-elimination.html

[elibarzilay]

@kaizhu256: it might be that some promises are forced in async-ly in a different tick and therefore there is no problem with tail calls, but that might not always be the case. For example, this code (or some similar variant):

const loop = n =>
  new Promise((res,rej) => res((n > 0) ? loop(n-1) : Promise.resolve(n)));

should be a loop even though there's no such deferring. But more generally, in the simple case of

const loop = n => foo(() => loop(n-1));

I'm relying on foo to call my callback in a tail position, and if syntactic tail calls are a thing, that means that I now need to ask foos author nicely to fix the code.

@concavelenz: (1) yes, that was the point I made later: losing bindings from tail-calles is the feature of reducing resource use; (2) if there are "ways to record context", then wonderful -- tail calls could use the same tools.

For (3), quite the opposite. The statistical sampling fits perfectly with tail calls, if f does almost nothing before tail calling g it will not be shown -- at this point the developer is concerned about runtime and g (the main focus) is still just as visible. If, OTOH, f does a non-tail-call to g (or of no tail call elimination is done), then you get what you have today: you would have an f block and on top of that a g block that is the same width, and you need to infer from that that f is noise and the real runtime hit is from g.

Finally, in those let a = f(); return foo(); cases the runtime does need keep a, because it might be used in a stack trace. A tail call means that it can be GCed.

@concavelenz / @kaizhu256: yes, it's that mess that I'm referring to, but IIRC the relevant post had some "final" in its title and that's when Guido dropped a concrete lid on it.

[concavelenz]

@getify I fully understand why folks want tail recursion. The text requires that everything be reused or released, nothing remains of the stack frame. The implementation of the shadow stack has nothing to do with that text.

@elibarzilay RE: (2) recording the context for every function that has a call in the tail position is impractical, as it involves recording the stack trace, which is expensive. Doing this for timer, which are far rarer, is sometimes reasonable.

RE: (3) You imply that a you only care about the runtime of a function, but in my experience how you reach the function (why it was invoked at all) is usually as important that the cost of the function proper.

[elibarzilay]

Well, you brought it up... It can be expensive, but you have mentioned the optional shadow stack thing...

And in that context I obviously cared only about runtime, since I was talking about one common concern about the usability of flame graphs. (And further, in a statistical profiler context you generally don't care about precise information which is why the random stack polling works fine.) The "why it was invoked" can be addressed using tools that were mentioned many times in this thread; tools that can be used for the sake of skeptics who don't believe functional programmers who say that in practice this is not a problem. (Discussing why it's not a problem in practice will further derail this thread so I'll avoid doing that.)

[hax]

This thread is very unfortunate that nothing we (programmers) can do except waiting. But I still want to share my thought about PTC vs STC.

1. I agree that any action is better than no action.
2. I think tail call is important for FP, without it, JS can never have real support FP even we add many FP features like pipeline operator.
3. I prefer STC. The problem of PTC is, it's too implicit. If someone write a code rely on PTC, it could be easily ruined by others because we don't know whether a tail call is intentionally.

Example:

function f() {
  if (x > max) return
  ++x
  return f()
}

This is a valid PTC code. Consider if we introduce consistent-return rule, ESLint will complain. How to fix? Someone will just change return f() to f(), and break PTC, and may introduce a potential bug which normal test cases can never find --- even you have a test case for it, you need a correct max to trigger stack overflow, it should be big enough, but not too big (for speed of test running, especially if you run tests on low-end mobile devices). You may need config different max for different user agent, and new versions of them could just increase their stack and fail you...

[sbuller]

'consistent-return' seems a little misguided here. Proper type checking reveals that the types remain consistent in your example.

I regularly notice when my code would benefit from TCO, and it's disappointing to know that there's this misguided resistance to it. There's a long history to TCO, and its absence in JS is a serious deficiency.

[pygy]

What more, the technical reason for not adding them (the ChakraCore engine having the wrong calling conventions hardwired) is soon going away since MS is co-opting Chromium.

The other technical problem isn't really one (cross-realm calls can't be eliminated in Firefox) since that can be spec'ed around.

[jswalden]

Last I heard, ChakraCore will continue to exist as a JS engine for use in any number of projects not including a browser, so that point doesn't quite work out that way.

[sarimarton]

Once PTC starts to spread in practice due to its actual availability, and it eventually becomes a problem to identify TCs in code, IDEs will very quickly kick in with syntax highlight help - just as they do with dead code by fading it.

[hax]

@sarimarton Not everyone in a team use same IDE/editor. And lightweight editors, tools may not identify TCs easily without full AST parsed.

[kaizhu256]

i write code exclusively in vim (and use a customized version of jslint that allows ignoring foreign code-blocks)

[littledan]

I have a feeling that we might be able to make more progress on explicit tail call syntax in JavaScript. If everyone able to think through and agree on a syntax, I think it'd be worth bringing back to TC39 to see if we can get consensus on pursuing this proposal.

[elibarzilay]

@sarimarton, yes -- identifying tail calls is in most cases extremely simple, almost to the point of regexp-ing it. With a parser it becomes straightforward.

@littledan, the problem is not in agreeing on a syntax -- the real problem is the fact that "opt-in" syntax is going to nullify much of the point of tail calls. See the last bullet in my original post above.

[littledan]

Well, if we can't agree on a syntax, I imagine the current situation will remain as is.

[elibarzilay]

@littledan, please see that comment that I pointed to: the problem is not the syntax, it's the fact that you'll need to modify code to "opt into" a tail call, which would lead to situations like what I describe there. (IOW, this is an objection to any explicit syntax for tail calls.)

[ljharb]

I'm still not clear on why that's an obstacle - libraries needing to change their code is a temporary problem, as new code would theoretically be written with the explicit syntax where desired.

[elibarzilay]

I'll try to explain it with an actual (but very simple) example.

• I implement a useful text-lines library, which has:

  // calls f on a list of lines from the file
  function withLines(file, f) {
  let lines = file2lines(file);
  return f(lines);
  }

• You use this libray in:

  function waitUntilItsOK() {
  withLines("some-file", lines => isOK(lines) || continue waitUntilItsOK());
  }

• Your code is perfectly fine, except that it's broken because my code doesn't use continue. You'll ask me nicely, but I don't want to complicate my code with strange keywords, or I need the code to work on older JS engines, or my company won't allow me to invest time to change code that already works, or I don't like this new "PTC" feature, or I heard that it code run slower, or whatever. You're now stuck.

In a theoretical world where none of these reasons hold, everyone would eventually add continue at all places that matter — but then what was the point of adding an extra keyword in the first place?

The only "real" answer to that is to make it easier to verify that what you think is a tail call, is ineed one. IMO, this makes exactly the same sense that the "capture clause" in C++ lambdas does. And for all I know, you might like that feature (maybe because you're doing C++ at google). But like I said previously, in the current non-theoretical world, the much likelier result is that tail calls will not be widely used (if only because of the natural bias against longer code).

[ljharb]

You could also imagine the syntax enabling proper tail calls for the entire stack it generates - avoiding the need to add the keyword all the way down.

[elibarzilay]

That sounds interesting, but it's not what's suggested in this proposal...

(Sidenote: I imagine that you're talking about something that happens at the beginning of the stack since going the other way is impossible. Something that I thought could work is some "use strict and tail" declaration where all code under it creates a dynamic stack context that makes all later calls be tail calls. If that's what you're talking about then that sounds like great idea. Again, regardless of actual syntax: I don't really care if it's a top "use strict and tail" or some (()=> continue { ... }) wrapper around my whole file.)

[PinkaminaDianePie]

Enabling the entire stack it generates? Why not go other way and have PTC but provide unfolded stack trace when required? So my app will have PTC, but if i'll struggle with something, I will be able to enforce unfolded stack trace, debug my app, and disable it back again? But having ETC keyword which enables PTC for stack trace is fine enough if I will be able to do it once at the root of my app (e.g. in index.js) and work with PTC everywhere. Anyway, explicit TC, which I will be able to use only once at the top of app will be good thing, but have it explicit everywhere will be much much worse

[getify]

I prefer "implicit PTC" (current spec) but with revised wording that would widen the possibilities for various engine implementations, as discussed earlier in this thread.

However, it seems almost impossible for that to ever happen from the current state. The years-long stalemate is bad for JS and for engines.

As for whether TC39 could remove PTC without consensus, I have changed my mind from earlier positions: I now strongly feel that TC39 can, and should, remove PTC from the spec. This needs to happen first before any possible progress on tail calls can happen.

The webkit folks don't have standing to object to the removal, as this doesn't betray or invalidate their implementation at all. Had PTC never been added, I believe webkit could have added this "feature" as an optimization without being in spec violation. So even if the spec part is removed, that doesn't harm webkit.

I believe the editor of the spec should make an executive decision that any spec'd feature (or typo or bug or wording or ..) which is later openly defied by implementation(s) -- irreconcilably so -- is a post facto veto, and therefore can and must be removed, retroactively stricken from the standard.

I imagine there may be some "legal" objection (ECMA, etc) to that, so in effect the next best thing is just to inline the change in the next spec revision without need for consensus.

In any case, we need to do a reset on this topic so we can then have productive discussion about any possible path forward.

[ljharb]

That is not within the power of an editor - every delegate has standing to block anything for any reason, including removing PTC from the spec. Consensus is required for normative changes, which is why we’re in the stalemate we’re now in.

I agree that no progress can be made until PTC is removed.

[fselcukcan]

I just do not understand why we cannot have an important programming feature like TCO (not the syntactical proposal, but natural as it supposed to be) due to some claims that it has negative effects on somethings which does not have much ground as we have seen in different threads so far.

• debugging
• education?. Gee, Are you serious!? Ok sorry let me rephrase that. Gee, are you serious!?
• what is next, health?

Shortly, for debugging I can say; We debug code when we debug code, not when we run code. Is it not possible to have a switch like Disable TCO (when Developer Tools are open, that is when we debug)? We have this for disabling caches Disable Cache (when Developer Tools are open, that is when we debug). Disable Tail Call Optimizations (When Developer Tools are open) #25

[mlhaufe]

Gilad Bracha has shared some advice on how to have your cake and eat it too with PTC. Stack Compression, Heap Allocation, Ring Buffers, etc.

https://gbracha.blogspot.com/2009/12/chased-by-ones-own-tail.html

[douglascrockford]

TC39 made a mistake in naming the feature 'Proper Tail Calls'. Lots of coders have no interest in propriety. It should have been called 'Tail Call Optimization'. Coders will jump through all sorts of hoops to optimize, even when that work has no observable benefit.

The only change TC39 should make is to correct the name of the feature. Meanwhile, all JavaScript engines should implement it in order to be called Standards Compliant, and debuggers should be enhanced to mitigate the debugging experience.

[ljharb]

Standards document reality, they don’t dictate it, and the proper thing to do would be to remove PTC from the spec, and attempt to reintroduce it through the modern proposal process.

[getify]

the proper thing to do would be to remove PTC from the spec

I asserted years ago that TC39 could/should adopt the position that any feature in the spec which is willfully violated by the majority of implementations is de facto already voted down, and should thus be removed/demoted (back to stage 2 or 3) as an editorial change, requiring no further consensus vote.

Even if webkit still wants to keep its PTC implementation, they should be allowed to do, because there's no current assertion being made that the spec would include a prohibition on PTC. It could/should just be an experimental feature that webkit implemented that distinguishes them from the other engines. Therefore, it doesn't require a consensus vote to remove the specification of the feature.

If PTC/TCO was demoted to stage 2 (or even 1), it would (as @ljharb suggests) be given the time to go through a more rigorous exploration and specification cycle. If it ever reached stage 4 again, it would certainly have done so after convincing all of its detractors, and we'd be much more likely to get actual implementations of it (beyond webkit). Of course, webkit could still veto any substantial design changes (such as STC) while it was in those earlier stages. But that should be a healthy debate process that's allowed to happen, instead of the stalemate we currently have.

---

Side note 1: I would feel differently if two or more implementations had a current shipping implementation... in that case, I would argue that TC39 must at a minimum move the feature specification into appendix B, in the spirit of documenting web reality even if the specification doesn't necessarily endorse some feature or behavior as a first-class citizen.

Side note 2: I still absolutely strongly feel JS should have PTC (or TCO -- yes, I'm agreeing with Doug). My assertion above should not be construed as being against the feature, but rather against the (IMO) unacceptable indefinite standoff and willful violation by most engines.

[elibarzilay]

1. There is no point talking about "majority of implementations." It's very clearly a term that could freely be substituted with "Google" -- and seen in this light, leaving anything for the majority is just serving at some idealistic "community forces that shape the language" when in fact it's a single-company dictatorship, and what you suggest should be an obviously bad idea to most sane people.

   (That "indefinite standoff" was almost resolved when this exact dictator was going forward with an implementation, and then decided to drop it for reasons that are not visible to outsiders. So yeah, there was no discussion on the merits of the feature, just a bunch of the usual misguided FUD as a weak explanation --- and no way to properly comment about or discuss that conclusion. This further demonstrates why it's nonsensical to leave it for "the majority" when that majority is one company.)

2. There is also no point in trying to make it into some kind of a symmetrical "both sides" argument WRT enforcing no tail calls. How would such an enforcement look like? Any way of defining this would require going down to implementation levels which doesn't make sense for a language spec. For example, "must maintain a stack", "must throw a stack overflow error after N calls" are both nonsensical for a spec.

3. Re the name, "tail call optimization" is a bad term that makes pretty much anyone confused about the motivation and the expected results. Example: way too many people hear "optimization" and immediately conclude "runs faster", and when they learn that it doesn't, they see no point in it. As usual, an "optimization" is not a reliable language feature but an opportunistic way of running your code in some better-than-expected-way, and if it's beyond the scope of the compiler then you can't really complain. "Proper tail calls" (or "tail call elimination") are a language feature that allows writing code in a way that could not be done without it: IOW, you write code that depends on this feature, and therefore it is not "an optimization".

4. Finally, re "it would be given the time to go through a more rigorous exploration and specification cycle" -- that's bogus in this context too. The feature itself is very well-researched, well-used, and overall well known. It had time to "mature" since the 70s. Everything that could be explored around it had been explored, many times in many contexts. As productive as TC39 could ever dream of being, it pales in comparison to decades of actual research and actual use, so it's only logical for them to defer to existing work instead of "exploration". That holds also for specification: the actual spec of tail calls is relatively small, and not too complicated.

   What keeps astonishing me is the one thing to take from the fact that there is one "relatively known" browser that has had tail calls for a while now, and that hasn't lead to the total disaster of destroying debuggability which is at the core of the anti-PTC crowd -- and in fact was the very same reason given by that company when they dropped the work that they already did toward an implementation (though the droppage predates the webkit implementation).

   Javascript has had a major surge in functional-style work in the last decade, and it's about time that people will be more vocal about getting the last bit that makes it possible to use functional style over being forced into while loops.

   (And perhaps a slightly more anecdotal point: one of the main problems with syntactic tail calls is that they abolish a substantial part of the benefits. One of the major points of PTC is that it "just works" even through a HO function from an independent library. Suggesting a syntax for tail calls (especially one as obnoxiously verbose as return continue is guaranteed to keep people away from the feature, which means that the benefits would be only the ones that are limited to your own code. And as long as you care about just your own code, then switching your healthy thought process into the sick world of stateful loops and such is something that is doable -- and effectively enforced by the dictatorship (since nobody sane would require a non-v8 runtime). And so the wheel spins year after year with no light on the horizon.

   The bottom line here is that it took decades to get to this point: dropping the requirement to some "in the future" appendix is going to take it further back.)