backcalls: let's add them

[michaelficarra]

Copied proposal from https://github.com/jashkenas/coffee-script/issues/2662#issuecomment-13026081 below

---

Taken pretty much from the coffee-of-my-dreams list, the best way to explain my preferred backcall syntax is through example:

• The continuation is captured and tacked on as the final arg by default

  <- fn a
  ...
  b
  

  compiles to

  fn(a, function(){
    ...;
    return b;
  });
  

• The left side of the arrow specifies the argument list for the continuation. When only one is given, parens can be omitted.

  a <- fn b
  ...
  c
  

  compiles to

  fn(b, function(a){
    ...;
    return c;
  });
  

• And for the rare occasion where the continuation is not the final argument of the function, we need to be able to specify where to put it with a marker like the <&> I use below. I'm open to suggestions for a better marker we can simply use an anonymous function to force the position of the callback.

  (a, b) <- (_) -> fn c, _, d
  ...
  e
  

  compiles to

  (function(_){ return fn(c, _, d); })(function(a, b){
    ...;
    return e;
  })
  

  or, if we detect this case, the simpler:

  fn(c, function(a, b){
    ...;
    return e;
  }, d);
  

this must be preserved in the continuation by rewriting references as we currently do in bound functions. I don't think we should worry about arguments rewriting, but that's your call.

Here's a real-world example @paulmillr gave in #1942:

compile = (files, callback) ->
  async.sortBy files, @sort, (error, sorted) =>
    return @logError error if error?
    async.map sorted, @map, (error, mapped) =>
      return @logError error if error?
      async.reduce mapped, null, @reduce, (error, reduced) =>
        return @logError error if error?
        @write reduced, (error) =>
          return @logError error if error?
          do @log
          callback? @getClassName()

and here it is using backcalls:

compile = (files, callback) ->
  (error, sorted) <- async.sortBy files, @sort
  return @logError error if error?
  (error, mapped) <- async.map sorted, @map
  return @logError error if error?
  (error, reduced) <- async.reduce mapped, null, @reduce
  return @logError error if error?
  error <- @write reduced
  return @logError error if error?
  do @log
  callback? @getClassName()

edit: dropped the callback position indicator, opting for "hugs" style

[jashkenas]

It's unfortunate that there won't be parity between => and <-, but I don't think that's so bad.

I think it's fairly bad. Again, imagine trying to explain to a beginner, ->, <- and =>. All different syntaxes for writing slightly different versions of a "function". If <- is semantically kin to =>, but syntactically an inversion of a normal function callback, it gets pretty confusing.

As a small anecdote -- I tried to explain this proposed feature (in all it's glory) to some JS developers last night on the back of a cocktail napkin, and it didn't really translate well, especially with the subtleties.

[00dani]

Just to throw another spanner into the works, how do backcalls work in conjunction with promises? Since promises are monadic and backcalls are do notation, they really should work well together, but as far as I can tell a promise do block has to be written "explicitly" like this:

y <- (f x).then
z <- (g y).then
h z

Is there any chance of cleaning that up without crippling backcalls' other applications? (Obviously we could make backcalls y <- x desugar to x.then (y) ->, rather than x (y) ->, but that would render the syntax pretty much useless in non-promises situations.)

[jamesonquinn]

@00Davo : I think that you'd probably write a library function to make it look better:

after <- After f, x
after <- after g
after.final h

... or something of the sort. But in general, I think it's OK if promises and backcalls are largely separate solutions to similar (but not identical) problems.

Afterthought: a library like this could do a similar thing as promises do in terms of passing error handling to the next available handler.

[jamesonquinn]

@jashkenas : how do you feel about ~> and <~?

Also, how would the thread in general feel about using the same symbol as a placeholder? I mean:

(a, b) <- fn c, <-, d

or maybe

(a, b) <~ fn c, ~, d

I'm sympathetic with the pro-hugs argument (simple syntax) but if jashkenas favors the pro-placeholder side (easier to read and learn) then that's fine too.

[jamesonquinn]

New idea: what about =< ? It's not as pretty as <= would be, and "hugs" aren't as cute looking, but it does clearly signal that this is bound syntax.

[michaelficarra]

@jamesonquinn: I like =< a lot. I still think we should do without a placeholder syntax for now. It can always be added in later.

[jashkenas]

So, in order to make some progress on this ... personally, I'm still not quite entirely sold, but am very much sitting on the fence. If someone want's to cook one up, I'd love to see two linked pull requests.

1. Implement Backcalls along the lines of Michael's description, sans placeholders, with some sort of rationale and solution for "bound backcalls".
2. A separate pull request that attempts to implement all of the compiler's asynchronous bits in the most idiomatic way, using the new feature.

Either way, with a merge or without, I think that would be instructive.

[epidemian]

I'm really not sold on the idea of backcalls yet. I think it complicates the syntax and adds new corner-cases for not much gain.

I think the problem with the (in)famous JS Callback Hell is not so much the indentation per se, but the complexity of having all those different scopes and possibly delayed executions intermixed. I think the proposed backcall syntax does not solve the problem of the complexity of callbacks, it just makes them look like sequential code (when its execution is not necessarily sequential), but the complexity is still there: lots of different function scopes mixed-up, returns and exceptions not working as in sequential code, etc.

The most compelling use-case for backcalls i've seen is removing some of the tedious boilerplate and gratuitous indentation AMD modules or similar constructs introduce (e.g. jQuery.ready). Maybe for those cases we could introduce a simpler construct, like a different kind of function token, like a long arrow -->, that encloses everything after it without needing an extra level of indentation.

If backcalls get implemented, i hope they ends up being a very simple transformation as @michaelficarra proposes (no placeholder syntax or anything... the "hug" idea was a joke; i wouldn't like seeing it become an indiom xD). But i fear it'll still be unclear when they should be used and where normal callbacks should be used instead.

[jashkenas]

You make fine points. I more or less agree -- one little clarification though, in backcalls' defense...

it just makes them look like sequential code (when its execution is not necessarily sequential)

... the point of the backcalls is that they are the serial transformation of the continuations, not any parallel version. If you write a series of backcalls in a single block of code (no indentations) ... then every line within that block will be executed sequentially ... just not necessarily right away. That's part of the reason why they make some sense.

But your points about return and exceptions are quite right.

we could introduce a simpler construct, like a different kind of function token, like a long arrow

... This is that different kind of function token. That's all that this is.

[jamesonquinn]

I think that the simplest use cases are nice, but not compellingly necessary. The reason I think this language feature is important is that it would enable organic growth of powerful tools. My iced-cs-as-a-library and backcall-style-promises-library ideas, mentioned above, are only two of the many possibilities. Iced-cs features are powerful (read the iced-coffeescript docs for why: http://maxtaco.github.io/coffee-script/ ) and to me, backcalls are by far the cleanest way to enable that style of programming in coffeescript. (In fact, that's what led me to this issue in the first place.)

I am confident that if this were enabled, we'd soon have libraries to enable tricks we don't currently imagine. Some of these would turn out to be dead ends, but some could catch on.

Let a thousand flowers bloom.

[jashkenas]

Let a thousand flowers bloom.

Indeed, and absolutely. A thousand flowers and a thousand forks (of which we're currently at nearly 900). That said...

I am confident that if [backcalls] were enabled, we'd soon have libraries to enable tricks we don't currently imagine.

I don't think so. Backcalls are actually a somewhat restrictive way to add a flavor of syntax that favors a particular asynchronous style -- e.g. serial-continuation-as-the-final-callback-to-the-function. They actually make it less easy and less likely for other styles (icey await and defer, promises of various stripes) to become successful within CoffeeScript.

[jamesonquinn]

As to "tricks we don't currently imagine" I'm thinking in terms of

subtrick? =< trick fun, arg1, arg2

where "trick" is some kind of "meta-function" or "decorator" for fun arg1, arg2. Three ideas I can already imagine for "trick":

1. iced-like parallel callback bundling: like "await", this would be something that could easily refactor to a serial version by just switching places between it and "for".
2. promise-like error-handling: some way to declare a single error handler for a whole chain of callbacks.
3. placeholders to ensure the callback comes last.

But I suspect if I can come up with 3 such ideas off the top of my head, that further possibilities exist. I also suspect it would take a bit of experimentation get an implementation for any of the above ideas (or the combination of them) really "right".

[paulmillr]

I don’t think backcalls are worth it.

Node will switch soon to yield, it is already in v8 and actually works (people started creating modules for it).

Browser async code is less a mess.

We have promises until yield which solve all callback problems and provide returns and throws.

Yield code:

var sorted = yield sort(files, sorter);
var mapped = yield async.map(sorted, this.map);
var reduced = yield async.reduce(mapped, null, reduce);
yield thiswrite(reduced);

Promises code:

async.sortBy(files, sort)
  .then (sorted) =>
    async.map sorted, @map
  .then (mapped) =>
    async.reduce mapped, null, reduce
  .then (reduced) =>
    @write reduced

backcalls code (looks shitty):

compile = (files, callback) ->
  (error, sorted) <- async.sortBy _, files, @sort
  return @logError error if error?
  (error, mapped) <- async.map _, sorted, @map
  return @logError error if error?
  (error, reduced) <- async.reduce _, mapped, null, @reduce
  return @logError error if error?
  error <- @write _, reduced
  return @logError error if error?
  do @log
  callback? @getClassName()

[jamesonquinn]

@paulmillr : You've left the error handling out of the promise and yield code, which isn't very fair, even if you're right that it's cleaner than the naive backcall code you gave. But my point is that libraries would develop so that the backcalls code would be prettier. Something roughly like:

compile = (files, callback) ->
  seq = new Sequence
    errorHandler: (err) -> @logError err
  (seq, sorted) =< seq async.sortBy, seq, files, @sort
  (seq, mapped) =< seq async.map, seq, sorted, @map
  (seq, reduced) =< seq async.reduce, seq, mapped, null, @reduce
  seq =< seq, @write, seq, reduced
  do @log
  callback? @getClassName()

The above is a very rough indication of the kind of thing you could do. For instance, "Sequence" is the first name that occurred to me; I'm pretty sure you could find better words for that class and object. The Sequence class should be written so that the above works exactly like the code you wrote. Notice that in the above, a Sequence instance is assumed to have both placeholder and error-passing functionality.

[paulmillr]

@jamesonquinn it is absolutely fair -- 98% of the time you don’t want to specify error handlers for every single error. Still, with yield it is super-simple.

Sequence is just another package for solving a problem that is more effectively solved with yield or promises.

Yield:

try {
  yield ...
  yield ...
} catch(err) {this.logError(err)}

Promises:

// just last step
.then(@write, @logError)

[jamesonquinn]

My point is that

1. yield doesn't exist today, and won't reliably exist in the browser for years.
2. Promises give one way to solve this problem, but it's possible to imagine that backcalls would give another. If the hypothetical "Sequence" package, whose functionality I dreamed up in under 5 minutes, has warts, that doesn't mean that any possible such package would have them.

... added later:

Here's a quick demo of a bunch of different things backcalls could enable:

katch =< trie (yeeld) ->
  arg3 =< yeeld someAsync, arg1, arg2 #yeeld soaks up err and passes it to katch
  arg5 =< yeeld annoyingArgumentOrderAsync, arg3, yeeld.cb, arg4 #look, placeholders!
  someList =< yeeld.noErr errorFreeAsync, arg5 #when cb isn't `(err, result) ->`
  hurryUp =< yeeld.either (yeeld2) ->  #two sources, we'll go with the fastest
    onePossibleDataSource yeeld2
    anotherDataSource yeeld2
  results =< yeeld.several (defer) ->  #look, iced-style parallelization!
    setTimeout defer.finish, 10000 #global timeout, unlike iced-cs, 
    defer.results = []
    for item, index in someList
      item.asyncGetVal defer (@[index]) ->
        #you could put a conditional defer.finish() or yeeld.throwe() here
  display results
katch (err) -> logError err #errors get sent skipping to here
katch.finaly -> cleanup()
moreStuff()

All of the above is possible with a smart enough "trie" package. As always, the above names are first-pass placeholders, and I'm sure a finished package could improve them.

That's a kitchen-sink vision of what could be; it includes more possible tricks than are available today with promises, yield, and iced combined. And thus it would require roughly 6 different kinds of sub-entities plus 6 special methods on those entities. I'm sure that many would consider that overkill. So perhaps eventually most of the community would settle on a package that did only half of that or less. And even the ultra-purists would be fine; they wouldn't have to use any package at all. They'd still get the power of =< require.

[jamesonquinn]

@jashkenas : "A separate pull request that attempts to implement all of the compiler's asynchronous bits in the most idiomatic way, using the new feature." Can you suggest a good place to start with that? (I'll see if I can make time to do 1 and start on 2 next week.)

[vendethiel]

We don't have a lot of async, so I'd say mainly command.

[jamesonquinn]

Ok, looking at command, you get things like:

watch = (source, base) ->

  compileTimeout = null

  ...

  compile = ->
    clearTimeout compileTimeout
    compileTimeout = wait 25, ->
      ...

If translated into a backcall, would that last line be:

    compileTimeout = =< wait 25
    ...

That is,

1. Should the value of the whole backcall expression be the return value of the async function call?
2. Even if it is, is the above = =< idiomatic?
3. Unrelated but inspired by this problem: what if you had compileTimeout =< wait 25 in the above? The simple backcall transform wouldn't work as expected, because compileTimeout already exists in outer scope. Should the compiler throw an error for cases like this; or should it add code to do what you'd expect, similar to the (@x) -> idiom?

[vendethiel]

= =< ?

[jamesonquinn]

If =< fn a turns into fn a, =>, then b = =< fn a would turn into b = fn a, =>, right? But yes, even if the compiler worked like that, I think that = =< is irredeemably ugly. So while I'm still very much in favor of backcalls, I'm not sure that there's any good place to actually use them in the codebase of coffeescript itself.

[epidemian]

Besides, =< alone looks like a sad smiley =<

[jashkenas]

irredeemably ugly

...

not sure that there's any good place to actually use them in the codebase of coffeescript itself

... if that turns out to the the case, that's a pretty excellent bit of evidence against adding them.

[michaelficarra]

@jashkenas: This compiler has no async interface. It exposes an entirely sync interface and doesn't interact with slow/network/blocking resources. This just isn't a good project to show off a feature like backcalls. I think the minimum requirement would be a library that finds value currently in depending upon caolan/async. At least that library would be passing around continuations.

[jamesonquinn]

@jashkenas : I agree with @michaelficarra that one wouldn't expect a compiler to be the best place to show off this feature. Given that, would either of you like to suggest how I should look for a good demo ground; that is, an existing project, in coffeescript and with a slow/network/blocking component?

[ricardobeat]

yield :crying_cat_face:

yield doesn't exist today, and won't reliably exist in the browser for years.

It's already in node, it has existed in FF for a couple years (though slightly different), and should be coming to webkit/blink in the near future. I'd bet on > 80% support about a year from now.

[jcc8]

Bounty on this issue here: https://www.catincan.com/bounty/backcalls-lets-add-them-issue-2762-jashkenas-coffee-script-github

[zhaizhai]

In case anybody is interested, I started implementing some of the suggested features in the backcall branch here:

https://github.com/zhaizhai/coffee-script

Some simple examples are in test/backcall.coffee. I figure it could be a useful starting point for others interested in exploring this feature.

[tashemi]

I like CoffeeScript because of its minimalism simplicity and clearance. With this construction I have too much questions: a <- fn b fn(b, function (a){}) Why does first argument b of fn go after second? Why does body of callback go after first argument of fn? Why is a an argument of callback? It confuses me a lot. I can not understand this construction after first time reading.

[00dani]

@tashemi:

Why does first argument b of fn go after second?

The arguments go in the order they do because of an existing convention: Most all asynchronous calls in Node.js take the form fn(arg1, arg2, …, argN, callback), so backcalls by default assume that the callback belongs at the end. The rest of the arguments arg1 to argN are not reordered by the backcall transformation.

res <- fn arg1, arg2, …, argN
# equivalent to
fn arg1, arg2, …, argN, (res) ->

Why does body of callback go after first argument of fn?

Well, it would anyway, even in vanilla JavaScript. It's probably more useful to view the body of the callback as going after the entire line a <- fn b. As for the reason it does that…

Why is a an argument of callback?

To make it look more like synchronous code. In fact, that's the entire purpose of backcall transformation, really. The rest of the function (well, the current indent block) is wrapped up in the callback body because that'll work more like sync code, and a is made the callback's argument because that'll work more like sync code, and so on.

Specifically, the arguments to the callback are pushed to the left side by analogy to the standard = operator: <- is meant to be viewed as a magical, asynchronous version of =. For instance:

# synchronous code
a = fn b
console.log a
# asynchronous code
a <- fn b
console.log a

[jamesonquinn]

I agree with 00Davo's response. But wasn't the =< syntax currently preferred over the <- one?

[tashemi]

@00Davo thanks for response. Now it looks more obviously.

[00dani]

I'm not totally sure where we got =< as the symbol for all backcalls. I believe it was suggested as an alternative specifically for bound backcalls, by analogy to =>, which would suggest that the corresponding unbound backcall syntax should be -<.

Both of those (-< and =<) look off to me, however, compared to <-. I'd really prefer <~ for bound backcalls and an analogous ~> for bound callbacks (which seems to be what Coco has already), although I doubt adding new syntax for bound callbacks is an option at this stage.

(It's perhaps worth nothing that the bound backcall =< is now pretty close to Haskell's inverse bind =<<. This… may not be particularly relevant for anything, but I thought it was interesting.)

[vendethiel]

Well, <- exists in haskell as well, doesn't it? :)

[jamesonquinn]

All backcalls are bound. A non-bound backcall would only be an evil gotcha; a sudden and drastic change in context without a corresponding change in indentation.

It would be great if we could use something perfectly symmetric. But <- for backcalls and -> for bound function declarations isn't going to happen because coffeescript is NOT going to radically change the function declaration syntax just for a feature that some might never use. And <= is simply not going to happen for similarly obvious reasons.

So we're choosing between various second-best options, principally the following:

• <-:
  • Pros: Looks like an arrow, similar to function declaration. Leads to cute-looking "hugs" idiom.
  • Cons: Resembles the unbound function declaration, but actually corresponds to a bound function. Yuck.
• =<:
  • Pros: Closer resemblance to bound function declaration than to unbound. Resembles an assignment operator, which helps develop intuition about what's happening here. Suggests the name "fat arrowtail", which helps remind developers that the assumption is that the callback is the tail argument
  • Cons: Leads to an less-pretty-looking "hugs" idiom: =< (_) => setTimeout _, 250 (Sad person, pac man, happy person? The "I need pac man" idiom?)
• <~:
  • Pros: Does not look like <-
  • Cons: Would work best if bound functions were ~>, but that's either a major backwards-incompatible change in removing => (not going to happen), or a confusing and unnecessary TMTOWTDI (there's more than one way...).

My vote is that =< gets a B, <~ a D, and <- an F.¹

¹ Off-topic: Voting works best by giving grades and taking the median grade; it solves most vote-splitting and is relatively simple and non-strategic for voters. Tied medians are resolved by some arbitrary rule such as most votes above median. This voting system is called Majority Approval Voting.

[vendethiel]

Well, that's what I call a detailed answer. I'd like to see "~>", maybe coffee2 ;).

[epidemian]

Well, one benefit of => over ~> is that it aligns pretty well with the new JS arrow function syntax.

I think @tashemi brings a valid concern in that backcalls would involve a not very obvious CS -> JS conversion compared to other constructs, which may be at odds with the "it's just JS" spirit. That being said, the JS generated by the class construct is also quite contrived when involving inheritance and other stuff; and we're all fine with that (probably because the benefits outweigh the drawbacks... IDK).

For the moment, i'd prefer to see Coffee supporting yield generators; and using normal calbacks when coding for platforms that don't support it.

[00dani]

@epidemian

I think the backcall transformation is no more complex than what the do keyword does, and class is much more complicated than both of them.

Speaking of yield, though, what if we did something crazy and made:

func = ->
  y <- f x
  z <- g q
  y + z

compile to:

func = function*() {
  var y, z;
  y = yield f(x);
  z = yield g(q);
  return y + z;
}

This is almost certainly a bad idea, since it makes backcall use with RequireJS and Gruntfiles and so on totally impossible (along with amb, but honestly people probably weren't going to use amb in CoffeeScript enough to want backcalls for it) and has an inverted function arrow <- that doesn't even produce a function in the output. So it's a possibility but not one really worth considering too much.

Still, CS does really need to support yield and generators at some point, and I think using some sort of operator in place of the actual yield keyword would suit the language.

[jamesonquinn]

From the discussion in another bug, here's how I'd expect a library to replace iced-coffeescript with backcalls to work:

Here's an example from the iced-coffeescript docs:

parallelSearch = (keywords, cb) ->
  out = []
  await 
    for k,i in keywords
      search k, defer out[i]
  cb out

I'd expect to write the "await" library so that the following would work as above:

parallelSearch = (keywords, cb) ->
  out =< await (defer) ->
    for k,i in keywords
      search k, defer i #defer effectively does: (i) -> return ((x) -> out[i] = x)
      #but it also keeps track and await only returns after all defer callbacks have fired.
  cb out #out is now an array of the same length as keywords

You could easily add extra options like defer.setGlobalTimeout(ms) which are impossible in iced.

[00dani]

@jamesonquinn

So your await library would work like the this.group() option provided by creationix/step, effectively? Reasonable, but:

1. Why does each defer require an index? That isn't needed in Step's design. Why not just search k, defer()?
2. There are other kinds of control flow you'd want to support which "real" await/defer can, and I don't think that your library as presented exposes functionality other than effectively that offered by async.parallel or Step's this.group(). You could have different basic functions in the vein of your await function for different kinds of control flow, but then you don't really have an await library, since such a library wouldn't have a single keyword await for generally any kind of "pausing" an asynchronous function.

Personally, I think encoding asynchronous operations in terms of their control flow at all is kind of a huge mess, compared to using value dependencies, i.e., promises. Note that for example the kriskowal/q version of parallelSearch is a mere:

parallelSearch = (keywords) -> Q.all keywords.map Q.nfbind search
# or if you need to maintain nodeback compat
parallelSearch = (keywords, cb) -> (Q.all keywords.map Q.nfbind search).nodeify cb

And that of course compiles to almost identical JS:

var parallelSearch;
parallelSearch = function(keywords, cb) {
  return Q.all(keywords.map(Q.nfbind(search))).nodeify(cb);
}

Considering how simple that is, I honestly can't really see the appeal of ICS-style await/defer, especially considering the shape of the compiled output. Still, people seem to like ICS's input, if not its output, so… (Also, await/defer doesn't handle or propagate errors, as far as I can tell?)

[jamesonquinn]

1. You're right, it could of course infer an index from order of creation of the callbacks. The above syntax is just an idea and would probably need refinement and/or flexibility.
2. Right, you could have a larger library, and then call the bits of it lib.awaite, lib.yielde, lib.catche... and other async equivalents. Again, the above was just a sketch to show how libraries could be written to make backcalls even more useful.

[00dani]

Hmm, true, there're lots of possibilities for varied workflows and such.

On the subject of libraries, I think something like this is all that's really needed for promise backcalls to work pretty nicely:

p = (p, cb) -> p.then cb
# for example
readShoutyJsonFile = (path) ->
  data <-p Q.nfcall fs.readFile, path, 'utf8'
  JSON.parse data.toUpperCase()

In general, I can imagine a library of single-letter functions that implement various different "bind" operations, kind of like this.

module.exports = m = monad = {}
m.l = monad.list    = (lst, cb) -> [].concat lst.map(cb)...
m.p = monad.promise = (p, cb)   -> p.then cb
m.r = monad.reader  = (e) -> (f, cb) -> cb f e

Some of those are likely more useful than others. :P

[Catincan]

A new pledge is available on this issue: https://www.catincan.com/bounty/https-github-com-jashkenas-coffee-script-issues-2762 .

[danschumann]

What if we care about the scope of our callback? maintaining the same arrow syntax makes sense

user =
  username: 'franky beano callienmano'
  save: (done) ->
    console.log 'saving ', @username

    setTimeout (=> done.call this, 'taco argument'), 1000

app.get '/', (req, res, next) ->
  <= user.save
  @username == undefined

  <- user.save
  @username == 'franky beano callienmano'

  (save_success) <- user.save
  console.log 'save was tacos?', save_success

This syntax is beautiful and should be implemented in 2 ways:

1: some async library that we can use with existing coffeescript ( our compiled javascript will have tabs, oh well ) 2: with yields and such.

+1

[danschumann]

Also, what about other types of deferred? Are we expecting the callback to be the last argument in the function?

# consider
$.post('/url').sucess(callback1).error(callback2)

# OPTIONAL ( ERROR OR SUCCESS )
post_deferred = $.post('/url')
(success_args...) <~ post_deferred.success
(error_args...) <~ post_deferred.error
if success_args
  return 'cool'
else
   return 'doh'

Hey look at that, a pretty decent reason to use ~, optional callbacks. What if we have multiple callbacks like this? How do we break them up? We could use <~~ to start a chain of optional callbacks and <~ for each subsequent one.

• 1 for options

[danschumann]

Also, what about parallel?

I don't see the need for async anymore, the whole point is to make coffeescript good enough to do it on it's own, right?

(err{users}, users) <== users_collection.fetch()
(err{pages}, pages) <== pages_collection.fetch()
# now a single arrow to execute the parallel functions
(err{documents}, documents) <= documents_collection.fetch()
# Err could be {users: 'user error', pages: 'pages error', documents: 'documents error'}

There is no err{users} right now, because there is no parallel / separate declaration of a single value. == two arrows, means you're doing two things(at once), in parallel.

In the past my suggestions have been shutdown, probably due to the difficulty that would be in implementing them, so it could end up being that a new compiled js language emerges, inspired by the beauty that is coffeescript, while fulfilling my wildest dreams.

[00dani]

@danschumann

maintaining the same arrow syntax makes sense

We can't have <= mean a bound backcall, because <= already means "less than or equal to". This is why Coco switched to ~> and <~ for bound functions, since that introduces no such ambiguity.

Are we expecting the callback to be the last argument in the function?

Yes. In earlier iterations support for putting a "placeholder" elsewhere (to relocate the callback) was suggested, but eventually it was decided that simply wrapping the expression with a function would suffice. Look for the "hug operator".

Hey look at that, a pretty decent reason to use <~, optional callbacks. What if we have multiple callbacks like this?

By far the most common pattern for callbacks is to use a single callback with an (err, res) signature: a nodeback. Backcalls are designed to work with a sequential chain of async calls following that pattern. There's really no particular support for a call that requires multiple callbacks, because such calls don't correspond to the concept behind a backcall, that being that <- is nothing more than a magical async =:

x =  someSyncCall  arg, arg
y <- someAsyncCall arg, arg
f x, y

Most calls that require more than one callback just don't make sense if you view <- as magic =. There's one common case that takes multiple callbacks and also makes perfect sense when used with backcalls, however: promises, which can take a normal callback and an error-case callback. Fortunately, however, you can still use them with backcalls easily enough, since the backcalls simply provide the success codepath:

p = (pr) -> (f) -> pr.then f
backcallCode = ->
  x <-p somePromiseCall arg, arg
  y <-p someOtherPromiseCall arg
  x + y
directPromisesCode = ->
  somePromiseCall(arg, arg).then (x) ->
    someOtherPromiseCall(arg).then (y) ->
      x + y

Error is simply propagated past the backcall chain, invoking none of the success-path calls. This essentially corresponds to the way the promise monad's error-handling basis, the Either monad, works.

I don't see the need for async anymore, the whole point is to make coffeescript good enough to do it on it's own, right?

Actually, no. If we were trying to make CoffeeScript's asynchronous support complex/powerful enough to model all async patterns by itself, we would probably have jumped straight to IcedCoffeeScript. The problem is that nearly all asynchronous patterns other than simple serial will compile to JavaScript code that's a lot less pleasant: Running calls in parallel requires some kind of reference-counter to be declared and tracked, for example, and at the deep end we end up with the monstrosity that is CPS-transformed JavaScript.

Backcalls are a simple enough syntactic transformation that the compiled JavaScript is not particularly horrible. All they do is flatten callback pyramids. They do not and will not model all async patterns, because that need is much better served by a library such as caolan/async. Such libraries would be used in conjunction with backcalls:

(err, [one, two]) <- async.parallel [oneF, twoF]
codeUsingAsyncResults one and two

[xixixao]

-1 Let's not. Promises or generators are the way out of callback hell, otherwise I (did) would use IcedCoffeeScript.

[00dani]

@xixixao Absolutely they are, but backcalls a) work quite well with promises in conjunction with a helper like the p I defined earlier and b) also work for things other than the callback hell promises solve. They're superior to using IcedCoffeeScript specifically because the syntactic transformation is so trivial: Your compiled JS isn't going to become CPS-conversion hell when you use backcalls.

Generators are definitely a more flexible way to make promise code look synchronous, but they're still not available in every browser, nor do they exist in CoffeeScript yet either. Meanwhile, generators are not going to help you at all with cases like:

export = (x) -> module.exports = x
grunt <- export
grunt.loadNpmTasks 'whatever'
grunt.allYourGruntStuffHere withNoIndent

Or:

$, _ <- define ['jquery', 'underscore']
requireJS.module goesHere withNoIndent

Or even:

result = do ->
  x <- amb [1..10]
  y <- amb [1..30]
  fail() if x * y < 10
  x*2 + y*2

[kibin]

Is this still valid? Does somebody investigate/develop them?