NodeUp: A Callback Convention Show

[FredKSchott]

@getify, @crabdude, and myself have all taken stabs at this, but there still isn't an agreed upon callback contract within the community. Conversations around the topic are usually enthusiastic, but twitter and 140 characters does not a good forum make.

NodeUp could be an interesting place to have this conversation with & in front of the community. Not necessarily to come to an agreement, but to discuss the finer points of what would belong in a contract / what a contract might look like if one existed.

Not sure if NodeUp has ever taken on a topic like this or a show in this format, but the possibilities sounded interesting :)

Relevant Links:

• https://fredkschott.com/post/2014/03/understanding-error-first-callbacks-in-node-js/
• https://gist.github.com/CrabDude/10907185
• https://twitter.com/getify/status/472219733605888001

[timoxley]

This issue has been discussed in a previous nodeup. Convention unanimously error first, and once async always async (don't release zalgo).

[timoxley]

AFAIK

[getify]

That agreement is fine but it doesn't address the more strict questions that @crabdude has posited.

[dshaw]

@FredKSchott I'm down

[CrabDude]

Sounds good to me.

[getify]

Me too :)

[getify]

As an anecdote, I just added an adapter to asynquence that allows you create wrappers around several (six, in fact) variations of function signature around callbacks (params-first vs params-last, errfcb vs splitcb vs simplecb) -- see the six config options.

It's a shame this sort of complexity is necessary, and were we to have a stronger contract convention (not just in node, but beyond) for such things, we would reduce the cognitive load AND the complexity of libs across the board.

[rvagg]

This is a nice topic but could it span an entire show? Can someone outline 3 broad sections they would take 40mins to an hour or is this more like a quick 20min show? I don't see a whole lot of.controversy here, in general there are strong conventions, outlined well by @crabdude in his gist, the only problem comes from browser abstractions leaking in where conventions aren't strong at all.

[FredKSchott]

@rvagg yea it had been a bit more of a topic at the time that this was posted, @crabdude had just posted that gist and there had been some discussion on Twitter. I haven't heard the topic mentioned much since.

I doubt this would ever be a controversial topic, the format would probably be more educational / exploratory in nature (check the OP). I still think the discussion would be interesting, but given how much else is going on and how busy it has been lately, I haven't looked into it any more than the original post.

[CrabDude]

@rvagg Yes, this topic is controversial.

Presently, I do an hour+ session on it in my Noderiety trainings in my weekly engagements. You could easily do 3 hour-long sessions on this.

Here's a rough outline:

1. Asynchrony a. What's a nodeback and why does a convention need a spec? b. What anti-patterns does the spec address? c. Why are Promises currently objectively superior to nodebacks in this regard?
2. Control-flow a. What are the minimum requirements of a control-flow library? (error handling & contract enforcement) b. Why are the most popular node.js control-flow libraries anti-patterns? (lack of contract enforcement)
3. Error Handling a. How is it broken and how do we fix it today? (throw vs pass, domains, try/catch) b. How do we really fix it? (going forward) c. Crash on uncaught exceptions is an unaddressed DoS liability and a hindrance to enterprise adoption.

FWIW, I have a 5 year history of strong public Promise opposition, so my pro-Promise stance is reluctant and therefore objective. Additionally, my wisdom regarding the above is largely gained from my experience with my async trycatch library that solves an issue that is otherwise presently completely broken and unaddressed in the community (think what domains should have been and what event source hooks won't solve).

I'm happy to sit down or Skype ahead of time on the above if you would like more details.

[CrabDude]

You can pretty much add the whole second half of this talk as a major contention point for asynchrony in node, "Netflix JavaScript Talks - Version 7: The Evolution of JavaScript"

TLDR; ES6/7 language features (Promises, async/await, Generators, Iterables, and Observables) are likely going to obviate callbacks and node.js Streams, in a good way, especially by making streams even more composable:

for(let data of readFileStream) {
    // ...
}

[getify]

Promises don't obviate callbacks, they simply let you pass callbacks to a different, more composable, more trustable mechanism. Generators(+promises) just hide where the callbacks are specified, async/await puts syntactic sugar on top of generators+promises, and observables even further abstract async functions into generators to allow multiple outputs.

No matter how much abstraction you layer on top, when you unravel the whole stack, it still comes back to callbacks.

[rvagg]

The risk for this show is it simply devolves into an argument about Promises and ES6 and ES7 features and whether they are the magic that will save the world or the spawn of the hell. It's kind of difficult to have a balanced and calm and objective discussion about these things without it being completely lop-sided, and we're certainly not after a single-person rant either.

Note how quickly we've shifted from a "callback convention show" to something entirely different, that's the nature of this topic area, almost without fail.

Perhaps @timoxley would be interested in taking this one on and crafting something interesting out of it? Also of interest here is the last io.js TC meeting discussion on crafting a statement on the adoption of a Promises API, see 2/3rds of the way through the video: https://github.com/iojs/io.js/pull/144/files

[CrabDude]

I'll respond more fully later, but I did want to mention that the devolution of the argument is in fact an indication that there is something good and of value beneath the surface, we just need to extract it and avoid the noise, and flames.

Also, my last comment was because the recent Netflix talk really drove home to me how objectively superior from a language integration and composability perspective promises/async/await will be to other means. I'm not trying to flame; I'm being earnest and encourage you to watch the talk and ask yourself:

1. How would this look with callbacks / co
2. How composable would that alternative be in comparison

I'll come back with a more focused response and revised proposal when I have a little more time.

[CrabDude]

@rvagg

The risk for this show is it simply devolves into an argument about Promises and ES6 and ES7 features and whether they are the magic that will save the world or the spawn of the hell. It's kind of difficult to have a balanced and calm and objective discussion about these things without it being completely lop-sided, and we're certainly not after a single-person rant either.

Yes. You are completely right. Any discussion would need to be focused and not just conjecture or opinion.

If there's interest, I propose the following outlines for 2 separate discussions:

Callback Contracts (no promises)

1. What are common anti-patterns are their solutions?
2. What would a formal callback contract look like?
3. How do we enforce the callback contract? (manual, control-flow libraries)
4. BONUS: Where does error handling fit in if at all? (try/catch/throw vs pass)

Promises, async, await, observables

1. What unique features do Promises offer?
2. What direction is core headed in for enabling Promise integration?
3. What direction is the JavaScript language headed in for dealing with async values? (ES7 Promise composability, async, await)

As an aside, from the io.js meeting minutes,

a callback API is unlikely to ever go away

Since I have no direct influence on core, my opinion and perspective is primarily as an application developer and trainer. In that regard, I think first-class promises/async/await/obervables integration will be adopted rapidly and are far easier to teach and become part of an "Async JavaScript: The Good Parts" type of rubric. I don't know where or if that opinion fits into nodeup in any way though.

[timoxley]

The risk for this show is it simply devolves into an argument about Promises and ES6 and ES7 features and whether they are the magic that will save the world or the spawn of the hell.

Sounds like a good show to me. I'd be happy to run that, I'm not emotionally invested in either camp.

@jonathanong would be good fuel for the fire.

Should probably pull in a TC member, maybe @bnoordhuis.

[jonathanong]

don't feel like bikeshedding on control flow in public...

my only opinions are primarily:

• promises + async/await. generators are only a stop gap and should never be exposed in a public API.
• uncaught exceptions means you wrote bad code somewhere and will become less of an issue w/ promises and async/await. i do not believe node core should focus on solving the problem of uncaught exceptions. unfortunately, i'm a believer in writing bug free code, so i'm not sure how big of a problem this is with other developers.
• i do not believe that core will never support promises in their API because when ES7 comes and core doesn't return promises, a lot of people will be mad. it's just a matter of time.

that's all i'd say really.

[timoxley]

@jonathanong that is great content. Even if you just read this aloud I think it's worth getting you on board. I'd also like to discuss things like the recent changes in co (thunks -> promises) and also your work on mz.

[bnoordhuis]

I want to politely decline, I'm not really cut out for interviews or debates.