search

mikeal / nodeconf2013

NodeConf 2013 Planning and Sessions
31 stars 4 forks source link

Domains Planning #6

Closed mikeal closed 11 years ago

mikeal commented 11 years ago

I'd like to use this ticket for planning the domains session.

I expect this session to have slightly more "talking" than a normal talk but I don't want that to take up so much time that people don't work with some real code.

I think a "sample app" is a good idea, something that is very simple but has more than one point where you can make it fail and it won't be trapped properly and people can add in support for domains and figure out, on their own, the best places for them.

I'd like to stay away from requiring any extra services or non-standard APIs. The redis client is a great use case but we don't want to run at the camp, or have each person run, a redis server. And I'd also like to stay away from any flow control libraries only because no more than 50% of the attendees will already be familiar with which ever one you choose.

@othiym23 @domenic

dominictarr commented 11 years ago

I think I'm still waiting for someone to write a test framework based on domains - seems like the perfect use-case. I certainly learnt everything I know about handling errors writing test frameworks.

mikeal commented 11 years ago

test frameworks can't inject domains in to the code they are testing, all they can do is wrap before and after.

there are a few things people need to know about domains

a test framework doesn't cover enough of that ground, and could suffer from being it's own conceptual barrier since test frameworks are where opinions turn in to code.

dominictarr commented 11 years ago

Ah, that said, I don't really know anything about Domains. Looking forward to this session!

domenic commented 11 years ago

The easiest and most winning thing to do, I think, is have a standard Node HTTP server (or maybe an Express one if that'd make it easier) and have the task be getting nice 500s out of it.

Then we could try a HTTP server that pools event emitters for whatever reason---maybe abstracted behind a "library"---and have attendees fix the pool-using "library" to bind callbacks.

othiym23 commented 11 years ago

@domenic Express is probably the worst framework to use to demonstrate the usefulness of domains, because TJ does a huge pile of crazy closure + try/catch malarkey to trap and log errors within Connect itself. Figuring out how to wrest errors free of Connect is one of the least pleasant things I've done recently (and if you look at what I do at my day job, I do a lot of pretty crazy / bad stuff). Bare require('http') or Restify would be my choices.

@mikeal, as pretty much everything I've done for New Relic so far is written to be self-contained, I think I should be able to come up with a simple app built on top of the bare http without much difficulty. I also typically don't use control flow libraries much for work, although I've been doing a bunch of stuff with Q lately for workflow stuff in things that aren't related to work. I think there's probably a place in the demo for showing how to use domains with Q and / or async, though, because those are the most popular control flow choices in the community and it's pretty informative / educational to see how domains interact with promises and quasi-CPS.

@domenic, when you say "bind callbacks" with respect to connection pools, do you mean something along the lines of my patch to node_redis? I'd like to get something in there making use of domain.add(), because that, along with domain.dispose(), is what confuses the most people trying to make sensible use of domains.

The tricky thing is that domains are made to simplify dealing with I/O gone wrong, so there should be some kind of call-out to an external service. It's probably adequate to have it do something with the filesystem, but I'm open to other ideas.

othiym23 commented 11 years ago

I'm going to put this out there even though I haven't really figured out how to work it into the session format: part of the value of the summer camp discussion of domains was that it kind of gave a good-sized group of people an opportunity to talk about where things were. There's some weird stuff still in domains -- @isaacs has been talking about nuking dispose(), and some of the issues that cxreg has been fixing show that it's not yet fully solidified at a code level either. Is there some kind of forum for getting together to hash some of this stuff out? There hasn't been a lot of love for domains in 0.10, and it would be nice to give them another round of tweaking in 0.12 / 1.0.

mikeal commented 11 years ago

there will be a lot of "unplanned" time, and during that we can have a larger conversation about domains. because of the size this year we really can't have the same kinds of discussions but we do have some new opportunities.

so, this is what i was thinking for the http app. just core http, and a few endpoints, like GET /setTimeoutException1 that throw, and for each endpoint they need to use a different strategy for capturing the exception. the hardest one is the socket pooling case, but we can figure it out.

othiym23 commented 11 years ago

I'm down with keeping it simple, but I'd at least like to use some kind of (non-Connect) router so the focus is on the domains stuff instead of boilerplate.

domenic commented 11 years ago

@domenic, when you say "bind callbacks" with respect to connection pools, do you mean something along the lines of my patch to node_redis? I'd like to get something in there making use of domain.add(), because that, along with domain.dispose(), is what confuses the most people trying to make sensible use of domains.

Something like that, yeah. I found the Q discussion informative, and here's my patch there, from which I get the "bind callbacks" idea. It sounds like you took a rather different approach?

Re: routers, I'm down with restify, although I think 2.0 might have added domains itself, haha.

othiym23 commented 11 years ago

I think we ended up doing mostly the same things, although things were a little complicated in the case of node_redis because it's already doing some "clever" things with error handling and deferred rethrowing (so that the parser could finish handling a command without leaving the connection in an undefined state). cxreg has since asserted that at least part of what I was working around was a bug in domains themselves, but I haven't had a chance to investigate very closely.

How about director for the router? Or Restify 1.x? I'm going to be taking a look at Restify's router for work soon. I'll have some insight onto how appropriate it is for this after that.

domenic commented 11 years ago

I've had some sad times with director recently, due to it being written for three use cases (browser, server, CLI) and not serving any of them terribly well :-/. But yeah, this is a minor point, whatever we go with will be fine.

mikeal commented 11 years ago

dude, really, just use a switch statement and exact url matching. this is being overthought, you don't need routes cause you don't need variables ;)

othiym23 commented 11 years ago

I just reaaaaaally hate boilerplate in sample code, but sure, a switch statement won't hurt nobody.

domenic commented 11 years ago

I've volunteered to give a talk on domains at a local meetup group in early April, which will force me to put together some slides and a flow for learning them. Will report back!

domenic commented 11 years ago

So I gave that talk and now understand domains pretty thoroughly. Slides are here, with some demo code linked throughout (including a web server with a few error-ing endpoints that need domains to fix them!).

One piece of feedback I got was to give more use cases than just sending 500s to the right response. Things like graceful cleanup and such.

mikeal commented 11 years ago

You guys have two people running this session. That might be enough, but if you feel like you need someone around to get hands on with people give me some names.

mikeal commented 11 years ago

on the slides:

let's focus on hands-on. start with a slide with an example, have them write it out by hand, and then walk through what it does and why it works and get them to tweak it. then go from there.

othiym23 commented 11 years ago

Yeah, my thought was that it would look something like:

It would be a service to everybody if we put extra material in the repo, perhaps with some code katas like the ones you discuss, @domenic. Some stuff it would be good to include as extra material:

As much as possible, I'd like to stick to the core interface of domain.create(), d.on(), and d.run(). d.enter() and d.exit() are occasionally useful (I use them all over the place, but I admit to being a special case). d.dispose() just doesn't need to be discussed at all.

hueniverse commented 11 years ago

Would love to make https://gist.github.com/hueniverse/5167027 better and useful as a reference for the various patterns.

othiym23 commented 11 years ago

@hueniverse That's a lovely backgrounder, but I think @mikeal is right that in an hour-long presentation will be more productive if we just dive into applications instead of talking about background and motivation. Maybe we can incorporate it into the repo?

hueniverse commented 11 years ago

I wasn't suggesting it was used for anything other than a reference material in the repo.

domenic commented 11 years ago

So I'm getting jealous of those web service guys and their fully-fleshed out plan. Let's maybe start firming things up.

I am pretty happy with my server at https://github.com/domenic/domains-romance. Maybe one starting point would be something along the lines of:

From there we could go in a few non-mutually-exclusive directions:

What do you think of the initial plan, and what items from the second list do you think we should prioritize?

othiym23 commented 11 years ago

Sorry I've been a little laggy, I've had a bunch of stuff going on and am just now switching to prioritizing NodeConf stuff.

As much as we can do this via kata, the better. If we can come up with 8 or so challenges, that should be enough to handle a wide variety of experience levels and give people something to play with after the session ends.

Do we need curl, or can we just put together a little Node client built on request? Something with some switches via optimist to trigger different failure modes would be nice.

Here's my rough prioritization:

  1. Express integration (getting domains working with express isn't obvious - process.domain / domain.active is not the best-documented feature in the module - and sooo many people use Express)
  2. pooling (need an example that everyone can preinstall and run locally, but Redis or node-MySQL would be ideal)
  3. req & res
  4. capture and shutdown vs recover
  5. how domains work, and how to work with the enter and exit

I'll try to put together a first pass at an outline tomorrow night, unless you throw one up first.

domenic commented 11 years ago

Agree with everything. One point to clarify before I head to bed:

Do we need curl, or can we just put together a little Node client built on request? Something with some switches via optimist to trigger different failure modes would be nice.

My idea was that if we supply them with a crash-prone server.js, it's best to make them inspect the server.js and find all the crash modes, and then trigger them manually by POSTing invalid data or whatever. This seems more interactive and more likely to get them to understand what's going on than just letting them get off easy with "run server.js, run crashserverjs --crashendpoint=1, run crashserverjs --crashendpoint=2, ...".

But yeah something simple and request-based that has the ~2 features of CURL that you need would work. (Viz specify the method, specify any POST data, and give nice pretty ingoing/outgoing diagnostics.)

mikeal commented 11 years ago

Anything we can include as a node dependency, the better. This might work: https://github.com/hakobera/ncurl

othiym23 commented 11 years ago

DRAFT

Unresolved questions:

Timeline

00:00 - 00:05

00:05 - 00:20 simple walkthrough of a server built with domains

"use strict";

var concatStream = require("concat-stream");
var domain       = require("domain");
var http         = require("http");
var url          = require("url");

http.createServer(function (req, res) {
  var d = domain.create();

  d.on("error", function (err) {
    res.statusCode = 500;
    res.setHeader("Content-Type", "text/plain");
    res.end("We encountered an error!\n\n" + err.stack);
  });

  d.add(req);
  d.add(res);

  d.run(function () {
    var parsedUrl = url.parse(req.url, true);
    console.log(req.method, parsedUrl.href);

    if (parsedUrl.pathname === "/sum-numbers" && req.method === "POST") {
      req.pipe(concatStream(function (data) {
        var numbers = JSON.parse(data);
        var sum = numbers.reduce(function (acc, number) { return acc + number; }, 0);

        res.statusCode = 200;
        res.setHeader("Content-Type", "text/plain");
        res.end("Sum: " + sum);
      }));
    } else if (parsedUrl.pathname === "/throw" && req.method === "GET") {
      throw new Error("Pretend I am deeply nested inside library code!");
    } else {
      res.statusCode = 404;
      res.setHeader("Content-Type", "text/plain");
      res.end("No content for " + req.url);
    }
  });
}).listen(1337);

00:20 - 00:30 using a domain with EventEmitters created before the domain

"use strict";

var domain = require("domain");
var mysql  = require("mysql");

var pool = mysql.createPool({
  host     : "db.kvwn.com",
  user     : "brick",
  password : "LOUDNOISES"
});

function withConnection(connection) {
  return function myCallback(error, rows, fields) {
    // do stuff here, including custom error handling
    connection.end();
  };
}

var outerD = domain.create();
outerD.on("error", function (error) {
  console.log("Unable to connect to database!\n\n" + error.stack);
});

pool.getConnection(outerD.intercept(function (connection) {
  // can't just add connection, because it's from a pool and will be reused

  var innerD = domain.create();
  innerD.on("error", function (error) {
    console.log("Error in callback inside connection.\n\n" + error.stack);
    connection.end();
  });

  connection.query("SELECT * FROM shows", innerD.bind(withConnection(connection)));
}));

00:30 - 00:40 using domains with Express

domain-errors.js:

var domain         = require("domain");
var defaultHandler = require("express").errorHandler();

module.exports = function errorHandlerGenerator() {
  return function errorHandler(err, req, res, next) {
    if (domain.active) {
      domain.active.emit("error", err);
    } else {
      return defaultHandler(err, req, res, next);
    }
  };
};

server.js:

var http          = require("http");
var domain        = require("domain");
var path          = require("path");
var express       = require("express");
var domainHandler = require("./domain-errors.js");

function databaseCall() {
  throw new Error("oh no the database exploded");
}

var app = express();

app.set("port", process.env.PORT || 1337);
app.set("views", __dirname + "/views");
app.set("view engine", "jade");
app.use(express.favicon());
app.use(express.logger("dev"));
app.use(express.bodyParser());
app.use(express.methodOverride());
app.use(app.router);
app.use(express.static(path.join(__dirname, "public")));
app.use(domainHandler());

app.get("/", function (req, res) {
  res.render("index", {title : "Express"});
});

app.get("/throw", function (req, res) {
  throw new Error("Imagine I'm deeper in the code");
});

app.get("/handle", function (req, res) {
  var subDomain = domain.create();
  subDomain.on("error", function (error) {
    res.send(500, "<html><head><title>whoops</title></head><body>" +
             "<p>Got error: '" + error.message + "'. Ignoring.</p>" +
             "</body></html>");
  });

  subDomain.run(databaseCall);
});

var mainDomain = domain.create();
mainDomain.on("error", function (error) {
  console.error("caught %s in main domain, shutting down", error.message);
  process.exit(1);
});

mainDomain.run(function () {
  http.createServer(app).listen(app.get("port"), function () {
    console.log("Express server listening on port " + app.get("port"));
  });
});

00:40 - 00:45 error-handling tradeoffs

00:45 - 00:55 building domains up from small pieces (how domains work)

othiym23 commented 11 years ago

Could use more ideas for katas, more questions for campers, more references to Anchorman. I need to write up the code samples, and we need to figure out what we want to use for the pooling service.

One thing I like about this outline is that I can almost guarantee there will be some new material in there for everyone. The questions is whether it's overstuffed as a result. Could use some feedback from everyone on that point.

othiym23 commented 11 years ago

ncurl is no bueno -- I could probably fork it and fix its issues, or I could write my own equivalent on top of request and optimist in about the same amount of time. Is there a reason we're assuming that people wouldn't have access to curl? Windows users? People who have been living in caves for the last 20 years and have a sort of vague memory of using lynx?

othiym23 commented 11 years ago

I guess what I'm saying is that if we can't assume curl will do the job / will be present, I'll go ahead and hack together my own simplified version of it this week.

domenic commented 11 years ago

Windows users is a big concern, yeah. Just hack something up; all it needs is:

More responses later.

othiym23 commented 11 years ago

Started hacking on my own tool here: https://github.com/othiym23/uncurled

Will try to get request method done tonight and curl-compatible POST from stdin done this week.

othiym23 commented 11 years ago

Uncurled does the things we need it to do for now. @domenic, if you could bang on it a little and file any issues / send me any pull requests for stuff you want it to do / do differently, that would be great. I updated the syllabus above to reflect some tweaks I've made to the first example server.

mikeal commented 11 years ago

Could someone take the initiative and add the dependencies you're going to need for this session to the nodeconf2013 package.json?

othiym23 commented 11 years ago

I'll do that this weekend. Still figuring out all the deps, and have been up to my eyeballs in work and conference / talk prep all week.

domenic commented 11 years ago

Sorry for disappearing for a bit. uncurled looks solid, tested it out, works great on Windows, whee.

How did the dry-run go? What can I help with? Slides, more katas, commiting things to the repo, ...? Did we get our dependencies nailed down so poor @mikeal can stop worrying?

othiym23 commented 11 years ago

More katas would be great. I can throw together a set of slides using Reveal.js later this week, but if you want to take a whack at it, I will not complain. My plan is a terse recap of the API reference docs on the relevant methods, and then the code samples on the slides, with some slides stashed towards the back for the advanced / discussion topics. The goal is to have this be as hands-on and direct as possible, with you and I spending most of our time circulating and helping people out.

The dry-run was informal (and I think I ate @substack's saag paneer – sorry dude), but one of the upshots was that I needed to ping you so we could figure out how we want to divvy up the labor, so I'm glad you saved me the ping! :tada: I'm totally fine handling the presenting, I just want to make sure we're in agreement on how we want to handle things in-session. When do you get into SF?

I don't want to force everyone to deal with the hassle of installing MySQL or MongoDB, so I adapted a little proof of work server I had lying around and am building a generic-pool wrapper around it, which allows me to both get domain.intercept in there as well as showing how you need to bind callbacks that are going to be passed off to the connection pool. It's kind of ridiculous and contrived but will get the job done. If you have a better idea (preferably in pure JS), it's totally welcome.

I put my Express example into src/domains (with the middleware stubbed out so as not to ruin everyone's fun) and will put what I have so far for the proof-of-work example before I go to bed tonight. I added uncurled to the list of dependencies (and checked in the Express example with its node_modules folder), but everything else we're using was in there already.

othiym23 commented 11 years ago

Also @domenic, this is totally optional, but if you wanted to put together a little explanation of why you don't think promises and domains work well together, I at least would be interested to hear it. Since you're The Promises Guy, there might be other people there who are curious too.

Raynos commented 11 years ago

I'd also be interested to see how the error handling mechanism around promises works at the large scale.

mikeal commented 11 years ago

I'd be curious too, the last time we talked about this (which i think was in a pull request for Q or maybe node-redis) it seemed like any flow control library would be able to take a good amount of the manual pain out of domains.

domenic commented 11 years ago

Hmm, I should probably be at least replying to this thread while at work. Feel free to bug me via GTalk/Jabber (ID = domenic@domenicdenicola.com) or IRC; I'm usually available even during work hours.

More katas would be great.

I noticed https://github.com/mikeal/nodeconf2013/tree/master/pkg/domains doesn't include any of the code from upthread at https://github.com/mikeal/nodeconf2013/issues/6#issuecomment-19092750, which seems not so great? It's not really in kata form yet anyway? I'll try to get some katas in there right now.

I can throw together a set of slides using Reveal.js later this week, but if you want to take a whack at it, I will not complain. My plan is a terse recap of the API reference docs on the relevant methods, and then the code samples on the slides, with some slides stashed towards the back for the advanced / discussion topics. The goal is to have this be as hands-on and direct as possible, with you and I spending most of our time circulating and helping people out.

Seems like a good plan. If you haven't gotten to it by the weekend I'll give it a shot.

I'm totally fine handling the presenting, I just want to make sure we're in agreement on how we want to handle things in-session. When do you get into SF?

I'm happy to help with presenting too, I guess we'll see how the split goes once we actually have some slides, haha. I get in to SF right before the first bus leaves actually, so we can coordinate/rehearse in the morning before the second bus arrives I think. (Thanks @mikeal for pointing out that camp counselers should be on the first bus, I was going to sleep in otherwise.)

I don't want to force everyone to deal with the hassle of installing MySQL or MongoDB, so I adapted a little proof of work server I had lying around and am building a generic-pool wrapper around it, which allows me to both get domain.intercept in there as well as showing how you need to bind callbacks that are going to be passed off to the connection pool. It's kind of ridiculous and contrived but will get the job done.

Looks pretty awesome, just make sure to write a nice readme so people understand what they have to care about versus what they don't. I guess this is in "fully finished" form, not kata form, right?

I added uncurled to the list of dependencies

I wonder if it would be easier to add instructions like "do npm install -g node_modules/uncurled" than to tell them to type ./node_modules/bin/uncurled all the time?

Also @domenic, this is totally optional, but if you wanted to put together a little explanation of why you don't think promises and domains work well together, I at least would be interested to hear it.

Heh. It's not quite like that, it's more that domains catch situations where promises don't---they're more of a safety net. E.g. with promises you have to buy in totally and use e.g. Q.delay instead of setTimeout, whereas since domains overwrite the implementation of setTimeout and everything else, you can just flip the big global domain switch and it'll handle errors in setTimeout callbacks etc. I would use both in a single app, with domains as the safety net and promises as the usual error flow/bubbling mechanisms. I think this is a bit off-topic so maybe just mention that I'm happy to talk your ear off about promises and how they address error handling and fit with domains, over beers later.

I'd be curious too, the last time we talked about this (which i think was in a pull request for Q or maybe node-redis) it seemed like any flow control library would be able to take a good amount of the manual pain out of domains.

Yeah, we did add that to Q, not sure how much of a difference it's made in practice. Might be worth seeing if we can fit in something.

domenic commented 11 years ago

OK added the crashy-server code as a kata:

https://github.com/mikeal/nodeconf2013/tree/master/pkg/domains/katas/1-crashy-server

I was thinking we should adapt your existing express-middleware to be a kata too? It's already "solved" though, and contains the subdomain complication, so I'm not sure what exactly to do there. Similarly for the POW server?

Should we perhaps add a "kata-solutions" folder? Seems prudent.

POW server doesn't seem to send out any 500 errors, but does have a bunch of clustering stuff going on, so that's confusing me a bit.

othiym23 commented 11 years ago

Feel free to bug me via GTalk/Jabber (ID = domenic@domenicdenicola.com) or IRC; I'm usually available even during work hours.

I personally have another crazy thing I have to get ready for that happens soon after NodeConf for my company's engineering team, so I'm pretty focused on that this week at work hours. I can be found on IRC as well, or hit up at spankysyourpal@gmail.com on GTalk.

I noticed https://github.com/mikeal/nodeconf2013/tree/master/pkg/domains doesn't include any of the code from upthread at #6, which seems not so great? It's not really in kata form yet anyway? I'll try to get some katas in there right now.

My intention was that everybody would type those first chunks of code from the slides in by hand, but I like your katafied version and it saves everybody a bunch of typing so :+1: to that.

Seems like a good plan. If you haven't gotten to it by the weekend I'll give it a shot.

I will definitely get slides done this week. I wanted to get all the code in by Wednesday, so that's what I've been concentrating on. Tomorrow night I'll make a first pass at the slides.

I'm happy to help with presenting too, I guess we'll see how the split goes once we actually have some slides, haha. I get in to SF right before the first bus leaves actually, so we can coordinate/rehearse in the morning before the second bus arrives I think. (Thanks @mikeal for pointing out that camp counselers should be on the first bus, I was going to sleep in otherwise.)

We have all day Thursday once we get to camp, modulo the time we use to set everything up. My hope is that the presenting load should be pretty minimal and that the bulk of the time will be spent workshopping, so the main reason to split the load is so that our first and sixth (last of the first day) sessions don't get crushed by nerves and / or exhaustion-induced apathy.

Looks pretty awesome, just make sure to write a nice readme so people understand what they have to care about versus what they don't. I guess this is in "fully finished" form, not kata form, right?

I added the piece that is in kata form tonight, although adventurous souls could go in and add a bunch of additional domains stuff to the proof of work server if they feel like it as well. I'll add some notes to that effect in the source. I'll also write up a README as soon as I add this note.

I wonder if it would be easier to add instructions like "do npm install -g node_modules/uncurled" than to tell them to type ./node_modules/bin/uncurled all the time?

Yes, I'll add a note to that effect. Good idea! Thanks!

othiym23 commented 11 years ago

Should we perhaps add a "kata-solutions" folder? Seems prudent.

I'd really rather not -- I'm fine with people leaning on each other (and us, obvs) for solutions, but I don't want to leave any attractive nuisances around. I often lack the self-control to avoid peeking. If we could do a 4clojure.org-style oracle that allows people to submit solutions and tell them whether or not their code works, that would be rad, but there's only a few hours between now and @mikeal's deadline, so I don't think I'm even going to try to get fancy.

POW server doesn't seem to send out any 500 errors, but does have a bunch of clustering stuff going on, so that's confusing me a bit.

It's meant to be an opaque chunk of infrastructure for the sake of this exercise, but yeah, it should make things more entertaining and blow chunks every so often. I think I'll modify both it and the connection pool to blow up every once in a while, just to keep people on their toes.

domenic commented 11 years ago

I added the piece that is in kata form tonight

Wow, either you are super-fast or we were working on this at the same time. I like that a lot, it seems great.

but there's only a few hours between now and @mikeal's deadline

Augh!

OK what about kata-ifying the express server? I think maybe remove the /handle route from the kata but instead put it on slides or such, since it's harder to explain? The idea is that the error handler they write should do a 500, I assume, and thus prevent the "caught %s in main domain, shutting down" from being reached.

domenic commented 11 years ago

Actually the express server is really confusing me :(. I thought the solution was more along the lines of http://stackoverflow.com/q/11599739/3191 but you've got an error handler thing going on? Halp.

othiym23 commented 11 years ago

The proof of work server example is now super crashy. Working with it feels unpleasantly risky, which seems about right.

I also added lots of comments for where people could play with adding domains to the various components if they want to experiment with adding domain support to various pieces of the system. I'll probably add a slide or two showing how the Express solution I have in mind can be adapted to work with connection pools (e.g. in a way analogous to what I've tried to do with node-redis).

Actually the express server is really confusing me :(. I thought the solution was more along the lines of http://stackoverflow.com/q/11599739/3191 but you've got an error handler thing going on? Halp.

So, I don't like that solution. I like being able to bind the domain's error listener to a specific route / handler, where I have context about what's going one. One of the difficulties with the way Connect runs middleware chains is that it basically prevents the domain system from seeing the errors. The solution I have in mind is the one in the outline up above, where a simple error-handling middleware will emit any errors it sees on the domain if one is active, and otherwise will use the default Express error handler.

What would be nice would be able to have one middleware that sets up a domain for every request at the beginning of the middleware chain, and then another one that runs at the end of the chain to capture any errors that aren't caught by the domain, with each route handler being able to specialize the error-handling to match their scenario. That's not really how Express and Connect work, though. Suggestions?

othiym23 commented 11 years ago

Oh, and note that the express example is already somewhat katafied -- I didn't fill out the domains-based error handler, which is what I intended to talk about in the actual session.

domenic commented 11 years ago

Re: express example. I guess the question is, what's our end goal? "Nice 500s" is a goal that makes sense to me. In particular, how do you recover from

app.get("/database", function (req, res, next) {
    setTimeout(function () {
        throw new Error("oh no the database exploded");
        next();
    }, 1000);
});

and in particular associate that with res so you can do a 500 on res?

Or is the goal just to log the exception, using the middleware to attach e.g. error.req and error.res for better logging? And still shut down the server?

othiym23 commented 11 years ago

Also I'm going to pull 1-crashy-server up from katas/ because now all three of the examples have katas embedded in them. I'll write a README for each one and an overall README providing context.

othiym23 commented 11 years ago

Re: express example. I guess the question is, what's our end goal? "Nice 500s" is a goal that makes sense to me.

If that's the path you want to follow, we can do something like this:

var domain = require("domain");

module.exports = function (req, res, next) {
  var d = domain.create();
  d.on('error', next);
  d.run(next);
});

and then use the default Express error handler to get its error page.

I think that's likely to make more sense to more people, I just like to have more explicitly control over that stuff per route. But that's also because I tend to write API servers rather than web apps, and tend to prefer Restify over Express.

I guess the best (read: most wishy-washy) solution is to explain both strategies to people, and then let them decide for themselves which makes the most sense. I'll redo the example to strip out the specific implementation, incorporate your sample route, and add comments as appropriate.