Closed mikeal closed 11 years ago
I'd like to define a simple but useful API, then have people implement it (or parts of it) using the three platforms. We'll have a server running that you ping once your own API service is up, and it will validate it by running a simulated client against it. Then we show on a screen who finished (or if you give me 40-node controlled light bulbs, we turn the light on when you are finished :-)).
For express, we'll prepare a template that already includes some useful middleware for handling API stuff like validation.
I'm not sold on raw node http
. What's the value here?
The overall scenario sounds good to me and I like the simulated client idea for checking people's work.
Teaching with the built-in http module would be good if teaching people nuts-and-bolts http module use is a priority. Having them develop using the built-in http module would also show exactly what abstractions hapi and express are providing beyond the built-in http module's capabilities.
... An alternative to providing a full built-in http module solution, however, could be to talk about how you'd have to do it with the built-in http module when going over certain parts the hapi and express solutions. i.e. Explaining the benefits of using hapi/express vs vanilla node http.
That would be a great introduction to the workshop. Start with bare http server, add a simple switch statement, etc. then start identify the real-world needs of a router and other facilities. We can do this in a few minutes using slides, then dive into two parts of express and hapi.
Nice... I like the intro idea.
Which features of express do you want to cover? What additional middleware? Since hapi has a lot of fully integrated functionality, it would be easier to start from the express side and then do the mapping.
I was thinking of something like this... implementing the API mostly from scratch:
So probably not using any non-Express components (except express-resource)...
Not even passport? or some other 3rd party login?
that seems like way to much to cover in the amount of time provided, how are they going to write that much code?
also, keep in mind that the range of knowledge will be from knowing little about HTTP to being experts. ideally you'd want to teach them how to handle requests with these APIs without teaching a class on HTTP.
@hueniverse Actually yeah, using passport would simplify things.
And you're right @mikeal ... too much for the time allotted. I'll look at this tonight and scale it down to the essentials.
Start with dropping XML :)
@hueniverse Haha you have a point.
Okay, here's a trimmed down outline... let me know if this still seems like it might be too much.
For authentication, something simple like basic auth might be the way to go given that people can easily relate to username/password authentication.
I think we should built it up, starting with express and moving into hapi, after we show a few quick slides about pure node with a big switch statement doing a simple router.
The express part will add a GET and POST endpoints, and cover how to add middleware. I think the biggest value talking about express is to show how middleware is loaded and how the router works. Add some console.log along the way, etc. This is something we can customize easily for beginners and experts, as most experts have never looked closely into the express router logic and how the middleware really works,
Once we have that, we can move to hapi, add the same two endpoints (which will take very little time given the integrated nature of the framework), and discuss how the hapi router is different and the benefits/disadvantages of the two approaches - ideally via another coding exercise, adding another endpoint to both side by side - maybe authentication / content negotiation / error handling.
I don't think we can do everything, and I think the real value here would be to do something hands on that exposes the inner working of both frameworks. You can read the readme of both to build a simple app, but we can offer something better by showing how a simple example translates to running code.
What would probably work best is spend 10m setting the stage, going over a super simple pure-node example of a GET and POST. Then let people do the same in express, followed by hapi. This will take about 10m each. Then spend the remaining 20m tasking people with adding one more endpoint to both demonstrating something more advance. We can have a few other ideas fully baked for the experts who finish fast and want to do more.
This plan sounds great... it grounds people in the context of Express/Hapi vs stock Node. Those time allotments sound sensible as well.
So I guess what we'll need to do is to design the endpoints, create the testing application, then add the Node/Express/Hapi template applications somewhere (this repo?).
yes, please use this repo. add all of your materials to this repo. everything in /package
will be installed before they enter your session. for your session, please use /package/green/
for anything you'd like them to have.
green?
explained in the README :)
https://github.com/mikeal/nodeconf2013/blob/master/readme.md
Because /package/webServices would have been too cryptic :)
not cryptic enough, i'm trying to make it so that people can't tell by downloading the package exactly what they'll be doing when :)
Can we rely on a local network? Internet access? Need to know if there is a point in building a validation server they can run their services against, and it will hit it and check if they got it right. The idea is for each person to create a server, run it, then go to a web page somewhere (local or remote) and test.
If a reliable network is not possible, we can just have them run both servers (the test server and their own implementation). That might be better to do anyway since its simpler and reliable.
Having them run both servers makes sense. Less to go wrong.
@mcantelon did you start working on something? If not, I've got time this week to get an initial session outlines and we can iterate off that.
@hueniverse I haven't started anything so if you want to rustle up an outline that sounds good.
Start with a bare node API server:
var http = require('http');
var server = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
});
server.listen(8000, localhost);
Move handler method out:
var http = require('http');
var handler = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
};
var server = http.createServer(handler);
server.listen(8000, localhost);
Add basic path routing:
var http = require('http');
var dispatcher = function (req, res) {
var result;
switch(req.url) {
case '/': home(req, res); break;
case '/user': user(req, res); break;
default:
res.writeHead(404, {'Content-Type': 'application/json'});
var result = { error: 'not found' };
res.end(JSON.stringify(result));
break;
};
var home = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
};
var user = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { id: 23123, name: 'joe' };
res.end(JSON.stringify(result));
};
var server = http.createServer(dispatcher);
server.listen(8000, localhost);
Highlight the function (req, res)
signature and discuss other basic routing facilities:
I'd like the make the API more fun by using some relevant data (since we can't rely on internet access). Can we get a json file (or the raw data) with all the participants, their twitter handle, and anything else we can find that's public? If I get the list I can grab some github info, twitter info, and maybe a photo. Then pack all of this into a json file they can load into the web services app and use to power the API.
If not, any other suggestions? I don't want to just build a dummy services. That's stupid.
we don't usually hand out the participant data because some of the sponsors like to use it for semi-sketchy marketing. it's one of those things where I totally trust some sponsors and definitely don't trust others so I just don't give it out.
i can get you a list of names but without email addresses. we didn't grab twitter or github info at registration.
Can you give me the list of emails so I can try and match them to avatars or twitter accounts? But the actual file will not include the emails. If not, got other ideas for fun little service that's not completely stupid?
we can do that, so long as the email addresses aren't in the tarball or anything.
On May 30, 2013, at 12:58PM, Eran Hammer notifications@github.com wrote:
Can you give me the list of emails so I can try and match them to avatars or twitter accounts? But the actual file will not include the emails. If not, got other ideas for fun little service that's not completely stupid?
— Reply to this email directly or view it on GitHub.
Initial outline looks good... I'm fleshing out the Express part and will post it by tomorrow.
@mikeal Can I get it or a sample so I can see what fun info I can get on the web excluding email that we can build a web service on?
(Added some Express bidniz.)
Start with a bare node API server:
var http = require('http');
var server = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
});
server.listen(8000, localhost);
Move handler method out:
var http = require('http');
var handler = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
};
var server = http.createServer(handler);
server.listen(8000, localhost);
Add basic path routing:
var http = require('http');
var dispatcher = function (req, res) {
var result;
switch(req.url) {
case '/': home(req, res); break;
case '/user': user(req, res); break;
default:
res.writeHead(404, {'Content-Type': 'application/json'});
var result = { error: 'not found' };
res.end(JSON.stringify(result));
break;
};
var home = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
};
var user = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { id: 23123, name: 'joe' };
res.end(JSON.stringify(result));
};
var server = http.createServer(dispatcher);
server.listen(8000, localhost);
Highlight the function (req, res)
signature and discuss other basic routing facilities:
Install Express in your working directory:
cd /your/working/directory
npm install express
Create an Express API server:
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(app.router);
});
app.get('/', function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
});
http.createServer(app).listen(3000);
The Express response object has a method for specifically returning a JSON response. This automatically returns the appropriate HTTP header and takes care of converting your JSON to a string. The string will also be "prettified" so as to be easier for humans to read.
To see this at work, change the above GET response logic to the logic below:
app.get('/', function (req, res) {
var result = { hello: 'world' };
res.json(result);
});
As with the Node HTTP module example, we can move the handler logic into a separate function:
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(app.router);
});
var handler = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
app.get('/', handler);
http.createServer(app).listen(3000);
We'll now add a new endpoint, a placeholder for a user data endpoint, as we did in the non-Express example:
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
var result = { id: 23123, name: 'joe' };
res.json(result);
};
app.get('/', home);
app.get('/user', user);
http.createServer(app).listen(3000);
Let's now make the user data endpoint useful by adding the ability to specify a user using a GET parameter.
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
app.get('/', home);
app.get('/user', user);
http.createServer(app).listen(3000);
We can create logic to add a new user as well. For this we need to enable Express's parsing of HTTP POST data.
Express handles HTTP POST data parsing via "middleware". Express middleware provides request pre/post-processing of HTTP requests/responses.
(diagram)
app.configure
is where middleware can be specified using the app.use
method. To enable parsing of POST data we enable the bodyParser
middleware.
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(express.bodyParser());
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
var userAdd = function(req, res) {
// use POST params to create user
};
app.get('/', home);
app.get('/user', user);
app.post('/user', userAdd);
http.createServer(app).listen(3000);
Custom middleware can also be created to do things like validate HTTP parameters or injust infortation into the HTTP response headers.
In the below code we add custom middleware to specify an API version.
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(function(req, res, next) {
res.setHeader("X-API-Version", "0.0.1");
return next();
});
app.use(express.bodyParser());
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
var userAdd = function(req, res) {
// use POST params to create user
};
app.get('/', home);
app.get('/user', user);
app.post('/user', userAdd);
http.createServer(app).listen(3000);
Last, but not least, you'll add some basic security to our API using HTTP basic authentication. You add authentication to the POST endpoint that allows a user to be added.
To add this you'll create a middleware component that leverages the built-in basicAuth
middleware.
var auth = express.basicAuth(function(username, password, callback) {
var result = (username === 'myuser' && pass === 'mypass');
callback(null, result);
});
In the following code you'll add this middleware to the application and apply it to the /user
POST endpoint.
var express = require('express')
, http = require('http');
var app = express();
app.configure(function(){
app.use(function(req, res, next) {
res.setHeader("X-API-Version", "0.0.1");
return next();
});
app.use(express.bodyParser());
app.use(app.router);
});
var auth = express.basicAuth(function(username, password, callback) {
var result = (username === 'myuser' && password === 'mypass');
callback(null, result);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
var userAdd = function(req, res) {
// use POST params to create user
};
app.get('/', home);
app.get('/user', user);
app.post('/user', auth, userAdd);
http.createServer(app).listen(3000);
If, for whatever reason, you need to server static assets from your API (a data resposity rendered to JSON, for example), Express middleware can take care of that for you.
I am not sure how we want to actually facilitate the hands-on part? Do we give people the answers to type? Do we give them "cards" with hints or related examples?
it might be good to have everyone type this, rather than just run something that was already written.
for the part where they do their own stuff, just let them run with it and ask questions if they get stuck, for a lot of them there might not be "one right answer"
the material and timeline both seem pretty solid to me.
i think you should present the code and make people type it. make sure all your deps are added to pkg/package.json
and if you want to have a stub directory or something for people to write in, go ahead and add it.
I created a Google form that @mikeal is going to send everyone coming to fill in before. We'll use that data to create a json file people will use to drive their API server. This will be more fun than a fake API and who knows, someone might create cool hacks with it.
You can see the form draft here: https://docs.google.com/forms/d/1CcUOfC3jE6KtVp1v6gsxmfTj5Li7YO5U_zzF3YXpYNE/viewform
Since we can't entirely rely on the internet work at the ranch you want to give me a date by which I should tell people they need to have filled this out before you'll pull it down and cache it locally for the session?
whoops, hit the close button, that's not right :)
First, everyone here should fill the form now!
https://docs.google.com/forms/d/1CcUOfC3jE6KtVp1v6gsxmfTj5Li7YO5U_zzF3YXpYNE/viewform
@mikeal We got 20 responses. I'd like to have at least 100. Are you going to update the nodeconf2013 npm module once you send the email out? I think a day or two after the email goes out should get us enough response to the form request.
(Ignore me while I'm reworking this comment over the next 24h)
In this session we will build a simple web services server using raw node, express, and hapi. Each provides it's own set of tradeoffs and benefits.
Start with a bare node API server:
var http = require('http');
var server = http.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
});
server.listen(8000, localhost);
Move handler method out:
var http = require('http');
var handler = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
};
var server = http.createServer(handler);
server.listen(8000, localhost);
Add basic path routing:
var http = require('http');
var dispatcher = function (req, res) {
var result;
switch(req.url) {
case '/': home(req, res); break;
case '/user': user(req, res); break;
default:
res.writeHead(404, {'Content-Type': 'application/json'});
var result = { error: 'not found' };
res.end(JSON.stringify(result));
break;
};
var home = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
};
var user = function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { id: 23123, name: 'joe' };
res.end(JSON.stringify(result));
};
var server = http.createServer(dispatcher);
server.listen(8000, localhost);
Highlight the function (req, res)
signature and discuss other basic routing facilities required in any web services framework:
Install Express in your working directory:
cd /your/working/directory
npm install express
Create an Express API server:
var express = require('express');
var http = require('http');
var app = express();
app.configure(function () {
app.use(app.router);
});
app.get('/', function (req, res) {
res.writeHead(200, {'Content-Type': 'application/json'});
var result = { hello: 'world' };
res.end(JSON.stringify(result));
});
http.createServer(app).listen(3000);
The Express response object has a method for specifically returning a JSON response. This automatically returns the appropriate HTTP header and takes care of converting your JSON to a string. The string will also be "prettified" so as to be easier for humans to read.
To see this at work, change the above GET response logic to the logic below:
app.get('/', function (req, res) {
var result = { hello: 'world' };
res.json(result);
});
As with the Node HTTP module example, we can move the handler logic into a separate function:
var express = require('express');
var http = require('http');
var app = express();
app.configure(function () {
app.use(app.router);
});
var handler = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
app.get('/', handler);
http.createServer(app).listen(3000);
We'll now add a new endpoint, a placeholder for a user data endpoint, as we did in the pure node example:
var express = require('express');
var http = require('http');
var app = express();
app.configure(function () {
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
var result = { id: 23123, name: 'joe' };
res.json(result);
};
app.get('/', home);
app.get('/user', user);
http.createServer(app).listen(3000);
Let's now make the user data endpoint useful by adding the ability to specify a user using a GET parameter.
var express = require('express');
var http = require('http');
var app = express();
app.configure(function () {
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
app.get('/', home);
app.get('/user', user);
http.createServer(app).listen(3000);
We can create logic to add a new user as well. For this we need to enable Express's parsing of HTTP POST data.
Express handles HTTP POST data parsing via "middleware". Express middleware provides request pre/post-processing of HTTP requests/responses.
(diagram)
app.configure
is where middleware can be specified using the app.use
method. To enable parsing of POST data we enable the bodyParser
middleware.
var express = require('express');
var http = require('http');
var app = express();
app.configure(function () {
app.use(express.bodyParser());
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
var userAdd = function(req, res) {
// use POST params to create user
};
app.get('/', home);
app.get('/user', user);
app.post('/user', userAdd);
http.createServer(app).listen(3000);
Custom middleware can also be created to do things like validate HTTP parameters or inject infortation into the HTTP response headers.
In the below code we add custom middleware to specify an API version.
var express = require('express');
var http = require('http');
var app = express();
var myMiddleware = function (req, res, next) {
res.setHeader('X-API-Version', '0.0.1');
return next();
};
app.configure(function () {
app.use(myMiddleware);
app.use(express.bodyParser());
app.use(app.router);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
var userAdd = function(req, res) {
// use POST params to create user
};
app.get('/', home);
app.get('/user', user);
app.post('/user', userAdd);
http.createServer(app).listen(3000);
Last, but not least, you'll add some basic security to our API using HTTP basic authentication. You add authentication to the POST endpoint that allows a user to be added.
To add this you'll create a middleware component that leverages the built-in basicAuth
middleware.
var auth = express.basicAuth(function (username, password, callback) {
var result = (username === 'myuser' && pass === 'mypass');
callback(null, result);
});
In the following code you'll add this middleware to the application and apply it to the /user
POST endpoint.
var express = require('express');
var http = require('http');
var app = express();
var myMiddleware = function (req, res, next) {
res.setHeader('X-API-Version', '0.0.1');
return next();
};
app.configure(function () {
app.use(myMiddleware);
app.use(express.bodyParser());
app.use(app.router);
});
var auth = express.basicAuth(function(username, password, callback) {
var result = (username === 'myuser' && password === 'mypass');
callback(null, result);
});
var home = function (req, res) {
var result = { hello: 'world' };
res.json(result);
};
var user = function (req, res) {
// use req.query.id to fetch user
};
var userAdd = function(req, res) {
// use POST params to create user
};
app.get('/', home);
app.get('/user', user);
app.post('/user', auth, userAdd);
http.createServer(app).listen(3000);
If, for whatever reason, you need to server static assets from your API (a data resposity rendered to JSON, for example), Express middleware can take care of that for you.
The same Express application is expressed using hapi as:
var Hapi = require('hapi');
var server = new Hapi.Server(3000);
server.ext('onPreResponse', function (request, next) {
request.response().header('X-API-Version', '0.0.1');
return next();
};
var home = function () {
var result = { hello: 'world' };
this.reply(result);
};
var user = function () {
// use this.query.id to fetch user
};
var userAdd = function() {
// use this.payload to create user
};
var validate = function (username, password, callback) {
var result = (username === 'myuser' && password === 'mypass');
callback(null, result);
};
server.auth('password', {
scheme: 'basic',
validateFunc: validate
});
server.route([
{ method: 'GET', path: '/', handler: home },
{ method: 'GET', path: '/user', handler: user },
{ method: 'POST', path: '/', handler: userAdd, config: { auth: 'pasword' } }
]);
server.start();
I'm going to rework the session to use most of the material above for the presentation part of each section, then have one slide showing a summery and giving some ideas to hack on. People can then redo the examples or load the final code sample from each section and hack on it.
Started working on the deck. Uploaded it to /slides/services along with all the fonts I use. Will finish all the planned slides later today.
@mikeal this should not be included in the npm module.
yeah, just keep it out of /pkg and it wont be in the npm package.
On Jun 17, 2013, at 4:44AM, Eran Hammer notifications@github.com wrote:
Started working on the deck. Uploaded it to /slides/services along with all the fonts I use. Will finish all the planned slides later today.
@mikeal this should not be included in the npm module.
— Reply to this email directly or view it on GitHub.
@hueniverse Ah good stuff... I'll check the slides out this evening.
@hueniverse The slides are lookin' good... I'll put some Express content in tomorrow. I'm using LibreOffice on the Mac... hopefully that won't cause any weirdness with the file.
@mcantelon I can take care of adding the express code from the plan to the deck. No worries.
reminder that all dependencies and materials you want on people's computers needs to be checked in to the package in this repo by tomorrow.
@hueniverse Awesome... thanks!
@mikeal Should I upload the tiny userbase file I have now (26 responses)? Will we be able to update later? With so few people, it doesn't seem worth using. I'm hoping more people fill it after you send the email with instructions.
You could just pass it around on USB keys or something.
Ok. So no need to include it in the module. We're good to go.
I'd like to use this ticket to plan the web services session.
The idea with this session is to pick a fairly simple web service that can show off the differences between different libraries commonly used to build web services.
For this session I'd like to use
hapi
,express
and vanilla nodehttp
.I think what will probably work best is to break the hour in to 3 parts, of 15 minutes each, with about 10 minutes flex time just in case.
How doable does this sound?
@hueniverse @mcantelon