port usage.md

[pelger]

Port usage.md to node proto-code, so we can see what it's like

[mcollina]

I already did my bit, please guys discuss!

[AdrianRossouw]

sorry. i was running around on other things.

I'm wondering if we shouldn't represent this as a duplex stream?

[mcollina]

In theory I'm ok, but I can't see how.

[AdrianRossouw]

There's many libs around, i think dominic had his hand in some of them too.

https://github.com/dominictarr/stream-spec#duplex

I'll read up this weekend.

[mcollina]

I didn't express me well, sorry. It's not how in code (I wrote ton of stream code myself), it's how we can match something we write to something we read from the duplex stream.

[mcollina]

I've recently stumbled upon oban, an HTTP router based on highland. I think this is what @Vertice would love for Graft. The question is: would we use that approach also for jschan, or we try to loosely follow libchan's API?

[AdrianRossouw]

I saw oban too! I've added it to the pitch/inspiration document.

I think jschan needs to stay as close to the official API as possible, and be something we can build something that is more accessible and amenable to general usage.

In my mind, the difference in scope of jschan and graft is similar to the split between http and express.js. jschan is how things talk, graft is how you tell them what to say.

virtualized streams

Gulp's API is also more of an inspiration than Highland, but I think they share important properties that might be good for jschan.

The vinyl virtual format seems like a powerful abstraction to deal with the embedded channels in jschan, and I suspect useful to get as close to the official API as possible.

being asynchronous pattern agnostic

Both Gulp and Highland allow you to use streams, callbacks, promises and (i think) es6 generators independently in their stream processing functions This should mean easier integration into existing codebases.

[mcollina]

I did some more research into how libchan works, and here it is from the protocol docs:

A channel is uni-directional: messages can only flow in one direction. So channels are more similar to pipes than to sockets.

I would say that we can send a Message through a channel, but we will get the answer through the Message itself, like so:

var jschan = require('jschan')
    , chan = jschan.memChan()
    , msg  = jschan.msg({ some: data })

msg.on('response', function(res) {
   // do something
})

chan.write(msg)

[AdrianRossouw]

That seems to be sufficient to get here:

jschan({some: data})            // send message
     .pipe(responseHandler)  // can return another stream

[mcollina]

@Vertice I don't get how the client can receive a response using the above pattern.

[AdrianRossouw]

Sorry about that, it's too tiny a snippet of code to see anything from.

I think that the abstraction you had about using the messages as response objects is great, and it starts to make me see what a graft layer could look like.

I think you should go with that for now, and we can iterate on top of that. There are some questions I wanted to ask you tho, so could we jump on skype?

[AdrianRossouw]

i really like the message.on(‘response’) pattern because I think you could build a convincing stream wrapper around it.

my current mental model

The code snippet I showed above is how I am making use of mikael's request.

    request.get(‘http://example.com:3000’, {json:myObject})
        .pipe(fs.createWriteStream('./filename.ext'));

This has the companion pattern on the server side, where you have to do:

     http.createServer().listen(3000);

The libchan model

This breaks in the libchan model, because the pipes are unidirectional, and there's no way to actually respond to a request.

Instead what I think needs to happen, is that each request() will have to do a listen() of it's own, using the msg identifier.

So the pseudocode looks (a little bit at least) like this in my mind:

function request(args.,., cb) {
    var msg = createServer(channel)

    // set up args, etc.

    msg.listen(channel); // assigns id

    channel.send(msg, function(msg) {
        cb(msg);
    });

    return msg;
}

Can you set up the in/out to different channels? Or have multiple channels acting concurrently?

[mcollina]

Wait, a channel is unidirectional but the server can send a reply through. All is handled by libchan through SPDY stream identifiers. We'll see what I came up with when I start implemting #4.

[mcollina]

Their interface changed somehow: https://github.com/docker/libchan/pull/38.

I have to get through it to grasp it fully.

[mcollina]

My updated proposal:

var jschan  = require('jschan');
var session = jschan.memorySession();

session.on('channel', function server(chan) {
  chan.on('request', function(req) {
    req.reply(req.data)
  })
})

function client() {
  var chan = session.sendChannel();
  var msg  = jschan.msg({ hello: 'world' });

  msg.on('response', function(res) {
    console.log('response', res.data);
  });

  chan.send(msg);
}

client();

[mcollina]

I've pushed a very basic implementation of my example above only in memory :dancers:.

Tomorrow I'll add some unit tests, and maybe the SPDY transport.

@rjrodger do you have an opinion on this interface?

@pelger @Vertice may I have the rights to push to NPM?

[mcollina]

I understood libchan better, and I think I might have made some mistakes in the porting of the API. I'll update ASAP.

[mcollina]

See https://github.com/docker/libchan/issues/45

[mcollina]

var jschan  = require('jschan');
var session = jschan.memorySession();

session.on('channel', function server(chan) {
  chan.on('data', function(msg) {
    var returnChannel  = msg.returnChannel;

    returnChannel.write({ hello: 'write' });
  })
})

function client() {
  var chan = session.sendChannel();
  var ret  = chan.createSubChannel('in');

  ret.on('data', function(res) {
    console.log('response', res);
  });

  chan.write({ returnChannel: ret });
}

client();

Here was the inspiration of this: https://github.com/dmcgowan/libchan/blob/fd5134755a480d1238549a44d358549ca03f6ba9/http2/session_test.go#L36-L44

[mcollina]

I updated the current API to match this, as the PROTOCOL is fixed.

[mcollina]

Closing as it's done in current master.