chrisdickinson / inflate

pure javascript inflate implemented as a through stream
http://npm.im/inflate
33 stars 6 forks source link

change core API #2

Open creationix opened 11 years ago

creationix commented 11 years ago

Instead of the core being a push-filter, it would be much easier and simpler for me to consume if it instead exported a byte oriented API:

var inf = inflate();

// This writes a single byte to the inflate engine and returns true if I should write more.
// This way I never send too many bytes and it's easier to interface with my
// state machine that's already parsing a byte at a time.
var more = inf.write(byte)

// Data will buffer up inside inflate till I ask for it.
var data = inf.flush()

// Same as before, resets the internal state-machine
inf.recycle();

https://github.com/creationix/git-parse-pack/blob/master/inflate.js

I'm not dead-set on the flush API. If you prefer, I could pass in an onData callback to inflate(onData) instead and create my own flush wrapper. I was just worried that calling onData too many times would hurt performance. Better for me to call in when I need all the data.

creationix commented 11 years ago

Actually, after thinking about it, it would be even faster if you instead returned the next state like my machine does. Then I can tightly integrate your state machine with mine and skip a couple function calls per byte.

var inf = inflate();

// init is both recycle and returns the start state as a function.
var state = inf.init();

for (var i = 0, l = chunk.length; state && i < l; i++) {
  state = state(chunk[i]);
}
var output = inf.flush();
if (!state) {
  // do stuff and then later
  state = inf.init();
}

// When do don't want anymore bytes, state will be falsy
creationix commented 11 years ago

Also I found a couple places where I needed to modify your min.js to work. (probably things you missed while refactoring it)

84c84
<     // return stream
---
>     return stream
718c718
<     emit(null, binary.from(vals))
---
>     emit(binary.from(vals))
creationix commented 11 years ago

Actually, instead of returning me a falsy state, how about I hand you the next state in my machine. When you are done consuming the data, simply return me back my state and I won't need any conditionals inside the hot inner loop.

I will still need some way to get data out of inflate. I still think the ideal interface is something where I ask for a flush and get all the pending data. That way I ask for it when it's optimal for me.

var inflate = require('inflate');

var inf = inflate();

// Pass in my next state for inflate to send back when it's done.
// Calling start also resets/recycles the internal state in the inflate instance.
var state = inf.start(myNextState);
// My inner loop will look like
    state = state(byte);
// Then when I want to pull out data
var data = inf.flush();