Remove feed() and add remaining()

[tjvr]

Closes #43.

[nathan]

What is remaining() good for?

[tjvr]

It's supposed to help implement the streaming strategies in #43. But now you mention it, I suppose you can workaround this by taking token.index + token.size, and keeping a copy of the buffer yourself.

[nathan]

So if you're using streams you'd now do the following?

const lexer = moo.compile({ … })

let state
fs.createReadStream(INPUT)
.pipe(split(…))
.pipe(new Transform({
  readableObjectMode: true,
  transform(chunk, _, cb) {
    lexer.reset(chunk.toString(), state)
    for (const t of lexer) this.push(t)
    state = lexer.save()
    cb()
  },
})

EDIT: state

[tjvr]

Yes, although you need to save() state between chunks and restore it.