juanjoDiaz / streamparser-json

Streaming JSON parser in Javascript for Node.js and the browser
MIT License
138 stars 11 forks source link

Add support for WHATWG streams #6

Closed juanjoDiaz closed 1 year ago

tobiasfuhlroth commented 2 years ago

Hi @juanjoDiaz

First of all thank you very much for writing and sharing this nice library – much appreciated!

I'm not sure if this is the right place, as I don't know how you intended to accomplish the integration with WHATWG streams.

But as I'm currently working with streams I gladly share what I came up with.

import { JSONParser } from '@streamparser/json'

class JSONParserTransformer {
    constructor (parserOptions) {
        this.parser = new JSONParser(parserOptions)
        this.parser.onValue = this.onValue.bind(this)
    }

    start (controller) {
        this.controller = controller
    }

    transform (chunk) {
        this.parser.write(chunk)
    }

    onValue (value, key, parent, stack) {
        this.controller.enqueue({ value, key, parent, stack })
    }
}

Using one of your examples, this could then be used like:

  const response = await fetch('http://example.com/');
  const reader = response.body.pipeThrough(new TransformStream(new JSONParserTransformer())).getReader()
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    console.log(value); // Do something interesting with the parsed data.
  }

Let me know if this is something that you're interested in and I would be happy to discuss the integration and create a PR.

Thanks again!

juanjoDiaz commented 2 years ago

Hi @tobiasfuhlroth ,

I actually have this implemented since ages ago. Unfortunately, I want both interfaces (plain js and whatwg stream) to be npm workspaces under this monorepo and I haven't been able to get typescript to work 🙁

juanjoDiaz commented 1 year ago

Done in v0.11 as a separate package: https://www.npmjs.com/package/@streamparser/json-whatwg