...Streams?

[btford]

/cc @dominictarr

After much investigation, node streams seem sort of like the right model. I say "sort of" because I just read all the things about them, and worry that this is a solution looking for a problem type deal.

Notes/Observations

• Being able to compose streams with .pipe() is perfect. You aren't coupled to either end of some pipeline.
• Having pipes convert from one type of stream to another is nifty. I could potentially see some transforms operating on Mozilla-style ASTs and some on the uglify AST. AST interop is much easier thanks to this.
• node-transfuse
  • This is cool. Seems like the analog to falafel & co.

    Questions

• Is there a streaming JavaScript parser?
  • I found a few implementations for parsing JSON as a stream but that's not the same. Once parsed, the AST is JSON, and streaming the full thing is easy. Would just be nice to avoid that buffering.
  • Does such a thing even make sense?
  • even if 'yes,' such a thing would seemingly only benefit large files.
  • files should be kept files small and focused, yo.
• Is there a sane way to queue objects in a stream? *Is encoding/decoding JSON the way to go?
  • if so, is the overhead of encoding/decoding JSON all the time even worth it?
  • it seems like this would result in re-tokenizing or re-constructing the parse tree in each 'pass'.

[dominictarr]

it might be better to have a pattern that embraced the tree structure of source code, I.e. that of a tree.

I can think of some prior art that is already used to traverse and manipulate trees, https://npmjs.org/package/traverse https://npmjs.org/package/burrito and https://npmjs.org/package/falafel and of course, our old favorite https://github.com/jquery