tree-sitter / node-tree-sitter

Node.js bindings for tree-sitter
https://www.npmjs.com/package/tree-sitter
MIT License
649 stars 114 forks source link

Add async API for parsing native text buffers #11

Closed maxbrunsfeld closed 6 years ago

maxbrunsfeld commented 6 years ago

This PR adds a new asynchronous API that allows parsing superstring text buffers on background threads.

const Parser = require('tree-sitter');
const {TextBuffer} = require('superstring');
const JavaScript = require('tree-sitter-javascript');

async function test() {
  const buffer = new TextBuffer('const x = y;');
  const parser = new Parser().setLanguage(JavaScript);
  const tree = await parser.parseTextBuffer(buffer);
  console.log(tree.rootNode);
}

As with the regular parse function, parseTextBuffer can re-use an existing syntax tree:

const newTree = await parser.parseTextBuffer(buffer, oldTree);

Implementation

In parseTextBuffer, we call the TextBuffer object's C++ APIs directly. Currently, I do this by including a bunch of header files from superstring in the vendor folder of node-tree-sitter.

/cc @nathansobo

nathansobo commented 6 years ago

Pretty cool that you can interop between two native Node extensions this way. 🤘

maxbrunsfeld commented 6 years ago

Ok, I have a new, better approach. In https://github.com/atom/superstring/pull/65, I've added an API in superstring that allows other native modules (like this one) to access the contents of a TextBuffer::Snapshot directly from C++ in terms of simple primitives and STL objects, without having to call any methods from superstring.

This is also cleaner because it means I only have to vendor one small header file from superstring in this repo.