How to walk/visit the ast

[jnavb]

First of all thank you for this package, I've been using it for the last year and is awesome.

My use case is extracting all the Identifier tokens of an expression in order to build a dependency graph.

I'm making a custom hook for that but I don't know if such feature is feasible with the current hooks. Tried after-expression, gobble-token and after-token with no luck:

type Plugin = Parameters<typeof registerPlugin>[0];
type Jsep = typeof jsep;
const dependencies = [];

const plugin: Plugin = {
  name: "the plugin",
  init: function (jsep: Jsep) {
    jsep.hooks.add("after-token", function myPlugin(env) {
      (env as any).node.type === 'Identifier' && dependencies.push(env.node)
    });
  } as Plugin['init']
};

I found an old comment that talk about this. Nonetheless, I would like to avoid do the traverse again for performance reasons.

[jnavb]

After reviewing the codebase, If I'm not mistaken I think this could not be done with the current state of the package.

A new gobble-identifier hook could do the job, would you be open to merge the PR with such changes?

[6utt3rfly]

@jnavb - we originally had more hooks, but we scaled them back to avoid impacting performance too much and since all of the current plugins were able to work with what's there now. There could feasibly be after-literal, after-identifier, after-member, after-args, after-group, after-array, (and maybe some before hooks there too), but at the time there didn't seem to be a need for so many hooks.

You could potentially make your requirement work from one of the existing hooks, if you handle various node types. This quick example would pull all identifiers out of the entire tree at the end

function getIdentifiers(node) {
  if (node.type === 'Identifier') {
    return [node.name];
  }
  return Object.values(node)
    .reduce((arr, v) => (v && typeof v === 'object') ? [...arr, ...getIdentifiers(v)] : arr, []);
}

jsep.hooks.add("after-all", function myPlugin(env) {
  console.log('identifiers', getIdentifiers(env.node));
});

Maybe I'm misunderstanding if your need is to tap into the AST as it's being built for accumulating results (versus manipulating/expanding the parsing)? Could you also provide a few example expressions with what you're looking to achieve?

[jnavb]

@6utt3rfly - that's exactly what I wanted, ideally I would want the next response:

const [ast, identifiers] = parse('a.propA + b.propB + await fn()');
console.log(identifiers) // [a, propA, b, propB, fn]

I can see your solution is working but that would make a complete traverse again, actually is there any difference doing that outside the parser?

// No plugins involved
const ast = parse('a.propA + b.propB + await fn()');
getIdentifiers(ast);

In order to do the work while the parsing is running I only see as solution adding the new hook after-identifier. I would like to avoid forking the project so please let me know if I missed something and it could be achieved in other way 🙂

[6utt3rfly]

@jnavb - you're right that you could traverse the tree without the hook by simply using the ast result. Traversing the tree is much faster than parsing the string (which is why ASTs are used), and would probably be faster than adding more hooks, if that is your concern. There are also packages for traversing, like estree-walker.

[LeaVerou]

Yeah, a tree walker shouldn't need extra hooks, since it should operate on the AST result. We could provide a plugin with a tree walker that provides hooks as well, so that plugins can modify it in case they modify something that would affect how walking is done. Similarly, a customizable serializer plugin might also be useful. I'd say both should be relatively low priority, since they are pretty easy to implement by anyone using JSEP.

[jnavb]

@6utt3rfly @LeaVerou - agree this should not be a priority (at least from a user perspective). I would definitely try that package or build my own walker.

I still think that for my use case it would be much faster having a new identifier hook and deleting the calls to the rest of them than traversing the tree again but maybe the improvement gain is trivial (For context, I make the parsing of 20~200 scripts at page initialisation time so every improvement is welcome).

I'll close the issue since all questions were answered, thank you both!