alangpierce
commented
5 years ago Hey @ggoodman! I've actually thought about very similar ideas for how to make webpack faster when using Sucrase. (Webpack spends a bunch of time re-parsing the Sucrase output to gather information that Sucrase might have been able to provide without another parse.) But the webpack API is all centered around ASTs, so it would be a significant (but not impossible) change to get all that to work.
At the moment, I'd rather not add complexity to the Sucrase API unless there's a clear use case and it doesn't slow down other use cases. I'm certainly willing to make a Sucrase change, but I'd want to see a good prototype to motivate it. I can think of two approaches that might work here:
- Import the parser directly and try to make use of that. It's not a stable/documented API, but should work in practice.
import {parse} from "sucrase/dist/parser";should do it. You don't get an AST, but often tokens are really all you need, you just need to be a bit more careful. For example, I'm pretty sure that if you see animporttoken not followed by a(token, then the next string token is the path being imported. (The tokenizer is smart enough to know when a token is a realimporttoken rather than an object key or anything like that.) You could follow the logic inCJSImportTransformer.processto make sure you take the same general approach as Sucrase to see what the imports and exports are. - Fork Sucrase and make it expose the information that you need. It's unclear to me whether this is best done in the parser or transformer, but either might work. For the parser,
parseImportin statement.ts callsparseExprAtomto skip past the string literal, but you could capture the value then instead. You might keep a new array of information onState, similar to thescopesarray. At transform time, if you're using the imports transform, that would beCJSImportTransformer, otherwiseESMImportTransformer. Note thatESMImportTransformerdoesn't actually recognize all import/export forms because some things don't need to be transformed at all.
Let me know if you try going down one of these approaches, I'm interested to hear how it goes!
Would it be possible to provide some mechanism to obtain the list of observed imports and exports (CJS / TS / ESM) during transformation? Even better would be the ability to hook into and transform relative import filenames.
I noticed that there doesn't seem to be an intermediate AST representation, which prevents someone like me from obtaining imports / exports from that. Of course, I bet that is a big part why it is so fast.
Since this project is light-weight (in bytesize) and is fast, I can imagine it being an ideal way to smooth the interop challenges between module formats for in-browser or edge-worker use-cases. From my perspective, I'm thinking of a tool like https://next.plnkr.co/edit/ where the dependency graph is determined and executed at run-time but currently uses the behemoth that is TypeScript. Of course I love TypeScript but its current structure makes it hard to pick and chose only those pieces you need.
Looking forward to hearing your thoughts!