web-infra-dev / rspack

The fast Rust-based web bundler with webpack-compatible API 🦀️
https://rspack.dev
MIT License
10.04k stars 573 forks source link

Perf: Reuse AST passed from bultin:swc_loader to avoid double parse #8459

Open hardfist opened 4 days ago

hardfist commented 4 days ago

span is not valid after transformation

currrently Rspack will do double parse for all file processed by builtin:swc-loader, the reason Rspack do double parse is caused by swc transform will make the origin span invalid which cause it's bad for following dependency generation

we fix ast span using visit_mut_span in SpanFixerVisitor

struct SpanFixerVisitor {
   src_map_buf: SrcMapBuf
};
impl VisitMut for SpanFixerVisitor {
   fn visit_mut_span(&mut self, span: &mut Span){
             let codegen_span = self.src_map_buf.get(span);
             span.start = codegen_span.start;
             span.end = codegen_span.end;
   }
}

ast invalidation

we should mark the ast dirty | invalid, if the code is modified by following loader

reuse span in module.build

if the AST contains the valid span, then parse can reuse ast passing from builtin:swc-loader, no need to do double parse in module.build

https://github.com/web-infra-dev/rspack/blob/9112e35f97eeac430691e15c0305763fec56e890/crates/rspack_core/src/normal_module.rs#L542

GiveMe-A-Name commented 4 days ago

Except fix the span, we need pass ast from swc_loader to JavaScriptParserAndGenerator. We can use additional_data: anymap::Map<dyn CloneAny + Send + Sync> to pass ast.

// swc_loader
fn run(loader_context) {
 ....
 let mut additional_data = Default::default();
 additional_data.insert(ast);

 loader_context.finish((code, map, additional_data))
}

then get ast from additional_data

// parser_and_generator
fn parse(parse_context) {
   let ParseContext { additional_data } = parse_context;

   let ast = additional_data.get::<SwcAst>();

   ....
}
h-a-n-a commented 4 days ago

We can use additional_data: anymap::Map<dyn CloneAny + Send + Sync> to pass ast.

Voting for this solution.

Given the status-quo, AdditionalData is replaced by each loader, so there's no need to introduce the mental model to even invalidating the AST.

GiveMe-A-Name commented 3 days ago

Actually, we can't get mapping that from source span to transformed span using src_map_buf. The src_map_buf is used for generate source_map, It only maps a portion of the span's start or end positions to the corresponding source line and column numbers. We need other way.