tidy compiles very slowly

[matthiaskrgr]

1) modify some code of tidy 2) run ./x.py test => for the first ~6 seconds, tidy is being built in parallel, but for the remaining ~1 minute and 9 seconds llvm is doing some sequential work.

I'm not sure what takes so long, there must be one epic cgu that it is processing or something?

[onur-ozkan]

I wanted to check the problem but couldn't reproduce it locally.

[matthiaskrgr]

   Compiling tidy v0.1.0 (/home/matthias/vcs/github/rust/src/tools/tidy)
time:   0.000; rss:   44MB ->   45MB (   +1MB)  parse_crate
time:   0.000; rss:   45MB ->   46MB (   +1MB)  setup_global_ctxt
time:   0.000; rss:   47MB ->   47MB (   +0MB)  crate_injection
time:   0.019; rss:   48MB ->   84MB (  +37MB)  expand_crate
time:   0.019; rss:   48MB ->   84MB (  +37MB)  macro_expand_crate
time:   0.000; rss:   84MB ->   84MB (   +0MB)  AST_validation
time:   0.000; rss:   85MB ->   85MB (   +0MB)  finalize_macro_resolutions
time:   0.004; rss:   85MB ->   88MB (   +4MB)  late_resolve_crate
time:   0.000; rss:   88MB ->   89MB (   +0MB)  resolve_check_unused
time:   0.000; rss:   89MB ->   89MB (   +0MB)  resolve_postprocess
time:   0.005; rss:   84MB ->   89MB (   +4MB)  resolve_crate
time:   0.001; rss:   96MB ->   96MB (   +0MB)  drop_ast
time:   0.011; rss:   89MB ->   96MB (   +7MB)  looking_for_derive_registrar
time:   0.013; rss:   89MB ->   97MB (   +9MB)  misc_checking_1
time:   0.023; rss:   97MB ->  129MB (  +32MB)  coherence_checking
time:   0.139; rss:   97MB ->  158MB (  +61MB)  type_check_crate
time:   0.143; rss:  158MB ->  201MB (  +43MB)  MIR_borrow_checking
time:   0.026; rss:  201MB ->  201MB (   +0MB)  MIR_effect_checking
time:   0.011; rss:  202MB ->  202MB (   +1MB)  module_lints
time:   0.011; rss:  202MB ->  202MB (   +1MB)  lint_checking
time:   0.004; rss:  202MB ->  202MB (   +0MB)  privacy_checking_modules
time:   0.016; rss:  201MB ->  202MB (   +1MB)  misc_checking_3
time:   0.091; rss:  202MB ->  214MB (  +12MB)  generate_crate_metadata
time:   0.084; rss:  214MB ->  220MB (   +5MB)  monomorphization_collector_graph_walk
time:   0.021; rss:  220MB ->  225MB (   +5MB)  partition_and_assert_distinct_symbols
time:   0.972; rss:  225MB ->  409MB ( +184MB)  codegen_to_LLVM_IR
time:   1.080; rss:  214MB ->  409MB ( +194MB)  codegen_crate
time:   0.016; rss:  409MB ->  355MB (  -54MB)  free_global_ctxt
time:  52.535; rss:  249MB ->  532MB ( +283MB)  LLVM_passes
time:   0.011; rss:  523MB ->  216MB ( -307MB)  join_worker_thread
time:  51.617; rss:  355MB ->  216MB ( -139MB)  finish_ongoing_codegen
time:   0.270; rss:  216MB ->  217MB (   +1MB)  link_rlib
time:   0.074; rss:  217MB ->  217MB (   +0MB)  link_binary_remove_temps
time:   0.344; rss:  216MB ->  217MB (   +1MB)  link_binary
time:   0.344; rss:  216MB ->  216MB (   +0MB)  link_crate
time:  51.961; rss:  355MB ->  216MB ( -139MB)  link
time:  53.517; rss:   32MB ->  186MB ( +154MB)  total
time:   0.000; rss:   44MB ->   45MB (   +1MB)  parse_crate
time:   0.000; rss:   46MB ->   46MB (   +1MB)  setup_global_ctxt
time:   0.000; rss:   47MB ->   48MB (   +0MB)  crate_injection
time:   0.009; rss:   48MB ->   76MB (  +28MB)  expand_crate
time:   0.000; rss:   76MB ->   76MB (   +0MB)  check_unused_macros
time:   0.009; rss:   48MB ->   76MB (  +28MB)  macro_expand_crate
time:   0.000; rss:   76MB ->   77MB (   +0MB)  AST_validation
time:   0.000; rss:   77MB ->   77MB (   +0MB)  finalize_imports
time:   0.002; rss:   77MB ->   79MB (   +2MB)  late_resolve_crate
time:   0.000; rss:   79MB ->   79MB (   +0MB)  resolve_check_unused
time:   0.000; rss:   79MB ->   80MB (   +0MB)  resolve_postprocess
time:   0.002; rss:   77MB ->   80MB (   +3MB)  resolve_crate
time:   0.000; rss:   80MB ->   80MB (   +0MB)  complete_gated_feature_checking
time:   0.001; rss:   80MB ->   82MB (   +2MB)  looking_for_entry_point
time:   0.002; rss:   80MB ->   83MB (   +3MB)  misc_checking_1
time:   0.001; rss:   83MB ->   87MB (   +4MB)  coherence_checking
time:   0.030; rss:   83MB ->  111MB (  +29MB)  type_check_crate
time:   0.023; rss:  111MB ->  123MB (  +12MB)  MIR_borrow_checking
time:   0.004; rss:  123MB ->  124MB (   +1MB)  MIR_effect_checking
time:   0.002; rss:  124MB ->  126MB (   +2MB)  module_lints
time:   0.003; rss:  124MB ->  126MB (   +2MB)  lint_checking
time:   0.003; rss:  124MB ->  126MB (   +2MB)  misc_checking_3
time:   0.061; rss:  126MB ->  153MB (  +27MB)  monomorphization_collector_graph_walk
time:   0.010; rss:  153MB ->  155MB (   +2MB)  partition_and_assert_distinct_symbols
time:   0.000; rss:  156MB ->  159MB (   +3MB)  write_allocator_module
time:   0.437; rss:  165MB ->  288MB ( +122MB)  codegen_to_LLVM_IR
time:   0.513; rss:  126MB ->  288MB ( +161MB)  codegen_crate
time:   0.012; rss:  288MB ->  238MB (  -50MB)  free_global_ctxt
time:   1.769; rss:  189MB ->  273MB (  +85MB)  LLVM_passes
time:   0.001; rss:  228MB ->  225MB (   -2MB)  join_worker_thread
time:   1.367; rss:  238MB ->  226MB (  -12MB)  finish_ongoing_codegen
time:   0.879; rss:  226MB ->  226MB (   +0MB)  run_linker
time:   0.884; rss:  226MB ->  226MB (   +1MB)  link_binary
time:   0.884; rss:  226MB ->  226MB (   +1MB)  link_crate
time:   2.251; rss:  238MB ->  226MB (  -12MB)  link
time:   2.859; rss:   32MB ->  152MB ( +120MB)  total
    Finished `release` profile [optimized + debuginfo] target(s) in 56.66s

[matthiaskrgr]

:thinking:

[matthiaskrgr]

aha lol! it boils down to issues.txt, an array of ~4K static &strs which is being inlined into the code it seems. When I cut the file down to just a handful of items, compilation time reduces to 7 seconds :upside_down_face: https://github.com/rust-lang/rust/blob/3d5528c287860b918e178a34f04ff903325571b3/src/tools/tidy/src/ui_tests.rs#L108

cc @workingjubilee I guess?

[onur-ozkan]

Can you check if #123339 reduces this slowness?

[matthiaskrgr]

It does indeed. Build completed successfully in 0:00:06

[workingjubilee]

Ah, but if people give the tests meaningful names, it will reduce the build time. :relieved: