search

open-i18n / rust-unic

UNIC: Unicode and Internationalization Crates for Rust
https://crates.io/crates/unic
Other
234 stars 24 forks source link

unic-ucd-name takes a long time to compile (nightly) #199

Open CAD97 opened 6 years ago

CAD97 commented 6 years ago

It's definitely degraded from when it was first merged. I'm trying to make a self-contained case to submit to the rust-lang repository as an issue in compile times.

cargo +nightly rustc --package=unic-ucd-name -- -Z time-passes

Output ``` Compiling unic-ucd-name v0.6.0 (file:///D:/Christopher/Documents/Code/Rust/rust-unic/unic/ucd/name) time: 0.001; rss: 18MB parsing time: 0.000; rss: 18MB recursion limit time: 0.000; rss: 18MB crate injection time: 0.000; rss: 18MB plugin loading time: 0.000; rss: 18MB plugin registration time: 0.000; rss: 18MB background load prev dep-graph time: 0.582; rss: 102MB expansion time: 0.000; rss: 102MB maybe building test harness time: 0.009; rss: 102MB maybe creating a macro crate time: 0.032; rss: 102MB creating allocators time: 0.011; rss: 102MB AST validation time: 0.122; rss: 120MB name resolution time: 0.026; rss: 120MB complete gated feature checking time: 0.000; rss: 120MB blocked while dep-graph loading finishes time: 0.150; rss: 168MB lowering ast -> hir time: 0.064; rss: 168MB early lint checks time: 0.183; rss: 176MB indexing hir time: 0.000; rss: 118MB load query result cache time: 0.000; rss: 118MB looking for entry point time: 0.000; rss: 118MB looking for plugin registrar time: 0.010; rss: 118MB loop checking time: 0.040; rss: 118MB static item recursion checking time: 0.040; rss: 131MB attribute checking time: 0.061; rss: 139MB stability checking time: 0.197; rss: 186MB type collecting time: 0.002; rss: 186MB outlives testing time: 0.001; rss: 186MB impl wf inference time: 0.024; rss: 193MB coherence checking time: 0.001; rss: 193MB variance testing time: 0.286; rss: 233MB wf checking time: 4.539; rss: 306MB item-types checking time: 0.042; rss: 311MB item-bodies checking time: 123.153; rss: 532MB const checking # !!!!!!!!!!!!!!!!!!!! time: 0.183; rss: 533MB privacy checking time: 0.026; rss: 533MB intrinsic checking time: 0.045; rss: 535MB match checking time: 0.011; rss: 535MB liveness checking time: 0.590; rss: 544MB borrow checking time: 0.024; rss: 546MB MIR borrow checking time: 0.007; rss: 546MB MIR effect checking time: 0.048; rss: 547MB death checking time: 0.000; rss: 547MB unused lib feature checking time: 0.164; rss: 548MB lint checking time: 0.000; rss: 548MB resolving dependency formats time: 1.097; rss: 597MB write metadata time: 0.083; rss: 598MB translation item collection time: 0.001; rss: 598MB codegen unit partitioning time: 0.001; rss: 602MB llvm function passes [49a7n47po4ttqjl7] time: 0.001; rss: 604MB llvm module passes [49a7n47po4ttqjl7] time: 0.017; rss: 623MB codegen passes [49a7n47po4ttqjl7] time: 0.000; rss: 642MB llvm function passes [2ny9ynlpevlhfa8x] time: 0.000; rss: 642MB llvm module passes [2ny9ynlpevlhfa8x] time: 0.000; rss: 642MB llvm function passes [16u6js6g0l3k1ic6] time: 0.000; rss: 642MB llvm module passes [16u6js6g0l3k1ic6] time: 0.000; rss: 642MB llvm function passes [2jqywn86b2gsqohu] time: 0.000; rss: 642MB llvm module passes [2jqywn86b2gsqohu] time: 0.000; rss: 644MB llvm function passes [3fh06zvk9bptbvw9] time: 0.000; rss: 644MB llvm module passes [3fh06zvk9bptbvw9] time: 0.000; rss: 644MB llvm function passes [2lyh15q6cjwzy18c] time: 0.000; rss: 644MB llvm module passes [2lyh15q6cjwzy18c] time: 0.000; rss: 644MB llvm function passes [9fcb3syd3ne5k0n] time: 0.000; rss: 644MB llvm module passes [9fcb3syd3ne5k0n] time: 0.000; rss: 644MB llvm function passes [8xzrsc1ux72v29j] time: 0.000; rss: 644MB llvm module passes [8xzrsc1ux72v29j] time: 0.011; rss: 646MB codegen passes [2jqywn86b2gsqohu] time: 0.017; rss: 646MB codegen passes [2ny9ynlpevlhfa8x] time: 0.000; rss: 646MB llvm function passes [4ypvbwho0bu5tnww] time: 0.000; rss: 646MB llvm module passes [4ypvbwho0bu5tnww] time: 0.000; rss: 646MB llvm function passes [9elsx31vb4it187] time: 0.000; rss: 646MB llvm module passes [9elsx31vb4it187] time: 0.015; rss: 647MB codegen passes [2lyh15q6cjwzy18c] time: 0.013; rss: 647MB codegen passes [9fcb3syd3ne5k0n] time: 0.015; rss: 647MB codegen passes [3fh06zvk9bptbvw9] time: 0.000; rss: 647MB llvm function passes [98g0d9x8aw3akpe] time: 0.000; rss: 647MB llvm module passes [98g0d9x8aw3akpe] time: 0.000; rss: 647MB llvm function passes [2xnvmuhjbhd7vxcm] time: 0.000; rss: 647MB llvm module passes [2xnvmuhjbhd7vxcm] time: 0.633; rss: 647MB translate to LLVM IR time: 0.000; rss: 647MB assert dep graph time: 0.000; rss: 647MB llvm function passes [1y16o1qfye96o7m0] time: 0.000; rss: 647MB llvm module passes [1y16o1qfye96o7m0] time: 0.029; rss: 647MB codegen passes [16u6js6g0l3k1ic6] time: 0.010; rss: 648MB codegen passes [9elsx31vb4it187] time: 0.000; rss: 648MB llvm function passes [524bze3gcv99ucga] time: 0.000; rss: 648MB llvm module passes [524bze3gcv99ucga] time: 0.013; rss: 648MB codegen passes [4ypvbwho0bu5tnww] time: 0.000; rss: 648MB llvm function passes [48721dc4k5qxei0u] time: 0.000; rss: 648MB llvm module passes [48721dc4k5qxei0u] time: 0.000; rss: 648MB llvm function passes [4yh8x2b62dcih00t] time: 0.000; rss: 648MB llvm module passes [4yh8x2b62dcih00t] time: 0.010; rss: 648MB codegen passes [98g0d9x8aw3akpe] time: 0.009; rss: 649MB codegen passes [1y16o1qfye96o7m0] time: 0.011; rss: 650MB codegen passes [2xnvmuhjbhd7vxcm] time: 0.011; rss: 650MB codegen passes [524bze3gcv99ucga] time: 0.009; rss: 650MB codegen passes [48721dc4k5qxei0u] time: 0.010; rss: 651MB codegen passes [8xzrsc1ux72v29j] time: 0.009; rss: 651MB codegen passes [4yh8x2b62dcih00t] time: 0.072; rss: 653MB llvm function passes [2iv7jmandrgcbb7e] time: 0.000; rss: 653MB llvm module passes [2iv7jmandrgcbb7e] time: 0.725; rss: 690MB persist query result cache time: 0.167; rss: 746MB persist dep-graph time: 0.892; rss: 746MB serialize dep graph time: 2.750; rss: 746MB translation time: 1.143; rss: 242MB codegen passes [2iv7jmandrgcbb7e] time: 2.453; rss: 206MB LLVM passes time: 0.002; rss: 204MB serialize work products time: 0.433; rss: 205MB linking Finished dev [unoptimized + debuginfo] target(s) in 135.69 secs ```

I think we may have stumbled across a quadratic time cost in the string interner, and my playing around with ways of formulating it back in #103/#125 stumbled across a case that at the time didn't do so bad for some reason. (1.25 Nightly)
CAD97 commented 6 years ago

I modified the name tables to use statics instead of consts, and compile time got somewhat worse, moving from const checking to borrow checking.

Time Passes ``` Compiling unic-ucd-name v0.6.0 (file:///D:/Christopher/Documents/Code/Rust/rust-unic/unic/ucd/name) time: 0.001; rss: 17MB parsing time: 0.000; rss: 18MB recursion limit time: 0.000; rss: 18MB crate injection time: 0.000; rss: 18MB plugin loading time: 0.000; rss: 18MB plugin registration time: 0.110; rss: 91MB background load prev dep-graph time: 0.687; rss: 159MB expansion time: 0.000; rss: 159MB maybe building test harness time: 0.011; rss: 159MB maybe creating a macro crate time: 0.041; rss: 159MB creating allocators time: 0.014; rss: 159MB AST validation time: 0.121; rss: 177MB name resolution time: 0.033; rss: 177MB complete gated feature checking time: 0.000; rss: 177MB blocked while dep-graph loading finishes time: 0.169; rss: 248MB lowering ast -> hir time: 0.079; rss: 248MB early lint checks time: 0.242; rss: 259MB indexing hir time: 0.038; rss: 243MB load query result cache time: 0.000; rss: 243MB looking for entry point time: 0.000; rss: 243MB looking for plugin registrar time: 0.017; rss: 243MB loop checking time: 0.043; rss: 243MB static item recursion checking time: 0.050; rss: 255MB attribute checking time: 0.069; rss: 262MB stability checking time: 0.212; rss: 303MB type collecting time: 0.002; rss: 303MB outlives testing time: 0.000; rss: 303MB impl wf inference time: 0.012; rss: 309MB coherence checking time: 0.001; rss: 309MB variance testing time: 0.363; rss: 355MB wf checking time: 7.722; rss: 436MB item-types checking time: 0.045; rss: 441MB item-bodies checking time: 0.510; rss: 481MB const checking time: 0.263; rss: 481MB privacy checking time: 0.027; rss: 481MB intrinsic checking time: 0.071; rss: 483MB match checking time: 0.013; rss: 483MB liveness checking time: 133.775; rss: 664MB borrow checking # !!!!!!!!!!!!!!!!!!!! time: 0.031; rss: 666MB MIR borrow checking time: 0.007; rss: 666MB MIR effect checking time: 0.053; rss: 666MB death checking time: 0.000; rss: 666MB unused lib feature checking time: 0.229; rss: 666MB lint checking time: 0.000; rss: 666MB resolving dependency formats time: 0.097; rss: 673MB write metadata time: 0.736; rss: 691MB translation item collection time: 0.081; rss: 755MB codegen unit partitioning time: 0.001; rss: 794MB llvm function passes [49a7n47po4ttqjl7] time: 0.000; rss: 795MB llvm module passes [49a7n47po4ttqjl7] time: 0.000; rss: 795MB llvm function passes [2iv7jmandrgcbb7e] time: 0.000; rss: 795MB llvm module passes [2iv7jmandrgcbb7e] time: 0.000; rss: 796MB llvm function passes [2ny9ynlpevlhfa8x] time: 0.000; rss: 797MB llvm module passes [2ny9ynlpevlhfa8x] time: 0.001; rss: 796MB llvm function passes [16u6js6g0l3k1ic6] time: 0.000; rss: 796MB llvm module passes [16u6js6g0l3k1ic6] time: 0.000; rss: 797MB llvm function passes [2jqywn86b2gsqohu] time: 0.000; rss: 797MB llvm module passes [2jqywn86b2gsqohu] time: 0.001; rss: 798MB llvm function passes [3fh06zvk9bptbvw9] time: 0.000; rss: 798MB llvm module passes [3fh06zvk9bptbvw9] time: 0.000; rss: 799MB llvm function passes [9fcb3syd3ne5k0n] time: 0.000; rss: 799MB llvm module passes [9fcb3syd3ne5k0n] time: 0.016; rss: 799MB codegen passes [2iv7jmandrgcbb7e] time: 0.014; rss: 799MB codegen passes [2ny9ynlpevlhfa8x] time: 0.000; rss: 799MB llvm function passes [8xzrsc1ux72v29j] time: 0.000; rss: 799MB llvm module passes [8xzrsc1ux72v29j] time: 0.000; rss: 799MB llvm function passes [4ypvbwho0bu5tnww] time: 0.000; rss: 799MB llvm module passes [4ypvbwho0bu5tnww] time: 0.026; rss: 799MB codegen passes [49a7n47po4ttqjl7] time: 0.015; rss: 799MB codegen passes [16u6js6g0l3k1ic6] time: 0.013; rss: 799MB codegen passes [2jqywn86b2gsqohu] time: 0.000; rss: 799MB llvm function passes [9elsx31vb4it187] time: 0.000; rss: 799MB llvm module passes [9elsx31vb4it187] time: 0.000; rss: 799MB llvm function passes [524bze3gcv99ucga] time: 0.000; rss: 799MB llvm module passes [524bze3gcv99ucga] time: 0.000; rss: 799MB llvm function passes [1y16o1qfye96o7m0] time: 0.000; rss: 799MB llvm module passes [1y16o1qfye96o7m0] time: 0.071; rss: 802MB llvm function passes [1xct7l288j7387hl] time: 0.000; rss: 802MB llvm module passes [1xct7l288j7387hl] time: 0.270; rss: 816MB codegen passes [1y16o1qfye96o7m0] time: 0.655; rss: 816MB translate to LLVM IR time: 0.000; rss: 816MB assert dep graph time: 0.278; rss: 816MB codegen passes [9elsx31vb4it187] time: 0.291; rss: 816MB codegen passes [9fcb3syd3ne5k0n] time: 0.274; rss: 816MB codegen passes [524bze3gcv99ucga] time: 0.281; rss: 816MB codegen passes [4ypvbwho0bu5tnww] time: 0.000; rss: 816MB llvm function passes [4yh8x2b62dcih00t] time: 0.000; rss: 816MB llvm module passes [4yh8x2b62dcih00t] time: 0.292; rss: 816MB codegen passes [3fh06zvk9bptbvw9] time: 0.619; rss: 857MB codegen passes [8xzrsc1ux72v29j] time: 0.329; rss: 857MB codegen passes [4yh8x2b62dcih00t] time: 0.848; rss: 916MB persist query result cache time: 0.144; rss: 956MB persist dep-graph time: 0.992; rss: 956MB serialize dep graph time: 3.080; rss: 956MB translation time: 1.030; rss: 324MB codegen passes [1xct7l288j7387hl] time: 1.829; rss: 277MB LLVM passes time: 0.005; rss: 277MB serialize work products time: 0.176; rss: 278MB linking Finished dev [unoptimized + debuginfo] target(s) in 149.88 secs ```
diff ```diff diff --git a/gen/src/writer/ucd/name.rs b/gen/src/writer/ucd/name.rs index 180ee9b..cd8484c 100644 --- a/gen/src/writer/ucd/name.rs +++ b/gen/src/writer/ucd/name.rs @@ -47,7 +47,7 @@ fn emit_name_tables(dir: &Path) { for piece in values.iter() { writeln!( values_contents, - "const {}: &str = \"{}\";", + "static {}: &str = \"{}\";", piece.replace('-', "_"), piece ).unwrap(); @@ -64,7 +64,7 @@ fn emit_name_tables(dir: &Path) { record .pieces .iter() - .map(|s| s.replace('-', "_")) + .map(|s| format!("&{}", s.replace('-', "_"))) .collect::>() .join(", ") ) diff --git a/unic/ucd/name/src/name.rs b/unic/ucd/name/src/name.rs index 05ea96f..5d249c0 100644 --- a/unic/ucd/name/src/name.rs +++ b/unic/ucd/name/src/name.rs @@ -12,7 +12,7 @@ use core::fmt; #[derive(Copy, Clone, Debug, Ord, PartialOrd, Eq, PartialEq, Hash)] pub struct Name { - pieces: &'static [&'static str], + pieces: &'static [&'static &'static str], } #[cfg_attr(feature = "clippy", allow(len_without_is_empty))] @@ -47,5 +47,5 @@ impl fmt::Display for Name { mod data { use unic_char_property::tables::CharDataTable; include!("../tables/name_values.rsd"); - pub const NAMES: CharDataTable<&[&str]> = include!("../tables/name_map.rsv"); + pub static NAMES: CharDataTable<&[&&str]> = include!("../tables/name_map.rsv"); } diff --git a/unic/ucd/name/tables/name_map.rsv b/unic/ucd/name/tables/name_map.rsv index c002b09..fb42443 100644 --- a/unic/ucd/name/tables/name_map.rsv +++ b/unic/ucd/name/tables/name_map.rsv @@ -1,31527 +1,31527 @@ -// omitted (generated) +// omitted (generated) diff --git a/unic/ucd/name/tables/name_values.rsd b/unic/ucd/name/tables/name_values.rsd index 454944e..264530d 100644 --- a/unic/ucd/name/tables/name_values.rsd +++ b/unic/ucd/name/tables/name_values.rsd @@ -1,13767 +1,13767 @@ -// omitted (generated) +// omitted (generated) ```

CAD97 commented 6 years ago

See https://github.com/rust-lang/rust/issues/48009 for the upstream issue report. Turns out this is only on nightly (I thought my default at the time was stable but it was nightly).

ishitatsuyuki commented 6 years ago

Some people at the miri thread has suggested to use an binary file for storing the items, then loading it with include_bytes!. This way it should be least stressing to the compiler.

Maybe try using some serde-based format (which doesn't need a separate schema file) or Cap'n Proto (which is mmap-friendly) to store the data?

ishitatsuyuki commented 6 years ago

I decided to work on a PR using Cap'n Proto. Stay tuned.

ishitatsuyuki commented 6 years ago

Giving up for now due to https://github.com/capnproto/capnproto-rust/issues/71 and extensive no_std usage inside this crate.

CAD97 commented 6 years ago

....I am so confused. The compile time unregressed on the latest nightly (27a046e93 2018-02-18) and is fine on stable/beta (~19s) / nightly (~16s) currently...