Cargo bench eats excessive memory on WSL2 ubuntu

[kigawas]

This issue supersedes https://github.com/rust-lang/cargo/issues/8961

TL;DR

This can be worked around by setting CARGO_BUILD_JOBS.

Description

Briefly, cargo bench eats excessive memory on WSL2 with a 10-core 20-thread i9-10900K Intel CPU and 32GB memory, due to:

1. Since the number of CPUs are 20, there will be 20 rustc processes and each process would take more than 2GB memory.

2. Memory gets drained, rustcs get killed.

   Caused by:
   process didn't exit successfully: `rustc --crate-name simple --edition=2018 bench/simple.rs --error-format=json --json=diagnostic-rendered-ansi --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no --cfg test --cfg 'feature="aes-gcm"' --cfg 'feature="pure"' --cfg 'feature="typenum"' -C metadata=5b7a3f7d5f63fca0 -C extra-filename=-5b7a3f7d5f63fca0 --out-dir /home/ryan/works/rs/eciesrs/target/release/deps -L dependency=/home/ryan/works/rs/eciesrs/target/release/deps --extern aes_gcm=/home/ryan/works/rs/eciesrs/target/release/deps/libaes_gcm-b689cfd45f541159.rlib --extern criterion=/home/ryan/works/rs/eciesrs/target/release/deps/libcriterion-92424be6afcb1e28.rlib --extern ecies=/home/ryan/works/rs/eciesrs/target/release/deps/libecies-b1dd2b46a70a3818.rlib --extern futures_util=/home/ryan/works/rs/eciesrs/target/release/deps/libfutures_util-31a01fcbcefd69d3.rlib --extern hex=/home/ryan/works/rs/eciesrs/target/release/deps/libhex-f0ac4054f3e10cf0.rlib --extern hkdf=/home/ryan/works/rs/eciesrs/target/release/deps/libhkdf-23697e084e864558.rlib --extern secp256k1=/home/ryan/works/rs/eciesrs/target/release/deps/libsecp256k1-97002a720e291605.rlib --extern rand=/home/ryan/works/rs/eciesrs/target/release/deps/librand-db608fc81ec83f78.rlib --extern reqwest=/home/ryan/works/rs/eciesrs/target/release/deps/libreqwest-626df515e3200886.rlib --extern sha2=/home/ryan/works/rs/eciesrs/target/release/deps/libsha2-3f02633d25ddd93e.rlib --extern tokio=/home/ryan/works/rs/eciesrs/target/release/deps/libtokio-a1ca85da5f2e46d4.rlib --extern typenum=/home/ryan/works/rs/eciesrs/target/release/deps/libtypenum-8754dce572e25207.rlib` (signal: 9, SIGKILL: kill)

   $ dmesg | egrep -i 'killed process'
   [ 1600.860772] Killed process 25878 (rustc) total-vm:70450572kB, anon-rss:25802536kB, file-rss:4kB, shmem-rss:0kB
   [ 1757.423082] Killed process 26023 (rustc) total-vm:70450572kB, anon-rss:25616460kB, file-rss:0kB, shmem-rss:0kB
   [ 2273.824947] Killed process 30194 (rustc) total-vm:70453132kB, anon-rss:25644020kB, file-rss:0kB, shmem-rss:0kB
   [ 2620.751454] Killed process 30529 (rustc) total-vm:70483340kB, anon-rss:25640772kB, file-rss:0kB, shmem-rss:0kB
   [ 2770.585678] Killed process 30583 (rustc) total-vm:70453132kB, anon-rss:25798272kB, file-rss:0kB, shmem-rss:0kB
   [ 3143.114175] Killed process 3061 (rustc) total-vm:74940760kB, anon-rss:25593988kB, file-rss:0kB, shmem-rss:0kB
   [ 3408.653964] Killed process 7694 (rustc) total-vm:70429604kB, anon-rss:25772680kB, file-rss:0kB, shmem-rss:0kB
   [ 4256.437701] Killed process 14596 (rustc) total-vm:70450468kB, anon-rss:25585988kB, file-rss:0kB, shmem-rss:0kB
   [ 9058.139548] Killed process 24291 (rustc) total-vm:70450468kB, anon-rss:25588892kB, file-rss:88kB, shmem-rss:0kB
   [ 9408.142325] Killed process 24403 (rustc) total-vm:70450468kB, anon-rss:25622152kB, file-rss:0kB, shmem-rss:0kB

Other information

1. All toolchains (stable, beta and nightly) share the same problem by Dec 9, 2020.
2. The bug does not occur on cargo build or cargo test, with or without --release.

Steps

1. git clone https://github.com/ecies/rs.git eciesrs && cd eciesrs in WSL2 ubuntu shell.
2. cargo bench --no-default-features --features pure
3. Watch your memory exploding

Possible Solution(s)

Set CARGO_BUILD_JOBS to a lower value.

Notes

Output of cargo version:

cargo 1.48.0 (65cbdd2dc 2020-10-14)
cargo 1.49.0-beta (b9216831a 2020-11-24)
cargo 1.50.0-nightly (63d0fe434 2020-12-02)

Possibly related issues

1. https://github.com/rust-lang/rust/issues/51309
2. https://users.rust-lang.org/t/cargo-build-crashes-with-sigkill/50035

[ehuss]

I don't think this is directly related to benchmarks, but due to const-eval using large amounts of memory due to these lines: https://github.com/ecies/rs/blob/9d893d19b7343aaecd30891f595b2601523dbcbd/bench/simple.rs#L6-L10

I think there are several other issues already open about similar issues, such as #79601, but I'm uncertain of the underlying cause (I'm not familiar with how const-eval works).

This program:

const BIGGER_MSG_SIZE: usize = 200 * 1024 * 1024;
const BIGGER_MSG: [u8; BIGGER_MSG_SIZE] = [2u8; BIGGER_MSG_SIZE];

fn main() {
    println!("{:?}", BIGGER_MSG[12]);
}

takes 30 seconds to build, and used over a gigabyte of memory.

[kigawas]

I don't think this is directly related to benchmarks, but due to const-eval using large amounts of memory due to these lines: https://github.com/ecies/rs/blob/9d893d19b7343aaecd30891f595b2601523dbcbd/bench/simple.rs#L6-L10

I think there are several other issues already open about similar issues, such as #79601, but I'm uncertain of the underlying cause (I'm not familiar with how const-eval works).

This program:

const BIGGER_MSG_SIZE: usize = 200 * 1024 * 1024;
const BIGGER_MSG: [u8; BIGGER_MSG_SIZE] = [2u8; BIGGER_MSG_SIZE];

fn main() {
    println!("{:?}", BIGGER_MSG[12]);
}

takes 30 seconds to build, and used over a gigabyte of memory.

Well, the array is exactly 200 megabytes, it'd be so hard to perceive that it could use gigabytes :(

Interestingly, this only occurs at cargo bench's building phase inside WSL2 (which means Windows itself is okay).

[jyn514]

@camelid this is unrelated to libtest or benchmarking, it's about compile times.

@kigawas can you confirm whether you have the same issue with cargo bench --no-run?

[kigawas]

I'll update the result about 8 hours later

====update==== @jyn514

--no-run did not help.