search

Gankra / abi-cafe

Pair your compilers up at The ABI Cafe!
https://faultlore.com/abi-cafe/book/
218 stars 15 forks source link

Compile multiple tests in parallel #6

Closed bjorn3 closed 2 months ago

bjorn3 commented 2 years ago

This should significantly reduce the time it takes to run all tests as most of the time is spent during compilation, which is currently single-threaded.

Gankra commented 2 years ago

Main problem to solve is properly buffering up compiler messages. Right now things are fairly interactive (although CC is annoyingly prone to spitting out tons of stuff on the assumption it's running in a build.rs still).

bjorn3 commented 2 years ago

It seems that output buffering is already implemented for rustc. For clang and gcc using .output() should work fine too I think. For cc using a separate process for the cc crate usage would work I think.

Gankra commented 2 years ago

That's probably the simplest/best approach. Main alternative is to do some upstream work to make cc-as-library-but-not-in-build.rs more usable.

Gankra commented 2 years ago

I started stubbing this out with rayon par_iter but unfortunately the current test code races on the filesystem when setting up various dirs. So this is sadly a bit non-trivial.

Gankra commented 2 months ago

As of #20 this is done -- the entire test suite now runs under tokio async, with a task spawned for every test. In addition, there is now an extensive caching/Once system, which ensures that we only generate a given source file and static lib at most once, for a given TestKey (test_name, abi_impl, callside, calling_convention, value_generator, write_impl). Filenames are also unique-per-key so there's never any conflict.

For safety I've set a limiter of 8 compiler invocations running at a time in parallel, but this can be freely bumped up by anyone with confidence. Even with this, the whole thing runs really fast now. Most of the output is also collated with tracing spans, so the output is still pleasant (actually it's nicer now).

Gankra commented 2 months ago

As an aside the concurrency limiter was actually only added at the 12th hour when my machine ran out of ram, after I added way more tests. Immediately after I discovered it wasn't concurrency out of control, but just a testcase with 256 x 12 fields creating superlinear cc/rustc compiler blowup from enormous function bodies.