rust-osdev / bootimage

Tool to create bootable disk images from a Rust OS kernel.
Apache License 2.0
764 stars 67 forks source link

Use rayon when running `bootimage test` #24

Closed acheronfail closed 6 years ago

acheronfail commented 6 years ago

Fixes #21

This PR adds rayon and uses it during the bootimage test command so that all the tests are run in parallel.

There's a few caveats though:

Aside from those, the actual tests themselves are run in parallel, just not when they're built..

Suggestions welcome for improvements!

phil-opp commented 6 years ago

Very nice, thanks!

While the tests are building they block each other while waiting to build

This is a bit unfortunate. It also causes some very verbose output:

``` Blocking waiting for file lock on x86_64-blog_os's sysroot Blocking waiting for file lock on x86_64-unknown-linux-gnu's sysroot Blocking waiting for file lock on x86_64-unknown-linux-gnu's sysroot Blocking waiting for file lock on x86_64-unknown-linux-gnu's sysroot Blocking waiting for file lock on build directory Blocking waiting for file lock on build directory Finished dev [unoptimized + debuginfo] target(s) in 0.34s Building bootloader v0.3.4 Finished dev [unoptimized + debuginfo] target(s) in 0.36s Building bootloader v0.3.4 Finished dev [unoptimized + debuginfo] target(s) in 0.39s Building bootloader v0.3.4 Finished dev [unoptimized + debuginfo] target(s) in 0.05s Building bootloader v0.3.4 Blocking waiting for file lock on x86_64-bootloader's sysroot Blocking waiting for file lock on x86_64-bootloader's sysroot Finished release [optimized + debuginfo] target(s) in 0.04s Blocking waiting for file lock on x86_64-bootloader's sysroot Blocking waiting for file lock on x86_64-unknown-linux-gnu's sysroot Blocking waiting for file lock on x86_64-bootloader's sysroot Finished release [optimized + debuginfo] target(s) in 0.05s Blocking waiting for file lock on x86_64-unknown-linux-gnu's sysroot Finished release [optimized + debuginfo] target(s) in 0.04s Finished release [optimized + debuginfo] target(s) in 0.04s ```

Maybe it's possible to first build all tests sequentially and then run them in parallel? I think this could work by iterating twice: In the first iteration we call build::build_impl for each tests so that we get an iterator of test_paths as result. In the second iteration we use par_iter to launch QEMU for each test_path and collect the test results.

acheronfail commented 6 years ago

Maybe it's possible to first build all tests sequentially and then run them in parallel?

That makes things both easier and gives much nicer output. 👌

Output ``` BUILD: test-exception-breakpoint Finished dev [unoptimized + debuginfo] target(s) in 0.05s Building bootloader v0.3.4 Finished release [optimized + debuginfo] target(s) in 0.04s BUILD: test-basic-boot Compiling blog_os v0.2.0 (/Users/cosmotherly/src/blog_os) Finished dev [unoptimized + debuginfo] target(s) in 0.38s Building bootloader v0.3.4 Finished release [optimized + debuginfo] target(s) in 0.04s BUILD: test-panic Finished dev [unoptimized + debuginfo] target(s) in 0.05s Building bootloader v0.3.4 Finished release [optimized + debuginfo] target(s) in 0.04s BUILD: test-exception-double-fault-stack-overflow Finished dev [unoptimized + debuginfo] target(s) in 0.05s Building bootloader v0.3.4 Finished release [optimized + debuginfo] target(s) in 0.04s RUN: test-exception-breakpoint RUN: test-basic-boot RUN: test-panic RUN: test-exception-double-fault-stack-overflow OK: test-panic OK: test-exception-breakpoint OK: test-basic-boot OK: test-exception-double-fault-stack-overflow All tests succeeded. ```
phil-opp commented 6 years ago

Awesome, thanks a lot!

phil-opp commented 6 years ago

Almost forgot to push a new version to crates.io. Version 0.5.7 is out.