awslabs / shuttle

Shuttle is a library for testing concurrent Rust code
Apache License 2.0
670 stars 34 forks source link

Batch semaphore, causality tracking #151

Closed Aurel300 closed 2 months ago

Aurel300 commented 2 months ago

This adds batch semaphores into Shuttle (not originally implemented by me) as the core primitive to be used for other synchronisation primitives. Vector clocks were added to the batch semaphore to track causality across acquire, release, and try_acquire calls.

Implementing other primitives using the batch semaphore will thus also cause causality to be tracked. The eventual goal is for primitives to only rely on the API of batch semaphore, without calls to e.g. ExecutionState. As a result, batch semaphore and the runtime internals can be pulled out of Shuttle into a separate shuttle-core crate, while both the stdlib and tokio primitives can be placed into new sibling crates, shuttle-stdlib and shuttle-tokio, respectively.

An open question is how to proceed with the crate restructuring. One option is for the crate called shuttle now to "become" shuttle-stdlib, as it will contain the same primitives, but this would require various re-exports. We could also publish a wrapper crate called shuttle (intended to be deprecated?) which will re-export items from both shuttle-core and shuttle-stdlib.

Planned for this PR still:

sarsko commented 2 months ago

Sweet.

[...] while both the stdlib and tokio primitives can be placed into new sibling crates, shuttle-stdlib and shuttle-tokio, respectively.

An open question is how to proceed with the crate restructuring. One option is for the crate called shuttle now to "become" shuttle-stdlib, as it will contain the same primitives, but this would require various re-exports. We could also publish a wrapper crate called shuttle (intended to be deprecated?) which will re-export items from both shuttle-core and shuttle-stdlib.

There's more.

Shuttle currently:

  1. "Core": all the functionality to run the program under test, ie starting up, pausing and resuming tasks, shutting down, etc.
  2. The schedulers (scheduler).
  3. std-lib mirrors (sync,thread.rs, hint.rs, thread_local! in lib.rs).
  4. future, ie a mix of futures::executor and tokio::task::JoinHandle.
  5. rand mirror.
  6. lazy_static mirror.
  7. The "external" interface: lib.rs (minus thread_local! and lazy_static!) and current.rs

My opinion is that all of these should be its own crate.

The question then becomes whether something which is called Shuttle should stick around. I think the thing called Shuttle will be the crate corresponding to bullet number 7. There is a question of whether it should also provide reexports of some of the other crates. I have not thought about that enough to have a definitive opinion, but I am generally of the opinion that the correct approach is to use the wrapper crates over what is provided by Shuttle directly. I am also of the opinion that future should be deprecated (ie it should not be provided by Shuttle, but may be provided by some crate).