Share async runtimes among tests

[autotaker]

Is your feature request related to a problem? Please describe.

I'm trying to share a connection pool among tests for async functions.

My first trial is the following code:

async fn setup() -> &'static Pool<SomeConnection> {
  static INSTANCE: Pool<SomeConnection> = OnceCell::new();
  INSTANCE.get_or_init(async {
    ...
  }).await
}

#[tokio::test]
async fn test_foo() {
  let pool = setup().await;
  ...
}

#[tokio::test]
async fn test_bar() {
  let pool = setup().await;
  ...
}

Unfortunately, this approach doesn't work because:

• for each test, a new async runtime is created/dropped, and
• connections are registered to the IO Driver of current runtime, and cannot be used after the async runtime has dropped.

Describe the solution you'd like My proposal is to add a string property shared_runtime to #[tokio::test] to share async runtime among tests.

• If the property is specified, tests with the same shared_runtime value are run with the same async runtime.
• If the property is not specified, tests are run with independent async runtimes.

async fn setup() -> &'static Pool<SomeConnection> {
  static INSTANCE: Pool<SomeConnection> = OnceCell::new();
  INSTANCE.get_or_init(async {
    ...
  }).await
}

/// this test is run with a shared runtime identified by "my_module::key"
#[tokio::test(shared_runtime="my_module::key")]
async fn test_foo() {
  let pool = setup().await;
  ...
}

/// this test is run with the same runtime as `test_foo`.
#[tokio::test(shared_runtime="my_module::key")]
async fn test_bar() {
  let pool = setup().await;
  ...
}

/// this test is run with a new async runtime
#[tokio::test]
async fn test_baz() {  ...
}

Describe alternatives you've considered Current workaround is to share async runtime manually, for example:

fn runtime() -> &'static Runtime {
  static RUNTIME: OnceCell<Runtime> = OnceCell::new();
  RUNTIME.get_or_init(|| Builder::new_current_thread().enable_all().build().unwrap())
}

async fn setup() -> &'static Pool<SomeConnection> {
  static INSTANCE: Pool<SomeConnection> = OnceCell::new();
  INSTANCE.get_or_init(async {
    ...
  }).await
}

#[test]
fn test_foo() {
  runtime().block_on(async {
    let pool = setup().await;
    ...
  });
}

#[test]
fn test_bar() {
  runtime().block_on(async {
    let pool = setup().await;
    ...
  });
}

I think this workaround requires too much boilerplates.

Additional context

[hds]

Thanks for writing up this issue.

What is the use case you have for sharing a runtime across multiple tests?

My (personal) opinion is that individual test cases should be independent of one another. Sharing a runtime between tests would remove some of that independence, or even make the tests dependent on each other or the order in which they are run. This is generally undesirable behaviour.

[Noah-Kennedy]

I don't think that tokio necessarily needs to support this, but I don't have strong opinions either way here.

I would suggest using either once_cell or lazy_static to have a shared static runtime for your use case though.

[Darksonn]

Perhaps we could add a utility in the tokio-test crate?

[Noah-Kennedy]

I would not be opposed to this.

[autotaker]

@hds

What is the use case you have for sharing a runtime across multiple tests?

Typical use case is sharing a connection pool. The cost to create database connection is often not negligible.

I created a benchmark to compare with/without connection pool. The result shows sharing a connection is 30x faster than using independent connections.

     Running benches/benchmark.rs (/tmp/target/release/deps/benchmark-62dd168396478ae1)
Select Now with an independent runtime
                        time:   [6.9064 ms 6.9880 ms 7.0753 ms]
                        change: [-4.6350% -2.9059% -1.1734%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

Select Now with a shared runtime
                        time:   [236.29 µs 241.22 µs 247.55 µs]
                        change: [-39.479% -20.590% +4.8383%] (p = 0.13 > 0.05)
                        No change in performance detected.

https://gist.github.com/autotaker/ebcf3a5bb1d21fc6c59bf16d3a4446db

Besides, when one tries to use shared connection pool, they don't notice that they have to share the runtime as well. The reasons are:

• errors occur at running time, and
• some connection pool implementation (e.g. sqlx) often hides these errors, treating them as a general network error.

[silence-coding]

Any progress on this issue? I also think there are usage scenarios for shared runtimes,

[silence-coding]

I don't think that tokio necessarily needs to support this, but I don't have strong opinions either way here.

I would suggest using either once_cell or lazy_static to have a shared static runtime for your use case though.

Developers can share runtimes with once_cell or lazy_static , but I think we should promote the use of tokio::test.

[Noah-Kennedy]

@silence-coding no one has been actively driving this forward, so there has been no progress.

If you are interested in this, please consider specing out in here what this might look like. Once the group can discuss design details, you can move on to the implementation in a PR.

[silence-coding]

It should only need to implement process sharing, and there is no need to select different runtimes according to different keys, because cargo will start a new process for each test file, and we should only perform one type of test in a single test file.

#[tokio::test(flavor = “shared")]  // or  #[tokio::test(shared)] 
async fn test_bar() {
  let pool = setup().await;
  ...
}

[silence-coding]

After the shared runtime is implemented, can we supplement the test phases such as setup and after? Like Junit's @Before/@After called