handle references in function results

[yanns]

when using join_all , we can use references in function results, like:

async fn say_hello_in_vec(who: &str) -> Vec<Vec<&str>> {
    let handles = (1..=5).map(|i| hello_in_vec(i, who));

    // intra-task concurrency
    let handle = join_all(handles);
    let results = handle.await;
    results
}

async fn hello_in_vec(i: i32, who: &str) -> Vec<&str> {
    println!("Job {i} started");
    tokio::time::sleep(std::time::Duration::from_secs(1)).await;
    let result = vec!["hello", who];
    println!("Job {i} finished");
    result
}

When trying to use scope_and_collect instead, this does not compile:

async fn say_hello_in_vec(who: &str) -> Vec<Vec<&str>> {
    let handles = (1..=5).map(|i| hello_in_vec(i, who));

    // with async_scoped
    let (_, results) = unsafe {
        async_scoped::TokioScope::scope_and_collect(|s| {
            for handle in handles {
                s.spawn(handle);
            }
        })
        .await
    };
    let results = results.into_iter().map(|res| res.unwrap()).collect();

    results
}

async fn hello_in_vec(i: i32, who: &str) -> Vec<&str> {
    println!("Job {i} started");
    tokio::time::sleep(std::time::Duration::from_secs(1)).await;
    let result = vec!["hello", who];
    println!("Job {i} finished");
    result
}

error: lifetime may not live long enough
  --> src/main.rs:80:35
   |
79 | async fn say_hello_in_vec(who: &str) -> Vec<Vec<&str>> {
   |                                - let's call the lifetime of this reference `'1`
80 |     let handles = (1..=5).map(|i| hello_in_vec(i, who));
   |                                   ^^^^^^^^^^^^^^^^^^^^ returning this value requires that `'1` must outlive `'static`

[rmanoka]

Hi, apologies for not responding on this.

Yes, currently, the implementation only works for 'static items returned from the futures. I may be wrong, but I think it's not possible to convince the type-system about the life-time of the output of the future itself, without boxing it. But boxing it defeats the purpose of returning the refs in the first-place itself.

[yanns]

Thanks for the answer!

I may be wrong, but I think it's not possible to convince the type-system about the life-time of the output of the future itself, without boxing it

Do you think that join_all have found a solution for the type system, or are they using boxing, or are they using another solution?

[rmanoka]

No, the issue stems from a std::mem::transmute trick that crates like ours use (other being crossbeam::threads): change the lifetime to 'static. This step is not reqd. in join_all.

Now, the design of this crate where Scope is generic over T doesn't make it easy to express the static version of T, so instead we need T: 'static. Similarily, even the future type F is generic, so indeed, we do an extra box-ing of the future to be able to name the output type.

[yanns]

Thanks for the explanation!

I let you choose if you want to keep this ticket open for future work, and if you want to close it as you don't plan to do it.