Bogus `higher-ranked lifetime error` in an async block

[sfackler]

Running stable 1.64.0.

Unfortunately this happened in a complex bit of code that I can't easily reduce :(

            task::spawn({
                let handle_service = handle_service.clone();
                async move {
                    if let Err(e) = handle_service.call(connection).await {
                        debug!("http connection terminated", error: e);
                    }
                }
            });

error: higher-ranked lifetime error
   --> witchcraft-server/src/server.rs:118:13
    |
118 | /             task::spawn({
119 | |                 let handle_service = handle_service.clone();
120 | |                 async move {
121 | |                     if let Err(e) = handle_service.call(connection).await {
...   |
124 | |                 }
125 | |             });
    | |______________^
    |
    = note: could not prove `for<'r> impl for<'r> futures_util::Future<Output = ()>: std::marker::Send`

The future returned by handle_service.call(connection) is definitely Send, and I can work around the failure by boxing it:

            task::spawn({
                let handle_service = handle_service.clone();
                async move {
                    // The compiler hits a `higher-ranked lifetime error` if we don't box this future :/
                    let f: Pin<Box<dyn Future<Output = Result<(), Error>> + Send>> =
                        Box::pin(handle_service.call(connection));
                    if let Err(e) = f.await {
                        debug!("http connection terminated", error: e);
                    }
                }
            });

[LetMut1]

Sorry, I do not understand. Is the problem being solved?

[alilleybrinker]

Looks like @danielhenrymantilla's comment is the most up-to-date look into what's wrong here. It also looks like @vincenzopalazzo was previously assigned this issue, but has removed themself. It's now unassigned, but is part of a tracking issue for lifetime errors in async code.

Based on Daniel's explanation, it looks like this is due to a bug. As he explained, auto traits (like Send or Sync) are checked later than other trait bounds, in a context where lifetimes have been erased. The specific requirements for triggering this bug appear to be:

• An auto-trait bound like Send or Sync
• Async (really, generators, which async uses under the hood)
• An associated type
• A lifetime

I'm not part of the team handling this, so I have no idea where this sits in their priority, but I do think this describes the current state. I highly recommend looking at Daniel's post (which I linked to at the start of this comment), which details a workaround.

[westonpace]

Could anyone point me to the workaround for the issue please?

One potential workaround I've discovered for the buffered/buffer_unordered issue detailed in this comment appears to be collecting the futures into a Vec before calling buffered / buffer_unordered. Rust playground example.

This does introduce some copying and is only really practical when the source of the futures is synchronous (though the futures themselves can be asynchronous). However, this workaround has worked for me in a real world example and I figured I would share it in case it helps someone else.

[williameric87]

Could anyone point me to the workaround for the issue please?

One potential workaround I've discovered for the buffered/buffer_unordered issue detailed in this comment appears to be collecting the futures into a Vec before calling buffered / buffer_unordered. Rust playground example.

This does introduce some copying and is only really practical when the source of the futures is synchronous (though the futures themselves can be asynchronous). However, this workaround has worked for me in a real world example and I figured I would share it in case it helps someone else.

Instead of collecting the futures into a Vec like this, one could also just call boxed() before buffered(), which achieves a similar effect with less code, and is implemented by Box::pinning each future. Using the same Rust playground example would then look like this, or similarly, using just an async fn.