rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
97.38k stars 12.59k forks source link

Bogus `higher-ranked lifetime error` in an async block #102211

Open sfackler opened 2 years ago

sfackler commented 2 years ago

Running stable 1.64.0.

Unfortunately this happened in a complex bit of code that I can't easily reduce :(

            task::spawn({
                let handle_service = handle_service.clone();
                async move {
                    if let Err(e) = handle_service.call(connection).await {
                        debug!("http connection terminated", error: e);
                    }
                }
            });
error: higher-ranked lifetime error
   --> witchcraft-server/src/server.rs:118:13
    |
118 | /             task::spawn({
119 | |                 let handle_service = handle_service.clone();
120 | |                 async move {
121 | |                     if let Err(e) = handle_service.call(connection).await {
...   |
124 | |                 }
125 | |             });
    | |______________^
    |
    = note: could not prove `for<'r> impl for<'r> futures_util::Future<Output = ()>: std::marker::Send`

The future returned by handle_service.call(connection) is definitely Send, and I can work around the failure by boxing it:

            task::spawn({
                let handle_service = handle_service.clone();
                async move {
                    // The compiler hits a `higher-ranked lifetime error` if we don't box this future :/
                    let f: Pin<Box<dyn Future<Output = Result<(), Error>> + Send>> =
                        Box::pin(handle_service.call(connection));
                    if let Err(e) = f.await {
                        debug!("http connection terminated", error: e);
                    }
                }
            });
eholk commented 2 years ago

We discussed this in the wg-async triage today. We think the best course of action is to try to get a minimal test case for this. Afterwards, we'll probably want to get T-Types input on this. @vincenzopalazzo is going to try to reproduce this and minimize the test case.

eholk commented 2 years ago

@rustbot label AsyncAwait-Triaged

vincenzopalazzo commented 2 years ago

@rustbot claim

danielhenrymantilla commented 2 years ago

I wonder if the following snippet wouldn't be a reduction:

struct Type<'a, 'b>(&'a &'b (), ::core::marker::PhantomData<*mut ()>);

// Note: if this had an *implicit* `'b : 'a` things would work.
unsafe impl<'b : 'a, 'a> Send for Type<'a, 'b> {}

fn foo() -> impl Send { async {
    let local = ();
    let r = &&local;
    async {}.await;
    let it = Type(r, <_>::default());
    async {}.await;
}}
eholk commented 1 year ago

@sfackler - Would it be possible to share more of the surrounding code for this example so we can see more of what's going on?

sfackler commented 1 year ago

I'm working on minimizing it.

sfackler commented 1 year ago

Here is a semi-minimized repro. It still depends on Hyper, but it could probably be replaced with some PhantomData inserts with some more work: https://gist.github.com/sfackler/a7f3aaf2ce3f5ff4c99e1a3d67f828ed

error: higher-ranked lifetime error
   --> witchcraft-server/src/lib.rs:162:5
    |
162 | /     spawn(async move {
163 | |         let _ = handle_service.call(stream).await;
164 | |     });
    | |______^
    |
    = note: could not prove `for<'r> impl for<'r> futures_util::Future<Output = ()>: Send`
sfackler commented 1 year ago

For whatever reason, the presence of the AuditLogService in the future seems to be a trigger. If you change line 160 to just let request_service = HandlerService it compiles.

vincenzopalazzo commented 1 year ago

Thank @sfackler,

I think the core concept missed in your previous report are:

Using a wrapper around Future like

pub struct AdaptorFuture<F> {
    inner: F,
}

impl<F, T> Future for AdaptorFuture<F>
where
    F: Future<Output = T>,
{
    type Output = Result<T, io::Error>;

    fn poll(self: Pin<&mut Self>, _: &mut Context<'_>) -> Poll<Self::Output> {
        panic!()
    }
}

and your handle_service implementation

impl<S, R> hyper::service::Service<R> for AdaptorService<S>
where
    S: Service<R>,
{
    type Response = S::Response;

    type Error = io::Error;

    type Future = AdaptorFuture<S::Future>;

    fn poll_ready(&mut self, _: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
        Poll::Ready(Ok(()))
    }

    fn call(&mut self, req: R) -> Self::Future {
        AdaptorFuture {
            inner: self.inner.call(req),
        }
    }
}
vincenzopalazzo commented 1 year ago

@sfackler I'm trying to reproduce your error without hyper, and till now I had no luck.

Maybe you can help with some tips? I had the code there https://github.com/vincenzopalazzo/rio/pull/13

sfackler commented 1 year ago

Not sure I have a better idea of how to minimize than you, unfortunately.

brandonros commented 1 year ago

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=b247265803ef4fbc2223a4ddcec31432

I think I was able to recreate it?

It took me like 35 minutes to figure out where it was because the compiler only gives this message with nothing else and no extra details:

error: higher-ranked lifetime error
   --> bot/src/main.rs:141:23
    |
141 |         let handle2 = tokio::task::spawn(robinhood_task());
    |                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: could not prove `impl Future<Output = ()>: Send`
vincenzopalazzo commented 1 year ago

This looks promising @brandonros! I will give it a shot and also try to reproduce inside https://github.com/vincenzopalazzo/rio/pull/13

Looks like that when the runtiime try to span the following task there is an error like the one reported from you

async fn robinhood_task() {
    let robinhood = Robinhood::new();
    let token = String::from("fake");
    let instrument_ids = vec![String::from("fake")];
    robinhood.get_options_market_data(token, instrument_ids).await;
}

In particulat I play a little bit with it and I found the following example to reproduce the error

use futures::StreamExt;
use log::info;
use serde::{Deserialize, Serialize};

pub struct Robinhood;

impl Robinhood {
    pub fn new() -> Robinhood {
        return Robinhood {};
    }

    async fn foo(&self, t: String, b: Vec<String>) -> Vec<String> {
        // Looks like that the problem is generated from the following code
        let b: Vec<&[String]> = b.chunks(2).collect();
        let futures = b.into_iter().map(|it| {
            return async { vec![] };
        });

        let results: Vec<String> = futures::stream::iter(futures)
            .buffer_unordered(4)
            .collect::<Vec<_>>()
            .await
            .into_iter()
            .flatten()
            .collect();
        results
    }
}

async fn foo_ok(t: String, b: Vec<String>) -> Vec<String> {
    let b: Vec<&[String]> = b.chunks(2).collect();
    let futures = b.into_iter().map(|it| {
        return async { vec![] };
    });

    let results: Vec<String> = futures::stream::iter(futures)
        .buffer_unordered(4)
        .collect::<Vec<_>>()
        .await
        .into_iter()
        .flatten()
        .collect();
    results
}

async fn robinhood_task() {
    let robinhood = Robinhood::new();
    foo_ok(String::new(), vec![]).await;
    robinhood.foo(String::new(), vec![]).await;
}

fn main() {
    async {
        let handle1 = tokio::task::spawn(robinhood_task());
        let _ = tokio::join!(handle1);
    };
}
vincenzopalazzo commented 1 year ago

@rustbot label +A-async-await

our penguing triage bot it is not smart enough to reassign this in the next triage meeting so I need to do this trick!

rustbot commented 1 year ago

Error: Label AsyncAwait-Triaged can only be set by Rust team members

Please file an issue on GitHub at triagebot if there's a problem with this bot, or reach out on #t-infra on Zulip.

brandonros commented 1 year ago

I'm sure this is obvious to you, I just want to make sure it's documented somewhere:

switching let b: Vec<&[String]> = b.chunks(2).collect(); to let b: Vec<Vec<String>> = b.chunks(2).map(|c| c.to_vec()).collect(); works around the issue.

awesomelemonade commented 1 year ago

Unsure if this is the same exact issue, but I ran into a "higher-ranked lifetime error" when running https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=e8d2668ceca75a8ebf4f0c16cee04958. I tried to work around the failure by boxing it like suggested, but it still doesn't seem to work. The full error is

error: higher-ranked lifetime error
  --> src/lib.rs:25:18
   |
25 |     let handle = tokio::spawn(accept_connection(rpc));
   |                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: could not prove `impl futures::Future<Output = Result<(), anyhow::Error>>: std::marker::Send

EDIT: if I pin & box the lol future, it works: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=5d7d8b1dd6542571425b37b5e2cd6ee9

Tails commented 1 year ago

Had the same problem, but @awesomelemonade's example helped. It worked by pinning the try_join_all call and explicitly specifying the type in the assignment. Note that the Send bound made the difference in 'dyn Future<> + Send'.

        // let objects = futures::future::try_join_all(futs)
        //     .await?
        //     .into_iter()
        //     .collect::<HashMap<Self::Address, C>>();

        let f: Pin<
            Box<
                dyn Future<Output = anyhow::Result<Vec<(<Self as StorageProvider>::Address, C)>>>
                    + Send,
            >,
        > = Box::pin(futures::future::try_join_all(futs));

        f.await;
danielhenrymantilla commented 1 year ago

@Tails @awesomelemonade you can skip the heap allocation with a helper function:

fn assert_send<'u, R>(fut: impl 'u + Send + Future<Output = R>)
  -> impl 'u + Send + Future<Output = R>
{
    fut
}

let f = assert_send(futures::future::try_join_all(futs));
f.await

All this, and more is present in that Discord discussion I mentioned:

mcronce commented 1 year ago

I was running into this problem in a project tonight (which prompted me to go through a bunch of its dependencies and enable warn(clippy::future_not_send) :joy:); the assert_send() (and, indeed, Box::pin) solution didn't work, but switching to stable did. Not sure whether or not it's well known that this is a nightly-specific problem, or if I'm just hitting it in a nightly-specific way, so take that for what it's worth.

I did update my nightly to latest, but I'm not sure what version I was on before. Not something more than a few days old, though, I'm pretty sure.

kylezs commented 1 year ago

In my case, code was compiling in nightly-2022-08-08 but when bumping to nightly-2022-12-16 it started failing with this error.

The future was carrying a stream (that was Send) across an await boundary. So inspired by some solutions above, I had to Box::pin the stream (which was already the case before I bumped the Rust version) and use the assert_stream_send below. So it looked like assert_stream_send(Box::pin(stream))

fn assert_stream_send<'u, R>(
    strm: impl 'u + Send + Stream<Item = R>,
) -> impl 'u + Send + Stream<Item = R> {
    strm
}

Then it compiled.

hseeberger commented 1 year ago

Interesting. In my case I also have a future producing a stream, but I cannot get it to compile:

#![allow(incomplete_features)]
#![feature(return_position_impl_trait_in_trait)]

use anyhow::Result;
use futures::{pin_mut, Future, Stream, StreamExt};
use tokio::task;

#[tokio::main]
async fn main() -> Result<()> {
    Ok(())
}

async fn run<T>(mut foo: T) -> Result<()>
where
    T: Foo + Send + 'static,
{
    task::spawn(async move {
        let bar = assert_send(foo.bar());
        let bar = bar.await; // `impl Stream<Item = i32> + Send`
        let mut bar = stream_assert_send(Box::pin(bar)); // still `impl Stream<Item = i32> + Send`

        // higher-ranked lifetime error
        // bar.next().await;

        let next = assert_send(bar.next());
        // Also higher-ranked lifetime error
        next.await;
    });

    Ok(())
}

fn assert_send<'a, T>(
    fut: impl std::future::Future<Output = T> + Send + 'a,
) -> impl std::future::Future<Output = T> + Send + 'a {
    fut
}

fn stream_assert_send<'a, T>(
    stream: impl futures::Stream<Item = T> + Send + 'a,
) -> impl futures::Stream<Item = T> + Send + 'a {
    stream
}

trait Foo {
    fn bar<'a>(&'a mut self) -> impl Future<Output = impl Stream<Item = i32> + Send + 'a> + Send;
}
frxstrem commented 1 year ago

I'm having a similar issue, I've boiled it down to this minimal reproducible example:

use futures::{future::BoxFuture, stream, StreamExt};

async fn inner() {
    stream::iter(&Vec::<()>::new())
        .map(|_| stream::empty::<()>())
        .flatten()
        .collect::<()>()
        .await;
}

fn outer() -> BoxFuture<'static, ()> {
// error: higher-ranked lifetime error
    Box::pin(async move { inner().await })
//  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// = note: could not prove `Pin<Box<impl futures::Future<Output = ()>>>: CoerceUnsized<Pin<Box<
//         (dyn futures::Future<Output = ()> + std::marker::Send + 'a)>>>`
}

In Rust playground

The issue seems to be related to the closure |_| stream::empty::<()>(), since replacing the closure with a function pointer works:

fn f(_: &()) -> stream::Empty::<()> {
    stream::empty()
}

stream::iter(&Vec::<()>::new())
    .map(f)
    .flatten()
    .collect::<Vec<_>>()
    .await;

Additionally, using the assert_send_stream technique as mentioned above seems to work as well:

assert_stream_send(
    stream::iter(&Vec::<()>::new())
        .map(|_| stream::empty::<()>())
)
    .flatten()
    .collect::<Vec<_>>()
    .await;

Versions and dependencies:

rustc 1.66.0 (69f9c33d7 2022-12-12)

futures = "0.3.25"
emersonford commented 1 year ago

Here's another minimal reproducible example, with no dependencies:

use std::future::Future;

async fn foo() {
    let some_vec = Vec::<()>::new();

    let flat = some_vec.iter()
        .map(|_| std::iter::empty::<()>())
        .flatten();
    std::mem::drop(flat);
    async{}.await;
}

fn assert_send<'u, R>(fut: impl 'u + Send + Future<Output = R>)
  -> impl 'u + Send + Future<Output = R>
{
    fut
}

fn main() {
    let fut = assert_send(foo());
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=040ed9aedff7ccf4ed03d3bd3f48e5f3

desugared (think it's a bit clearer?):

use std::future::Future;

fn foo() -> impl Future<Output = ()> + Send {
    let some_vec = Vec::<()>::new();

    async move {
        let flat = some_vec.iter()
            .map(|_| std::iter::empty::<()>())
            .flatten();
        std::mem::drop(flat);
        async{}.await;
    }
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=716e2cafcb2f451e1e34b795820063c6

what's critical here is the flatten(). It looks like something has to take ownership of the Iterator from .map to trigger the error?

EDIT: for clarification, these will spit out the following error:

error: implementation of `Iterator` is not general enough
  --> src/main.rs:8:5
   |
8  | /     async move {
9  | |         let flat = some_vec.iter()
10 | |             .map(|_| std::iter::empty::<()>())
11 | |             .flatten();
12 | |         std::mem::drop(flat);
13 | |         dummy().await;
14 | |     }
   | |_____^ implementation of `Iterator` is not general enough
   |
   = note: `Iterator` would have to be implemented for the type `std::slice::Iter<'0, ()>`, for any lifetime `'0`...
   = note: ...but `Iterator` is actually implemented for the type `std::slice::Iter<'1, ()>`, for some specific lifetime `'1`

error: implementation of `FnOnce` is not general enough
  --> src/main.rs:8:5
   |
8  | /     async move {
9  | |         let flat = some_vec.iter()
10 | |             .map(|_| std::iter::empty::<()>())
11 | |             .flatten();
12 | |         std::mem::drop(flat);
13 | |         dummy().await;
14 | |     }
   | |_____^ implementation of `FnOnce` is not general enough
   |
   = note: closure with signature `fn(&'0 ()) -> std::iter::Empty<()>` must implement `FnOnce<(&(),)>`, for any lifetime `'0`...
   = note: ...but it actually implements `FnOnce<(&(),)>`

which is identical to what's found in #71671. This originally doesn't appear related to the higher-ranked lifetime error error, but I'm fairly certain this error is the root cause for this because you can desugar what @frxstrem came up with like so:

use std::future::Future;

use futures::{future::BoxFuture, stream, StreamExt};

async fn dummy() {}

fn inner() -> impl Future<Output = ()> {
    let some_vec = Vec::<()>::new();
    async move {
        let flat = stream::iter(&some_vec)
            .map(|_| stream::empty::<()>())
            .flatten();

        std::mem::drop(flat);
        dummy().await;
    }
}

fn outer() -> BoxFuture<'static, ()> {
    Box::pin(async move { inner().await })
}

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=145254a963e2e1551d919c286f3b4845

to get this error involving the iterator.

EDIT2: this only happens when you need a Send bound on the future, explaining why folks are running into this with tokio::spawn (BoxFuture from the futures crate also requires a Send bound). If you don't need a Send bound, then you don't get this error.

EDIT3: looks like it's related to #64552 as well?

EDIT4: based on https://github.com/dtolnay/async-trait/issues/215, this is definitely just the same issue as #64552.

danielhenrymantilla commented 1 year ago

So, from @emersonford's snippet:

let _: &dyn Send = &async {
    let _it =
        [()].iter()
            .map(|it| None::<()>)
            .flatten()
    ;
    async {}.await;
};

I have tried to figure out what was about .flatten() which would cause this, _when it doesn't happen for .flat_map()!_

It turns out, Flatten is defined in terms of an associated type (the Iterator::Item, which when dealing with .map(), involves a FnOnce::Output), whereas FlatMap introduces a new free type parameter which is then constrained to be equal to the associated type later on.

From that observation, I was able to craft an even smaller reduction, since it involves no Iterator shenanigans:

Note that the argument of fn() involving a lifetime, even if it's a hard-coded lifetimes such as 'static, is key to trigger this.

emersonford commented 1 year ago

Just wanted to share some things I noticed debugging this yesterday, using this code block:

use std::future::Future;

fn foo() -> impl Future<Output = ()> + Send {
    let some_vec = Vec::<()>::new();

    async move {
        let flat = some_vec.iter()
            .map(|_| std::iter::empty::<()>())
            .flatten();
        std::mem::drop(flat);
        async{}.await;
    }
}

I was curious why dropping + Send on impl Future makes this compile, so I spit out debug logs from the compiler for both a + Send version and no + Send version to figure out where they differ. As far as I can tell, this is the where they begin to differ:

Non + Send version:

│ │ ├─┐rustc_borrowck::type_check::canonical::normalize_with_category value=([[async block@lib.rs:6:5: 12:6]]; c_variadic: false)->[async block@lib.rs:6:5: 12:6], location=bb1[3], category=Boring
│ │ │ ├─0ms DEBUG rustc_borrowck::type_check::canonical output=([[async block@lib.rs:6:5: 12:6]]; c_variadic: false)->[async block@lib.rs:6:5: 12:6], constraints=None
│ │ ├─┘
│ │ ├─┐rustc_borrowck::type_check::canonical::normalize_with_category value=impl std::future::Future<Output = ()>, location=bb1[3], category=Boring
│ │ │ ├─0ms DEBUG rustc_borrowck::type_check::canonical output=impl std::future::Future<Output = ()>, constraints=None
│ │ ├─┘
│ │ ├─┐rustc_borrowck::type_check::relate_tys::relate_types a=impl std::future::Future<Output = ()>, v=-, b=[async block@lib.rs:6:5: 12:6], locations=Single(bb1[3]), category=Return(Normal)
│ │ │ ├─0ms DEBUG rustc_borrowck::type_check::canonical output=(), constraints=None
│ │ ├─┘

Send version:

│ │ ├─┐rustc_borrowck::type_check::canonical::normalize_with_category value=([[async block@lib.rs:6:5: 12:6]]; c_variadic: false)->[async block@lib.rs:6:5: 12:6], location=bb1[3], category=Boring
│ │ │ ├─0ms DEBUG rustc_borrowck::type_check::canonical output=([[async block@lib.rs:6:5: 12:6]]; c_variadic: false)->[async block@lib.rs:6:5: 12:6], constraints=None
│ │ ├─┘
│ │ ├─┐rustc_borrowck::type_check::canonical::normalize_with_category value=impl std::future::Future<Output = ()> + std::marker::Send, location=bb1[3], category=Boring
│ │ │ ├─0ms DEBUG rustc_borrowck::type_check::canonical output=impl std::future::Future<Output = ()> + std::marker::Send, constraints=None
│ │ ├─┘
│ │ ├─┐rustc_borrowck::type_check::relate_tys::relate_types a=impl std::future::Future<Output = ()> + std::marker::Send, v=-, b=[async block@lib.rs:6:5: 12:6], locations=Single(bb1[3]), category=Return(Normal)
│ │ │ ├─0ms DEBUG rustc_borrowck::type_check::canonical output=(), constraints=Some(QueryRegionConstraints { outlives: [(Binder(OutlivesPredicate('_#2r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) })), []), BoringNoLocation), (Binder(OutlivesPredicate('_#3r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) })), []), BoringNoLocation), (Binder(OutlivesPredicate('_#4r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) })), []), BoringNoLocation), (Binder(OutlivesPredicate('_#2r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(1, Some(lib.rs:8:14: 8:17 (#0))) })), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }), '_#2r), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(1, Some(lib.rs:8:14: 8:17 (#0))) }), '_#2r), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }), '_#3r), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }), '_#4r), []), BoringNoLocation)], member_constraints: [] })
│ │ │ ├─┐rustc_borrowck::type_check::push_region_constraints locations=Single(bb1[3]), category=Return(Normal)
│ │ │ │ ├─0ms DEBUG rustc_borrowck::type_check constraints generated: QueryRegionConstraints {
│ │ │ │ │     outlives: [
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     '_#2r,
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }),
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     '_#3r,
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }),
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     '_#4r,
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }),
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     '_#2r,
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(1, Some(lib.rs:8:14: 8:17 (#0))) }),
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }),
│ │ │ │ │                     '_#2r,
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(1, Some(lib.rs:8:14: 8:17 (#0))) }),
│ │ │ │ │                     '_#2r,
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }),
│ │ │ │ │                     '_#3r,
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │         (
│ │ │ │ │             Binder(
│ │ │ │ │                 OutlivesPredicate(
│ │ │ │ │                     RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }),
│ │ │ │ │                     '_#4r,
│ │ │ │ │                 ),
│ │ │ │ │                 [],
│ │ │ │ │             ),
│ │ │ │ │             BoringNoLocation,
│ │ │ │ │         ),
│ │ │ │ │     ],
│ │ │ │ │     member_constraints: [],
│ │ │ │ │ }
│ │ │ │ ├─┐rustc_borrowck::type_check::constraint_conversion::convert_all query_constraints=QueryRegionConstraints { outlives: [(Binder(OutlivesPredicate('_#2r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) })), []), BoringNoLocation), (Binder(OutlivesPredicate('_#3r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) })), []), BoringNoLocation), (Binder(OutlivesPredicate('_#4r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) })), []), BoringNoLocation), (Binder(OutlivesPredicate('_#2r, RePlaceholder(Placeholder { universe: U1, name: BrAnon(1, Some(lib.rs:8:14: 8:17 (#0))) })), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }), '_#2r), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(1, Some(lib.rs:8:14: 8:17 (#0))) }), '_#2r), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }), '_#3r), []), BoringNoLocation), (Binder(OutlivesPredicate(RePlaceholder(Placeholder { universe: U1, name: BrAnon(0, None) }), '_#4r), []), BoringNoLocation)], member_constraints: [] }

...

├─┐rustc_borrowck::region_infer::best_blame_constraint from_region='_#5r, from_region_origin=Placeholder(Placeholder { universe: U1, name: BrAnon(0, None) })

I'm not at all familiar with what the compiler is doing so apologies if this is not at all correct but it seems like Send is triggering a constraint propagation down to the async block? And what for whatever reason, that round of constraint propagation isn't properly handling lifetimes...?

Oh also, you can trigger this same thing with + Sync instead of + Send, which makes me further think it's related to the constraint propagation code

Dessix commented 1 year ago

I'm currently working with axum and axum-server and have hit this issue in a multitude of ways on both nightly-2022-11-18 and nightly-2023-01-18, all of which when building or otherwise utilizing generic implementations of tower_service::Service.

Directly boxing the future resulting from axum_server::bind(...).serve(my_service) works, e.g.:

// Must reside in a non-async context, e.g. a function returning [BoxFuture]
axum_server::bind(addr).serve(service).map_err(From::from).boxed()

But using an async block around it produces a block which fails to prove Send:

async move {
    axum_server::bind(addr).serve(service).map_err(From::from).await
}.boxed()

What's odd, and may give a clue as to what's going on, is that proving also fails if the surrounding block is async, even when directly boxing:

async move {
    axum_server::bind(addr).serve(service).map_err(From::from).boxed().await
}.boxed() // This box fails to prove `Send` despite the prior succeeding

Even more unusual, I noted some success when introducing BoxCloneService, but I've yet to discover a pattern with this, and it does not appear to consistently result in a fix on its own, nor produce further breakage. While I've yet to discern a pattern in what precise boxing addresses it, the most useful conclusion I've been able to draw is that async Send constraints appear to propagate further than their last .boxed() call and into the next synchronous context.

In summary, a workaround for axum users experiencing this issue (which may generalize further) is to box within a synchronous function returning BoxFuture, to (presumably) terminate the constraint propagation before it breaks.

emersonford commented 1 year ago

Just for reference, looks like this is the same issue as #71723.

EDIT: and #102870, #99492, #98380, #89976

(possibly #90656, #97515, #92415?)

vincenzopalazzo commented 1 year ago

Ok, I think it is time to bring it up again, and discuss it with the async-wg in the next triage meeting, there was good progress on how to reproduce it

@rustbot label +I-async-nominated

Swatinem commented 1 year ago

It is also a blocker to make progress on https://github.com/rust-lang/rust/pull/107562

msdrigg commented 1 year ago

Not sure if it is the same issue, but I am getting a similar error on stable 1.67.0

Error Here

error: higher-ranked lifetime error
  --> crates/utils/src/supervisor_error.rs:13:9
   |
13 |         Box::pin(self.0.into_future())
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: could not prove `Pin<Box<impl futures::Future<Output = Result<(), Box<(dyn StdError + std::marker::Send + Sync + 'static)>>>>>: CoerceUnsized<Pin<Box<(dyn futures::Future<Output = Result<(), Box<(dyn StdError + std::marker::Send + Sync + 'b)>>> + std::marker::Send + 'c)>>>`

Reproducible example here

use futures::{future::BoxFuture, Future, FutureExt};

pub async fn identity<F: Future>(future: F) -> F::Output {
    future.await
}

type BoxError = Box<dyn std::error::Error + Send + Sync + 'static>;

pub struct Supervisor2(Supervisor1);

impl Supervisor2 {
    fn into_future(&mut self) -> BoxFuture<'_, Result<(), BoxError>> {
        Box::pin(self.0.into_future())
    }
}

impl Supervisor1 {
    pub async fn into_future(&mut self) -> Result<(), BoxError> {
        let child = self.child.into_future().map(|res| res);

        identity(child).await;

        return Ok(());
    }
}

pub struct Supervisor1 {
    pub child: Box<Supervisor2>,
}

There are several things changes that will keep the error from happening

  1. Changing the BoxError type to something simpler like Box<String> stops the error from happening.
    - type BoxError = Box<dyn std::io::Write + Send + Sync + 'static>;
    + type BoxError = Box<String>;
  2. Removing the unnecessary .map(|res| res) also stops the error from happening.
    - let child = self.child.into_future().map(|res| res);
    + let child = self.child.into_future();
  3. Removing the identity call stops the error as well
    -  identity(child).await;
    +  child.await;

I tried reducing it further, but I can't make any progress at the moment.

zeenix commented 1 year ago

FWIW, I hit this in zbus while trying to reduce the bloat from generics. :( Unfortunately the futures are quite complex in my case to be easily able to provide a reduced reproducer. :(

rustc 1.67.1 here as well.

rbtcollins commented 1 year ago

FWIW, I also hit this using streams, solved it with an explicitly typed Pin temporary variable around the entire combinator driven expression.

Munksgaard commented 1 year ago

I don't know if it is related, but I had a similar error using Rocket on rustc 1.70.0-nightly (f63ccaf25 2023-03-06):

error: higher-ranked lifetime error
   --> src/main.rs:186:1
    |
186 | #[get("/")]
    | ^^^^^^^^^^^
    |
    = note: could not prove `Pin<Box<[async block@src/main.rs:186:1: 186:12]>>: CoerceUnsized<Pin<Box<(dyn futures::Future<Output = Outcome<rocket::Response<'f>, Status, rocket::Data<'g>>> + std::marker::Send + 'h)>>>`
    = note: this error originates in the attribute macro `get` (in Nightly builds, run with -Z macro-backtrace for more info)

Here's the function in full:


#[get("/")]
async fn index<'a>(client: &State<reqwest::Client>, token: Token) -> Result<Template, String> {
    let (accounts, accounting_years) = try_join!(
        get_accounts(client, &token),
        get_accounting_years(client, &token)
    )?;

    let entries: Vec<Entry> = stream::iter(&accounting_years)
        .then(|accounting_year| {
            let client = &client;
            let token = &token;
            let year = &accounting_year.year;
            async move { get_entries(client, token, &year).await }
        })
        .try_concat()
        .await?;

    Ok(Template::render(
        "index",
        context! { accounts: accounts, accounting_years: &accounting_years },
    ))
}
zjp-CN commented 1 year ago

Here's a case too: playground

#[async_trait]
trait FromText<'r> {
    type Output;
    async fn parse(text: &'r Text) -> Self::Output;
}

// ok
#[async_trait]
impl<'r, A> FromText<'r> for (A,)
where
    A: FromText<'r>,
{
    type Output = (A::Output,);

    async fn parse(text: &'r Text) -> Self::Output {
        let a = A::parse(text).await;
        (a,)
    }
}

// error: higher-ranked lifetime error, could not prove 
// `Pin<Box<[async block@src/lib.rs:35:52: 39:6]>>: CoerceUnsized<Pin<Box<(dyn Future<Output = (<A as FromText<'f>>::Output, <B as FromText<'g>>::Output)> + Send + 'h)>>>`
#[async_trait]
impl<'r, A, B> FromText<'r> for (A, B)
where
    A: FromText<'r>,
    B: FromText<'r>,
    A::Output: Send,
{
    type Output = (A::Output, B::Output);

    async fn parse(text: &'r Text) -> Self::Output {
        let a = A::parse(text).await;
        let b = B::parse(text).await;
        (a, b)
    }
}
```rust error: lifetime may not live long enough --> src/lib.rs:35:52 | 27 | impl<'r, A, B> FromText<'r> for (A, B) | -- lifetime `'r` defined here ... 35 | async fn parse(text: &'r Text) -> Self::Output { | ____________________________________________________^ 36 | | let a = A::parse(text).await; 37 | | let b = B::parse(text).await; 38 | | (a, b) 39 | | } | |_____^ cast requires that `'r` must outlive `'static` error: higher-ranked lifetime error --> src/lib.rs:35:52 | 35 | async fn parse(text: &'r Text) -> Self::Output { | ____________________________________________________^ 36 | | let a = A::parse(text).await; 37 | | let b = B::parse(text).await; 38 | | (a, b) 39 | | } | |_____^ | = note: could not prove `Pin>: CoerceUnsized>::Output, >::Output)> + Send + 'h)>>>` ```

The solutions:

RReverser commented 1 year ago

Also ran into this, but, thanks to https://github.com/rust-lang/rust/issues/102211#issuecomment-1380167541 at least could pinpoint it to try_flatten_unordered which seems to cause the same issues. Avoiding it in favour of manual try_fold works around the problem but the error is really hard to parse.

MachariaK commented 1 year ago

I am experiencing the same problem:

    let batch_size = 100;
    let step = cmp::min(batch_size, members_data.len());
    for i in (0..members_data.len()).step_by(step) {
        let first = i;
        let mut last = i + batch_size;
        if last > members_data.len() {
            last = members_data.len();
        }
        let batch = &members_data[first..last];
        let member_creates = batch.into_iter().map(|member_data| {
            client.member().create(
                member_data.surname.to_string(),
                member_data.first_name.to_string(),
                member_data.gender.to_string(),
                member_data.date_of_birth.into(),
                member_data.kra_pin.to_string(),
                member_data.nssf_number.to_string(),
                member_data.national_id_number.to_string(),
                member_batch::UniqueWhereParam::IdEquals(new_member_batch.clone().id),
                vec![
                    member::title::set(Some(member_data.clone().title.unwrap_or_default())),
                    member::title::set(Some(member_data.clone().other_names.unwrap_or_default())),
                ],
            )
        });

        //Problematic section - Compiles when this code is commented out

        let new_members: Vec<member::Data> = client
            ._batch(member_creates)
            .await
            .unwrap();
        assert_eq!(new_members.len(), last - first);

        // End of problematic section
        window
            .emit_all("member-added-to-database", Payload { member_index: last })
            .unwrap();
    }

error: higher-ranked lifetime error                         
   --> src\member_functions\create_member_data.rs:180:1
    |
180 |   #[tauri::command]
    |   ^^^^^^^^^^^^^^^^^
DzenanJupic commented 1 year ago

Similar issue here. Was able to boil it down to a minimal example ~without any dependencies~(with only a futures 0.3 dependency): [playground]

error: higher-ranked lifetime error
   = note: could not prove `Pin<Box<[async block@src/lib.rs:21:18: 34:10]>>: CoerceUnsized<Pin<Box<(dyn futures::Future<Output = std::result::Result<Vec<()>, Error>> + std::marker::Send + 'b)>>>`

Removing either the 'async_trait lifetime and using elided lifetimes as well as removing the and_then call gets rid of the error somehow. Both solutions aren't practical in my case though.

Edit:

rustc 1.68.0 (2c8cc3432 2023-03-06)
binary: rustc
commit-hash: 2c8cc343237b8f7d5a3c3703e3a87f2eb2c54a74
commit-date: 2023-03-06
host: x86_64-unknown-linux-gnu
release: 1.68.0
LLVM version: 15.0.6
oeed commented 1 year ago

I too have been running in to this a fair amount while doing refactoring to further adopt async. While attempting to find a work around I also ran in to a few "forcing query with already existing DepNode" panics like in #109655. cargo clean did resolve them.

I don't have a min repro, but a (somewhat involved) workaround for any other people who stumble upon this:

I used the assert_send function above to narrow down the exact location that caused the issue. i.e. if one .await is showing the error, try applying applying assert_send to all of the .awaits within the parent async function (and subsequent tree), it may move from the parent to one of these child .awaits.

Once you have the 'terminal' future (i.e. adding assert_send to child .awaits doesn't move the error) try moving any calculated function parameters to let bindings instead of calculating them when passed:

-  some_fn(hash_map.values().map(|i| *i).collect()).await;

+  let ids = hash_map.values().map(|i| *i).collect();
+  some_fn(ids).await;

I suspect it's something to do with the borrowed hash_map reference being passed to the future?

To be clear, the above example isn't a reproduction of this bug, but more a template of what my 'fix' looked like.

emersonford commented 1 year ago

Was poking at this again last night. This bug is triggered for any auto trait (#13231), e.g.

#![feature(auto_traits)]

auto trait SomeAutoTrait {}

fn main() {
    let _: &dyn SomeAutoTrait = &async {
        let _it = [()].iter().map(|it| None::<()>).flatten();
        async {}.await;
    };
}

will trigger the exact same bug (playground).

I also found it curious that @danielhenrymantilla's first example in this comment produces two error message, whereas the second example produces a single error message. This plus @eholk's comment in #96865 makes me think the root of the issue is with how auto trait resolution(?) is done with HRTBs?

danielhenrymantilla commented 1 year ago

[Replying to @emersonford's post here since it seems more appropriate]


nice post @emersonford 🙏

Regarding:

What's not immediately clear to me though is why this is so specific to auto traits + generators. Even if you don't need to prove Send for the generator, don't you still need to prove that Trait is implemented for fn(&'static ()) inside of the generator interior? I.e., why does

It's specific to:

  1. generators, because they're one place where the lifetimes used by the resulting type/enum that makes the Future are erased: Rust does not have the tools to express self-referential lifetimes, so to avoid being unsound with the "made up" lifetimes that can end up populating the compiler-generated self-referential datastructure backing generators / async, the compiler straight up erases them somehow so as to prevent lifetime-specific impls from applying (since in certain scenarios this could be unsound);
  2. auto-traits since they:
    • leak through -> impl Trait / existential trait abstractions (that is, contrary to other traits which are just stated explicitly as part of the -> impl Trait contract, so that they can be checked within the right context, auto-traits are not enforced there and then, and instead checked later on, in a context with erased lifetimes;
    • in a structural manner, meaning it delegates to each generator capture implementing the trait, as you mentioned, which brings us to 1.

Now, the culprit triggering this is indeed the associated type, <T as Trait>::Assoc. At that point, even if the data structure itself (Foo in my example) unconditionally requires T : Trait, the trait checker will nonetheless try to prove this again, and this time with the added difficulty of having erased lifetimes, even the 'static lifetime.

Workaround for library authors

Is then not to use associated types.

That is, replacing:

struct Foo<T : Trait>(
    T::Assoc,
);

with:

struct Foo<T : Trait<Assoc = R>, R = <T as Trait>::Assoc>(
    R,
    ::core::marker::PhantomData<T>,
);

that is, we introduce a new type R, which defaults to T::Assoc, and we also require that T::Assoc be equal to R. All in all, inference should work just as well, and basically anything else too, but for the type now containing a fully standalone R type. This will avoid the requirement to check T : Trait within an erased-lifetime scenario.

Illustration

Using .flat_map(|x| x) instead of .flatten() dodges the issues, even though they are semantically equivalent. The difference is that flatten does mention <I as Iterator>::Item, whereas flat_map uses a fresh U type parameter which is only constrained to be F::Output for extra trait implementations.


Potentially interesting?

Since using struct X(fn(&'static ()); dodges the issue too (since the X "type constructor" is no longer a "type with a movable lifetime within in"), I got curious about trying to formalize stuff using hand-rolled type constructors, i.e., HKTs:

trait WithLifetime<'a> { type T; }
type Feed<'a, T : WithLifetime<'a>> = T::T;
trait HKT = for<'a> WithLifetime<'a>;

type A = dyn for<'a> WithLifetime<'a, T = fn(&'static ())>;
type B = dyn for<'a> WithLifetime<'a, T = fn(&'a ())>;
  • with macro sugar:

    type A = HKT!(<'a> = fn(&'static ());
    type B = HKT!(<'a> = fn(&'a ());

Here, we have A and B both being fully-fledged types, but which happen to express a form of <'a>-genericity, which could be expressed in the following pseudo-code, sort to speak:

type A<'a> = fn(&'static ());
type B<'a> = fn(&'a ());

And doing that, we could then replace struct Foo<T>(<T as Trait>::Assoc) where T : Trait; definition with:

/// Idea: replace `T` with `Feed<'static, T_>`, for some `T_ : <'_>` "generic generic" type.
struct Foo<T_ : ?Sized + HKT>(
    <Feed<'static, T_> as Trait>::Assoc,
)
where
    Feed<'static, T_> : Trait,
;

With that, using Foo::<A> fails, but using Foo::<B> does not fail, even though they both amount to the same T = fn(&'static ()).

  • I think the distinction stems from A "capturing" 'static, and thus being detected by the lifetime-eraser heuristic,
  • whereas B does not capture anything on its own: it's just its associated type which does, but after being queried with a fed 'static to the WithLifetime trait. And somehow that lifetime in as WithLifetime<'static> does not get erased 🤷
TheZoq2 commented 1 year ago

I think I managed to reproduce the same or a similar issue in my project. This code gives the unhelpful error:

use tokio_stream::StreamExt;

async fn run_ui(tests: Vec<()>) {
    loop {
        let _ = tokio_stream::iter(&tests)
            .then(|_| async {
            })
            .collect::<Vec<_>>()
            .await;
    }
}

#[tokio::main]
async fn main() {
    tokio::spawn(run_ui(vec![]));
}
[dependencies]
tokio = { version = "1.28.2", features = ["full"] }
tokio-stream = "0.1.14"

Passing tests by value instead of by reference on line 4 seems to fix it

vincenzopalazzo commented 1 year ago

can you post also the unhelpful error?

TheZoq2 commented 1 year ago

Ah yes, of course:

  Compiling async_bug v0.1.0 (/tmp/async_bug)
error: higher-ranked lifetime error
  --> src/main.rs:20:5
   |
20 |     tokio::spawn(run_ui(vec![]));
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: could not prove `impl Future<Output = ()>: Send`

error: could not compile `async_bug` (bin "async_bug") due to previous error
hcsch commented 1 year ago

To add another example of code that causes the error:

trait Trait {
    type Item;
}

impl<B, I, F> Trait for (I, F)
where
    F: FnMut(I) -> B,
{
    type Item = I;
}

struct Struct1<I: Trait>(I::Item);

impl<I: Trait> Struct1<I> {
    fn new(i: I) -> Self {
        todo!()
    }
}

async fn af3() {}

async fn af2(_: &mut (), i: &()) {
    let d = Struct1::new((i, |c| c));
    af3().await
}

async fn af1() {
    let a = Box::leak(Box::new(()));
    let b = Box::leak(Box::new(()));
    spawn(af2(a, b));
}

pub fn spawn<T: Send>(future: T) {
    todo!()
}
error: higher-ranked lifetime error
  --> lib.rs:30:5
   |
30 |     spawn(af2(a, b));
   |     ^^^^^^^^^^^^^^^^
   |
   = note: could not prove `impl Future<Output = ()>: Send`

I've hit this in a codebase of mine that uses an Iterator with map() and peekable() in an async member function, holding that across an await point. The member function is called in an async function the future of which is passed to tokio::spawn.

Somewhat unrelated: https://github.com/marxin/cvise helped a lot in getting this example down to what it is now (though you need to fix the syntax after using it, since it's designed for C).

yuri-rs commented 1 year ago

I face this issue with futures::stream::Buffered (and BufferUnordered) wrappers. The following code reproduce the issue:

use futures::stream::StreamExt;

fn higher_ranked_lifetime_error(
) -> std::pin::Pin<std::boxed::Box<dyn std::future::Future<Output = ()> + Send>> {
    Box::pin(async {
        let vec_with_readers: Vec<Box<dyn std::io::Read + Send>> = Vec::new();

        let _ = futures::stream::iter(
            vec_with_readers
                .into_iter()
                .map(|_| futures::future::ready::<()>(())),
        )
        .buffered(1)
        .count()
        .await;

        ()
    })
}

while removing .buffered(1) allow the code to compile. Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=9cd1b46b17239d28184e14251910e8f7

Could anyone point me to the workaround for the issue please?

Workaround from https://github.com/rust-lang/rust/issues/102211#issuecomment-1513931928 will require changes for futures::stream::Buffered, so it is not what I'm looking for.

zertyz commented 1 year ago

I'm sure this is obvious to you, I just want to make sure it's documented somewhere:

switching let b: Vec<&[String]> = b.chunks(2).collect(); to let b: Vec<Vec<String>> = b.chunks(2).map(|c| c.to_vec()).collect(); works around the issue.

Or better yet, using @frxstrem's suggestion of replacing the closure for a function pointer and skipping copying things around unnecessarily: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=a4f501590c778b0dbe78ca620af23b4d

veeshi commented 1 year ago

Could anyone point me to the workaround for the issue please?

Workaround from #102211 (comment) will require changes for futures::stream::Buffered, so it is not what I'm looking for.

I'm facing the same issue after introducing BufferUnordered, attempted to Pin and Box the future that is causing the error but still getting the same error.

Anyone know another workaround?

Palmik commented 1 year ago

In my case the issue was when registering a route handler with axum for foo below. Note that foo itself compiles when used on its own, it's only when used as an axum handler that it runs into this issue.

To fix this, I had to change do_something_with_xs to take xs: Vec<X> instead of xs: &[X]. This is unintuitive to me, since it's not clear to me why the lifetimes from do_something_with_xs would be able to "escape" into the type of foo.

Another fix would be to not use buffered, but that's not acceptable in my case.

async fn foo() -> Result<()> {
  let xs = vec![...];
  do_something_with_xs(&xs).await
}

async fn do_something_with_xs(xs: &[X]) -> Result<()> {
  // Broken version:
  let ys = futures::stream::iter(xs.iter().map(|x| do_something_with_x(x)));

  // Working version when using xs: Vec<X>
  // let ys = futures::stream::iter(xs.into_iter().map(|x| async move { do_something_with_x(&x).await }));

  // Working version when using xs: &[X] without buffered
  // let ys = futures::stream::iter(xs.iter()).then(|x| do_something_with_x(x));

  tokio::pin!(stream);

  while let Some(y) = ys.next().await {
    // Do something with y
  }
}

// You can keep the reference here
async fn do_something_with_x(x: &X) -> Result<Y> {
  // ...
}
UVUUUVUU commented 1 year ago

i have a same problem.

pub async fn select_paginate() {
    match get_sql_db_conn().await.unwrap() {
        DBPool::AsyncPgPool(pool) => {
            let mut conn = pool.get().await.unwrap();
            conn.transaction::<(), diesel::result::Error, _>(|conn_mutex| async move {
                    let sql = "SELECT * FROM tableone WHERE id = 1".to_string();
                    diesel::sql_query(&sql).execute(conn_mutex).await.unwrap();
                    let mut db_query = tableone.into_boxed();
                    db_query = db_query.filter(title.eq("shenhu"));
                    db_query = db_query.limit(1).offset(1);
                    let result = db_query.paginate(1, 1).load_and_total::<Tableone>(conn_mutex).await.unwrap();
                    println!("{:?}", result);
                    Ok(())
                }.scope_boxed())
            .await;
        }
        _ => {}   
    };
}

the error:

higher-ranked lifetime error
could not prove `[async block@src/utils/select_async.rs:84:75: 93:18]: std::marker::Send`