Why implementation of iterator is not generic enough in async context?

[Stargateur]

Cross posting stackoverflow because it's look like a compiler bug/limitation.

---

Given the following snippet:

use futures::stream::{self, StreamExt};

async fn from_bar(bar: &[Vec<&u8>]) {
    let x = bar.iter().flat_map(|i| i.iter().map(|_| async { 42 }));
    let foo: Vec<_> = stream::iter(x).collect().await;
}

#[tokio::main]
async fn main() {
    for bar in vec![] {
        tokio::spawn(async {
            from_bar(bar).await;
        });
    }
}

I get the following errors:

error[E0308]: mismatched types
  --> src/main.rs:11:9
   |
11 |         tokio::spawn(async {
   |         ^^^^^^^^^^^^ one type is more general than the other
   |
   = note: expected type `std::ops::FnOnce<(&&u8,)>`
              found type `std::ops::FnOnce<(&&u8,)>`

error: implementation of `std::iter::Iterator` is not general enough
    --> src/main.rs:11:9
     |
11   |           tokio::spawn(async {
     |           ^^^^^^^^^^^^ implementation of `std::iter::Iterator` is not general enough
     |
     = note: `std::iter::Iterator` would have to be implemented for the type `std::slice::Iter<'0, &u8>`, for any lifetime `'0`...
     = note: ...but `std::iter::Iterator` is actually implemented for the type `std::slice::Iter<'1, &u8>`, for some specific lifetime `'1`

I was expecting no error because the lifetimes seem to be correct to me. Note that removing main() or removing the code inside from_bar() both eliminate the errors. Not only that, the error messages are also very strange. They may be related to a regression in the compiler, though more than that they seem to be in the wrong place (maybe related).

Version rustc 1.43.0 (4fb7144ed 2020-04-20):

[dependencies]
futures = '0.3.1'

[dependencies.tokio]
version = '0.2'
features = ['full']

---

Maybe related https://github.com/rust-lang/rust/issues/64650

[tmandry]

Triage: This error message is clearly confusing. We should see how we can improve it, or if this is already part of #64650 or #64552.

[jorendorff]

This bit me today. Reduced test case (playground):

pub trait Robot {
    type Id;
}

pub type DynRobot = Box<dyn Robot<Id = u32> + Send>;

impl Robot for DynRobot {
    type Id = u32;
}

struct IRobot<R: Robot> {
    id: R::Id,
    robot: R,
}

// stand-in for tokio::spawn
fn this_is_send<T: Send>(value: T) -> T {
    value
}

fn test(source: DynRobot) {
    let _my_task = this_is_send(async move {
        let _my_iter = IRobot {
            id: 32,
            robot: source,
        };
        tokio::task::yield_now().await;
    });
}

rustc output:

  Compiling playground v0.0.1 (/playground)
error: implementation of `Robot` is not general enough
  --> src/lib.rs:22:20
   |
22 |     let _my_task = this_is_send(async move {
   |                    ^^^^^^^^^^^^ implementation of `Robot` is not general enough
   |
   = note: `Box<(dyn Robot<Id = u32> + Send + '0)>` must implement `Robot`, for any lifetime `'0`...
   = note: ...but `Robot` is actually implemented for the type `Box<(dyn Robot<Id = u32> + Send + 'static)>`

error: aborting due to previous error

error: could not compile `playground`

To learn more, run the command again with --verbose.

[blueforesticarus]

This results in very confusing errors sometimes multiple function calls removed from the change. The join_all version works, but when changing to the version below, an error appears all the way in new(), despite A) new not changing B) the function being spawned not changing C) no function's type signature changing https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=a3d50c96bd2030e4c213e24f0ab01386

/*
error: implementation of `Iterator` is not general enough
  --> src/platform/mod.rs:13:9
   |
13 |         tokio::spawn(Thing::task(thing.clone()));
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `Iterator` is not general enough
   |
   = note: `Iterator` would have to be implemented for the type `std::slice::Iter<'0, i32>`, for any lifetime `'0`...
   = note: ...but `Iterator` is actually implemented for the type `std::slice::Iter<'1, i32>`, for some specific lifetime `'1`

error: implementation of `FnOnce` is not general enough
  --> src/platform/mod.rs:13:9
   |
13 |         tokio::spawn(Thing::task(thing.clone()));
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `FnOnce` is not general enough
   |
   = note: closure with signature `fn(&'0 i32) -> impl futures::Future<Output = ()>` must implement `FnOnce<(&i32,)>`, for any lifetime `'0`...
   = note: ...but it actually implements `FnOnce<(&i32,)>`
*/

use std::{sync::Arc, time::Duration};

use futures::{future::join_all, StreamExt};
struct Thing{}

impl Thing {
    pub async fn new() -> Arc<Self> {
        let thing = Arc::new(Thing{});
        tokio::spawn(Thing::task(thing.clone()));
        //Thing::task(thing.clone()).await; //This compiles
        thing
    }

    pub async fn task(self : Arc<Self>) {
        self.task_inner().await;
    }

    pub async fn task_inner(&self){
        let v = vec![1,2,3];

        let futs = v.iter() // make this into_iter() and it works
            .map(|a| async move {});

        //uncomment this and it works
        //let futs : Vec<_> = futs.collect();

        //always works:
        //join_all(futs).await;

        //results in error all the way in new()
        let stream = futures::stream::iter(futs).buffer_unordered(10);
        let results = stream.collect::<()>().await;
    }
}

Notably this only occurs with borrowed iterators, collecting or using into_iter fixes the issue. This is hinted at by the error message, so maybe it is the right message, just in the wrong place.

One question is, should task_inner compile at all? Rustc doesn't have an issue with calling it outside of a tokio::spawn.

[Zane-XY]

This results in very confusing errors sometimes multiple function calls removed from the change. The join_all version works, but when changing to the version below, an error appears all the way in new(), despite A) new not changing B) the function being spawned not changing C) no function's type signature changing https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=a3d50c96bd2030e4c213e24f0ab01386

/* error: implementation of Iterator is not general enough --> src/platform/mod.rs:13:9	13	tokio::spawn(Thing::task(thing.clone()));	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of Iterator is not general enough

= note: Iterator would have to be implemented for the type std::slice::Iter<'0, i32>, for any lifetime '0... = note: ...but Iterator is actually implemented for the type std::slice::Iter<'1, i32>, for some specific lifetime '1

error: implementation of FnOnce is not general enough --> src/platform/mod.rs:13:9	13	tokio::spawn(Thing::task(thing.clone()));	^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of FnOnce is not general enough

= note: closure with signature fn(&'0 i32) -> impl futures::Future<Output = ()> must implement FnOnce<(&i32,)>, for any lifetime '0... = note: ...but it actually implements FnOnce<(&i32,)> */

use std::{sync::Arc, time::Duration};

use futures::{future::join_all, StreamExt}; struct Thing{}

impl Thing { pub async fn new() -> Arc { let thing = Arc::new(Thing{}); tokio::spawn(Thing::task(thing.clone())); //Thing::task(thing.clone()).await; //This compiles thing }

pub async fn task(self : Arc<Self>) {
    self.task_inner().await;
}

pub async fn task_inner(&self){
    let v = vec![1,2,3];

    let futs = v.iter() // make this into_iter() and it works
        .map(|a| async move {});

    //uncomment this and it works
    //let futs : Vec<_> = futs.collect();

    //always works:
    //join_all(futs).await;

    //results in error all the way in new()
    let stream = futures::stream::iter(futs).buffer_unordered(10);
    let results = stream.collect::<()>().await;
}

} Notably this only occurs with borrowed iterators, collecting or using into_iter fixes the issue. This is hinted at by the error message, so maybe it is the right message, just in the wrong place.

One question is, should task_inner compile at all? Rustc doesn't have an issue with calling it outside of a tokio::spawn.

Is there any explanation for the cause of this? This seems to be a very frequently reported issue, and I didn't see any plausible explanation from the rust-lang team, other than just keep cross linking the similar issue in different threads. Can't believe this is still happening after 4 years! @tmandry