Unclear error when trying to write high-order function accepting an `async` function that itself takes a reference

[pvillela]

I tried this code:

use std::future::Future;

struct X;

trait Tx {}

fn main() {
    higher_order(f_x);
    higher_order_x(f_x);

    higher_order(f_tx);
    higher_order_tx(f_tx);

    higher_order_tx_nf(f_tx_nf);
}

fn higher_order<I, FUT>(_f: fn(I) -> FUT)
where
    FUT: Future<Output = ()>,
{
}

fn higher_order_x<FUT>(_f: fn(X) -> FUT)
where
    FUT: Future<Output = ()>,
{
}

fn higher_order_tx<FUT>(_f: fn(&dyn Tx) -> FUT)
where
    FUT: Future<Output = ()>,
{
}

fn higher_order_tx_nf(_f: fn(&dyn Tx) -> ()) {}

async fn f_x(_input: X) {}

async fn f_tx(_input: &dyn Tx) {}

fn f_tx_nf(_input: &dyn Tx) {}

I expected the code to compile without errors

Instead, I got a compilation error only for the line higher_order_tx(f_tx); in main:

error[E0308]: mismatched types
  --> general/src/bin/higher_order_async.rs:12:21
   |
12 |     higher_order_tx(f_tx);
   |     --------------- ^^^^ one type is more general than the other
   |     |
   |     arguments to this function are incorrect
   |
   = note: expected fn pointer `for<'a> fn(&'a (dyn Tx + 'a)) -> _`
                 found fn item `for<'a> fn(&'a (dyn Tx + 'a)) -> impl Future<Output = ()> {f_tx}`
   = note: when the arguments and return types match, functions can be coerced to function pointers
note: function defined here
  --> general/src/bin/higher_order_async.rs:29:4
   |
29 | fn higher_order_tx<FUT>(_f: fn(&dyn Tx) -> FUT)
   |    ^^^^^^^^^^^^^^^      ----------------------

Meta

I am using the stable version below. With the current nightly version rustc 1.72.0-nightly (83964c156 2023-07-08), I get error[E0635]: unknown feature 'proc_macro_span_shrink'.

rustc --version --verbose:

rustc 1.70.0 (90c541806 2023-05-31)
binary: rustc
commit-hash: 90c541806f23a127002de5b4038be731ba1458ca
commit-date: 2023-05-31
host: x86_64-unknown-linux-gnu
release: 1.70.0
LLVM version: 16.0.2

Backtrace

``` ```

[WaffleLapkin]

The type error is correct, I believe.

f_tx has type like for<'a> fn(&'a (dyn Tx + 'a)) -> impl Future<Output = ()> + 'a {f_tx}, i.e. the return type is using the lifetime of the for<'a> binder. higher_order_tx however expects that the return type (FUT) is a single type, that does not depend on the binder introduced implicitly by the ref parameter.

This is somewhat of a fundamental (types people can correct my if I'm wrong...) issue with the Rust type system, it is very hard to write functions that accept async functions taking a reference, because it's impossible to write the correct FnOnce bound...

You can somewhat workaround this issue by introducing a trait:

trait AsyncBorrowFn<'a, A: ?Sized + 'a>: Fn(&'a A) -> Self::Fut {
    type Out;
    type Fut: Future<Output = Self::Out> + 'a;
}

impl<'a, A, F, Fut> AsyncBorrowFn<'a, A> for F 
where
   A: ?Sized + 'a,
   F: Fn(&'a A) -> Fut,
   Fut: Future + 'a,
{
    type Out = Fut::Output;
    type Fut = Fut;
}

fn higher_order_tx(f: impl for<'a> AsyncBorrowFn<'a, dyn Tx + 'a, Out = ()>) { /* ... */ }

(play)

[pvillela]

Thanks for your thoughtful reply, @WaffleLapkin. Based on your code sample, I tried something simpler (see below) which seems to work fine. All I had to do was add a lifetime parameter 'a to higher_order_tx.

The compiler error message should suggest the addition of a lifetime parameter for the reference argument.

use core::future::Future;

#[tokio::main]
async fn main() {
    _ = higher_order_tx(f_tx).await;
}

trait Tx {
    fn show(&self);
}

impl Tx for u32 {
    fn show(&self) {
        println!("Value = {self}");
    }
}

async fn f_tx(input: &dyn Tx) {
    println!("f_tx executed");
    input.show();
}

async fn higher_order_tx<'a, Fut>(f: fn(&'a dyn Tx) -> Fut)
where
    Fut: Future<Output = ()>,
{
    f(&42u32).await;
}

[WaffleLapkin]

@pvillela yes, this is also an option, although do note that this requires passing parameters specifically with lifetime 'a. so, for example, you can't pass references to local variables there.

[pvillela]

Your point is well taken, @WaffleLapkin. I used your AsyncBorrowFn to implement a partial application higher-order function for a little architecture framework I'm developing. It works fine when the resulting closure is rendered as an Fn that returns a box-pinned future, but it doesn't work when the resulting closure is rendered as an AsyncBorrowFn. Please see below:

use std::future::Future;
use std::pin::Pin;

/// Represents an async function with a single argument that is a reference.
pub trait AsyncBorrowFn1b1<'a, A: ?Sized + 'a>: Fn(&'a A) -> Self::Fut {
    type Out;
    type Fut: Future<Output = Self::Out> + 'a;
}

impl<'a, A, F, Fut> AsyncBorrowFn1b1<'a, A> for F
where
    A: ?Sized + 'a,
    F: Fn(&'a A) -> Fut + 'a,
    Fut: Future + 'a,
{
    type Out = Fut::Output;
    type Fut = Fut;
}

/// Represents an async function with 2 arguments; the first is not a reference, the last is a reference.
pub trait AsyncBorrowFn2b2<'a, A1, A2: ?Sized + 'a>: Fn(A1, &'a A2) -> Self::Fut {
    type Out;
    type Fut: Future<Output = Self::Out> + 'a;
}

impl<'a, A1, A2, F, Fut> AsyncBorrowFn2b2<'a, A1, A2> for F
where
    A2: ?Sized + 'a,
    F: Fn(A1, &'a A2) -> Fut + 'a,
    Fut: Future + 'a,
{
    type Out = Fut::Output;
    type Fut = Fut;
}

/// Partial application for async function, where the resulting closure returns a box-pinned future.
pub fn partial_apply_boxpin<A1, A2, T>(
    f: impl for<'a> AsyncBorrowFn2b2<'a, A1, A2, Out = T>,
    a1: A1,
) -> impl for<'a> Fn(&'a A2) -> Pin<Box<dyn Future<Output = T> + 'a>>
where
    A1: Clone,
    A2: ?Sized, // optional Sized relaxation
{
    move |a2| {
        let y = f(a1.clone(), a2);
        Box::pin(y)
    }
}

/// Partial application for async function, where the result is an AsyncBorrowFn1r1.
pub fn partial_apply<A1, A2, T>(
    f: impl for<'a> AsyncBorrowFn2b2<'a, A1, A2, Out = T> + 'static,
    a1: A1,
) -> impl for<'a> AsyncBorrowFn1b1<'a, A2, Out = T>
where
    A1: Clone + 'static,
    A2: ?Sized + 'static,
{
    move |a2| {
        let y = f(a1.clone(), a2);
        y
    }
}

async fn f(i: u32, j: &u32) -> u32 {
    i + j
}

#[tokio::main]
async fn main() {
    let f_part = partial_apply_boxpin(f, 40);
    println!("{}", f_part(&2).await);

    let f_part = partial_apply(f, 40);
    println!("{}", f_part(&2).await);
}

The higher-order function partial_apply_boxpin compiles fine but I get a compilation error for partial_apply:

error: implementation of `AsyncBorrowFn1b1` is not general enough
  --> general/src/bin/async_borrow_fn_simplified.rs:60:5
   |
60 | /     move |a2| {
61 | |         let y = f(a1.clone(), a2);
62 | |         y
63 | |     }
   | |_____^ implementation of `AsyncBorrowFn1b1` is not general enough
   |
   = note: `[closure@general/src/bin/async_borrow_fn_simplified.rs:60:5: 60:14]` must implement `AsyncBorrowFn1b1<'0, A2>`, for any lifetime `'0`...
   = note: ...but it actually implements `AsyncBorrowFn1b1<'1, A2>`, for some specific lifetime `'1`

Your comments would be greatly appreciated.

[danielhenrymantilla]

@pvillela you can write this:

/// Partial application for async function, where the result is an AsyncBorrowFn1r1.
pub fn partial_apply<A1, A2, F, T>(
    f: F,
    a1: A1,
) -> impl for<'a> AsyncBorrowFn1b1<'a, A2, Out = T>
where
    A1: Clone + 'static,
    A2: ?Sized + 'static,
    F: for<'a> AsyncBorrowFn2b2<'a, A1, A2, Out = T> + 'static,
{
    fn nudge_inference<A1, A2, F, T, C>(
        closure: C,
    ) -> C
    where
        // this promotes the literal `|a2| …` closure to "infer"
        // (get imbued with) the right higher-order fn signature.
        // See https://docs.rs/higher-order-closure for more info
        // v
        C: Fn(&A2) -> <F as AsyncBorrowFn2b2<'_, A1, A2>>::Fut,

        A1: Clone + 'static,
        A2: ?Sized + 'static,
        F: for<'a> AsyncBorrowFn2b2<'a, A1, A2, Out = T> + 'static,
    {
        closure
    }

    nudge_inference::<A1, A2, F, T, _>(move |a2| {
        f(a1.clone(), a2)
    })
}

• Playground

[pvillela]

Thank you @danielhenrymantilla, this is great.