Tracking issue for Fn traits (`unboxed_closures` & `fn_traits` feature)

[aturon]

Tracks stabilization for the Fn* traits.

Random bugs:

• [x] https://github.com/rust-lang/rust/issues/45510 – type-based dispatch not working
• [ ] https://github.com/rust-lang/rust/issues/42736 – foo() sugar doesn't work where you have &Foo: FnOnce

[nagisa]

Inability to delegate calls to other FnOnce implementors like this:

struct A<T>(T);

impl<T, Args> FnOnce<Args> for A<T>
where T: FnOnce<Args> {
    type Output = <T as FnOnce<Args>>::Output;
    fn call_once(self, args: Args) -> Self::Output { FnOnce::call_once(self.0, args) }
}

is the reason I have a safety issue in libloading.

[petrochenkov]

FnOnce::Output is stabilized in https://github.com/rust-lang/rust/pull/34365

[nrc]

Could someone summarise what is blocking stabilisation here please?

[nagisa]

@nrc uncertainty about Args being the right thing given the possibility of variadic generics coming along around 2030 is the most major reason this is unstable.

[aturon]

@nrc also possibly some questions around the inheritance relationship, see https://github.com/rust-lang/rust/issues/19032

[brunoczim]

There should be a way of casting fn(A) -> B to Fn(A) -> B

[SimonSapin]

@brunoczim That coercion already happens implicitly, but Fn() as a type is a trait object so it needs to be behind some kind of pointer:

use std::rc::Rc;
fn main() {
    let _: &Fn() = &main;
    let _: Box<Fn()> = Box::new(main);
    let _: Rc<Fn()> = Rc::new(main);
}

[Michael-F-Bryan]

One issue we've been discussing on https://github.com/TheDan64/inkwell/issues/5 is the ability to mark something as unsafe to use. What about having an UnsafeFn marker trait which could be used to tell the compiler a callable is unsafe to call?

For context, inkwell is a wrapper around LLVM and we're trying to figure out how to return a function pointer to a JIT compiled function, when calling the function is fundamentally unsafe for the same reasons FFI code is unsafe to call.

[CodeSandwich]

There is another reason to add UnsafeFn: currently unsafe fns don't implement Fn. There is no way to pass unsafe fn as a function argument, they are second class citizen: example.

UnsafeFn could be implemented for things implementing Fn, so Fns can be used wherever UnsafeFn is required. There probably also should be UnsafeFnMut and UnsafeFnOnce.

[alexreg]

@Michael-F-Bryan @CodeSandwich This sounds like something for which an RFC would really be appreciated. It probably wouldn't be an overly long or intricate one to write, even. I would support it, for sure, and judging by an issue I saw about this not long ago (a long-standing one), others would too.

[CodeSandwich]

@alexreg Ok, I'll prepare it in spare time. Unfortunately I lack knowledge and experience to actually implement it or even fully understand the idea.

[alexreg]

@CodeSandwich Don't worry, so do I! Maybe get yourself on Rust's Discord (#design channel) and we can discuss it with some people who really know the nitty gritty? You can ping me there, same username.

[RalfJung]

With every unsafe function comes a manually written "contract" saying when that function may or may not be called.

With UnsafeFn, who is setting that contract?

[Michael-F-Bryan]

With UnsafeFn, who is setting that contract?

I'd say this is done on a case-by-case basis. It's really hard to specify the various invariants and assumptions you make in unsafe code in something as rigid as a type system, so anything that accepts an UnsafeFn would probably also need to document its assumptions.

Before writing up a RFC I thought I'd make a post on the internal forum. It'd be nice to hear what opinions other people have on this topic and the different solutions they come up with.

[eira-fransham]

I don't know how one would realistically implement this, but probably the ideal semantics is to have any function that takes an unsafe function as an argument to also be unsafe.

[SimonSapin]

A different interface/API likely requires a separate set of traits. Though multiplying the number of traits doesn’t sound great, especially if we later want to also support const fn closures (and unsafe const fn?).

[audreyality]

Is there a way to implement these as wrapper traits? Unsafe<T> where T : FnOnce, Const<T> where T : FnOnce, Async<T> where T : FnOnce and so on?

[peterjoel]

UnsafeFn sounds like an unsound abstraction. When using an unsafe function, it's your responsibility to read documentation and understand all assumptions and invariants that need to be satisfied. This cannot be abstracted away because those invariants and assumptions are individual to the function in question.

Personally, I think unsafe functions should just not implement any of the Fn traits. It's up to the caller to wrap the unsafe function in another function in order to pass it around. Unsafe code doesn't need to meet the same standards of ergonomics as safe code, provided that it's still possible to get optimal performance.

For example, CodeSandwich's example can be "fixed" using a closure: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2015&gist=308c3ee61419427c617879fc4cb82738

[alercah]

I agree. UnsafeFn is meaningless and breaks the mechanics of unsafe proof obligations.

[brunoczim]

I understand the argument against Unsafe Fn. But wouldn't this apply to unsafe fn pointers too?

[peterjoel]

I understand the argument against Unsafe Fn. But wouldn't this apply to unsafe fn pointers too?

Yes, I would say so.

[lucasjinreal]

what does the difference between Fn and fn?

[csmoe]

@jinfagang cc https://doc.rust-lang.org/std/ops/trait.Fn.html

[syntacticsugarglider]

What's the status of this? There are a number of F-unboxed_closures issues but this tracking issue doesn't appear to be... well, tracking the issue. I'm currently working with a project that requires erased closure types and supports #![no_std] and SmallBox-style tricks don't really help when I'm writing library code that lacks any information about the maximum size of the closure state.

[rcls]

Is it intentional that the code below compiles? It looks wrong to me, as it appears to move out of a mutable reference.

#![feature(fn_traits)]
#![feature(unboxed_closures)]
pub struct A {}
impl std::ops::FnMut<()> for A {
    extern "rust-call" fn call_mut(&mut self, args: ()) { self.call_once(args) }
}
impl std::ops::FnOnce<()> for A {
    type Output = ();
    extern "rust-call" fn call_once(self, _args: ()) { }
}

For comparison, this doesn't compile:

struct A { }
impl A {
    pub fn my_mut(&mut self) { self.my_once() }
    pub fn my_once(self) { }
}

[cuviper]

@rcls I think that's not calling your FnOnce, but rather the blanket impl<F: FnMut> FnOnce for &mut F that uses call_mut, so you'll have infinite recursion.

[joshtriplett]

We definitely want to stabilize the Fn family of traits at some point, allowing people to impl them.

Marking this as "design-concerns" because we need to determine if we should wait for variadic generics or stabilize the tuple-based rust-call ABI.

[bjorn3]

Should Args be turned into an associated type to prevent people from using it to implement operator overloading based on argument type?

[Lokathor]

no

[danielhenrymantilla]

@bjorn3 wouldn't that break higher-order signatures (which are implemented as overloads over the input lifetimes)? FWIW, this question is related to that of the Resume parameter for Generators.

a silly thing 🙈

```rust trait FnOnce<'lifetimes..> { // variadic lifetime-generics? type Args; type Output; extern "rust-call" fn call_once (self, _: Self::Args) -> Self::Output where Self : Sized, ; } ```

[bjorn3]

wouldn't that break higher-order signatures (which are implemented as overloads over the input lifetimes)?

Right, didn't think about that.

[clarfonthey]

Silly question, but is there any particular reason why the Fn* traits actually need to stabilise the rust-call ABI in order to be implementable?

I mean, we already have the custom Fn(A, B, ...) -> C syntax as sugar for Fn<(A, B,), Output = C>, so, I don't think it'd be unreasonable to adopt a special syntax just for implementing them too.

Maybe something like:

struct MyFn(u32);
impl MyFn {
    fn(self, x: u32, y: u32) -> u32 {
        x + y + self.0
    }
}

Could get desugared to:

struct MyFn(u32);
impl FnOnce<(u32, u32)> for MyFn {
    type Output = u32;
    extern "rust-call" fn call_once(self, (x, y): (u32, u32)) -> u32 {
        /* body */
    }
}

[yasuo-ozu]

@clarfonthey I think there's some needs to implement Fn* traits for arbitrary types.

1. Giving multiple function signatures to an object.
2. In following URL, there's some difficulty defining trait Handler, so we want to use Fn* trait directly. https://users.rust-lang.org/t/type-inference-in-closures/78399

[bjorn3]

Giving multiple function signatures to an object.

This is something I think we shouldn't support in the first place, even with Fn*.

In following URL, there's some difficulty defining trait Handler, so we want to use Fn* trait directly. https://users.rust-lang.org/t/type-inference-in-closures/78399

If I understand it correctly, preventing multiple impls of Fn* would fix this issue.

[yasuo-ozu]

@bjorn3 Thanks for replying.

we shouldn't support in the first place

preventing multiple impls of Fn*

Do you have any standings for preventing higher-order signaitures as mentioned below? https://github.com/rust-lang/rust/issues/29625#issuecomment-991226468

[bjorn3]

I don't know how to support higher-order signatures while at the same time preventing multiple call signatures for a type other than keeping Fn* perma-unstable.

[peterjoel]

I don't know how to support higher-order signatures while at the same time preventing multiple call signatures for a type other than keeping Fn* perma-unstable.

Changing the Fn traits to make the arguments an associated type could accomplish that. It would break the symmetry with Fn() constraints though.

Peter

[fogti]

@peterjoel but that wouldn't scale to function-like objects with generic arguments, for which an associated type wouldn't work. What we really want to prevent here are functions with multiple simultaneous argument counts (because varying argument types is already possible, although a bit complicated, using sealed traits), which could be solved with 2 additional traits (one for tuples, one for functions, which binds tuples to the count of directly contained objects, and functions to their argument counts, either via type-level integers, or using const generics + associated constants)

[bjorn3]

but that wouldn't scale to function-like objects with generic arguments, for which an associated type wouldn't work.

Closures can't be generic either.

[yasuo-ozu]

@zseri

What we really want to prevent here are functions with multiple simultaneous argument counts

Is this really harmful? it looks fine in current unstabilized fn-traits: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=157729b98808b439ecc992e4ba59273e

[bjorn3]

It may be fine from a technical perspective, but from a language perspective we have chosen to not allow function overloading. See for example https://internals.rust-lang.org/t/justification-for-rust-not-supporting-function-overloading-directly/7012, https://users.rust-lang.org/t/is-it-possible-to-specialize-hashmap-index-for-copy-types/7750/5, https://users.rust-lang.org/t/what-is-the-reason-for-not-having-overloaded-versions-of-fn/55208 and https://users.rust-lang.org/t/is-there-a-simple-way-to-overload-functions/30937.

[nyxtom]

I recently did some experiments with function traits that requires the use of trait specialization to get it working.

https://github.com/nyxtom/composition/blob/main/src/lib.rs

pub trait Func<Args, T> {
    type Output;
    fn call(&self, args: Args) -> Self::Output;
}

// Default implementation of a func for T as output
impl<A, B, Args, T> Func<Args, ()> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Fn<T>,
{
    type Output = B::Output;

    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call(args)
    }
}

// Subset of (A, B) T is (T,)
impl<A, B, Args, T> Func<Args, (T,)> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Fn<(T,)>,
{
    type Output = B::Output;
    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call((args,))
    }
}

Specifically it allowed me to compose between two functions that is already a tuple being returned and apply them as arguments, or in the natural case where the return value is a single type.

fn foo() {}

fn test() -> i32 {
    3
}
fn plus(a: i32) -> i32 {
    a + 1
}
fn multiply(a: i32, b: i32) -> i32 {
    a * b
}
fn output() -> (i32, i32) {
    (4, 2)
}

fn assert_func<Args, T>(_: impl Func<Args, T>) {}

#[test]
fn test_assert_funcs() {
    assert_func((foo, foo));
    assert_func((foo, test));
    assert_func((test, plus));
    assert_func((plus, plus));
    assert_func((multiply, plus));
    assert_func((output, multiply));
}

Does this constitute function overloading or just a use of trait specialization? I did later expand on this to support more composition (than 1 argument) by using the recursive tuple structure like so:

// Subset of (A, B) where A is already a tuple that implements Func
impl<A, B, Args, T, F> Func<Args, ((), (), F)> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Func<T, F>,
{
    type Output = B::Output;
    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call(args)
    }
}

// Subset of (A, B) where is A is Func and B takes (T,)
impl<A, B, Args, T, F> Func<Args, ((), (T,), F)> for (A, B)
where
    A: Fn<Args, Output = T>,
    B: Func<(T,), F>,
{
    type Output = B::Output;
    #[inline]
    fn call(&self, args: Args) -> Self::Output {
        let args = self.0.call(args);
        self.1.call((args,))
    }
}

This allowed me to perform expressions like so:

#[test]
fn test_assert_nested_func() {
    assert_func((multiply, (plus, plus)));
    assert_func((plus, (plus, plus)));
    assert_func((output, (multiply, plus)));
}

This does require the use of the Fn<T> rather than the parenthetical notation but it's quite useful here as it allows some nice composition to happen. As well, you can make use of variadics with just a macro that turns it into a recursive structure.

pub struct Function<F, T>(F, PhantomData<T>);

impl<F, Args, T> Fn<Args> for Function<F, T>
where
    F: Func<Args, T>,
{
    extern "rust-call" fn call(&self, args: Args) -> Self::Output {
        self.0.call(args)
    }
}

impl<F, Args, T> FnMut<Args> for Function<F, T>
where
    F: Func<Args, T>,
{
    extern "rust-call" fn call_mut(&mut self, args: Args) -> Self::Output {
        self.0.call(args)
    }
}

impl<F, Args, T> FnOnce<Args> for Function<F, T>
where
    F: Func<Args, T>,
{
    type Output = F::Output;
    extern "rust-call" fn call_once(self, args: Args) -> Self::Output {
        self.0.call(args)
    }
}

macro_rules! compose {
    ( $last:expr ) => { $last };
    ( $head:expr, $($tail:expr), +) => {
        ($head, compose!($($tail),+))
    };
}

macro_rules! func {
    ( $head:expr, $($tail:expr), +) => {
        Function(($head, compose!($($tail),+)), PhantomData::default())
    };
}

fn assert_fn<Args>(_: impl Fn<Args>) {}

#[test]
fn test_assert_fn() {
    assert_fn(func!(output, multiply, plus));
}

With function composition I can guarantee that the composition of func!(A, B, C) is the type safe equivalent of the input to A and output of C. The main thing here that makes it easier is having Fn<T> rather than Fn(A). As without the generics argument, I end up having to create an entirely different macro that implements these cases for Fn(A), Fn(A, B), Fn(A, B, C), ...etc.

I should note that the above example doesn't necessarily require the use of impl<Args, T> Fn<Args> for Function<Args, T> it just makes it easier to do:

compose!(plus, plus, plus).call((4,));

vs

func!(plus, plus, plus)(4)

[yasuo-ozu]

@bjorn3 Thanks very much, but I cannot understand the difference of higher-order function arguments and arbitrary function signatures(function overloading) clearly.

Thinking of this example, what is the main factor to distinguish the two concepts?

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=c13f39de7df4c9fbc26b19fd7da4a197

Is this the correct way forcing not to use arbitrary function signatures by language design?

[bjorn3]

Higher-order function arguments is when the function arguments only differ by lifetimes.

[yasuo-ozu]

@bjorn3 If so, the following is different signatures?

fn f<T: AsRef<[u8]>>(t: T) { unimplemented!() }
f("hello");
f("hello".to_owned());
f(vec![1,2,3]);

[parasyte]

"hello".to_owned() and vec![1,2,3] have the same lifetime (they are stack-allocated). &'static T is special because it is valid for every lifetime.

[Lokathor]

if by stack you mean heap, yes

[parasyte]

They both happen to point to the heap, but the structs themselves are stack-allocated when used in the argument position like that. Or when assigned to a variable with a let binding, for instance.

[Lokathor]

Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.

I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:

the desire to not have monomorphization-time errors in generics.

and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.

[nyxtom]

Anyway, the fact that people can already make up their own trait based operator overloading, and even paper over the arity issue with a do_call! macro or whatever, means that there's not much reason to make the Fn traits themselves specifically and magically reject the possibility of overloading. We're just giving people a hard time.

I found tour first reference link as to why not function overloading most interesting because the second reply is from an actual T-lang member that said:

the desire to not have monomorphization-time errors in generics.

and if overloading is happening strictly through the trait system it should end up preventing the post-monomorph errors.

This has been my experience (as I've seen implemented in other places). One macro to implement arity to fix the Fn(A), Fn(A, B)..etc and another to be able to call with do_call!(A, B, C). Having the generics on the actual trait here Fn<Args> makes this less difficult to work with and avoids the extra macro expansion. That being said, it's not strictly required to have an impl Fn<Args> for F since you can do this with the trait based approach but the ability to use the extern "rust-call" hack to turn a struct into a function call is a bit different.