fsharp / fslang-suggestions

The place to make suggestions, discuss and vote on F# language and core library features
345 stars 21 forks source link

Default interface methods #679

Closed cartermp closed 3 years ago

cartermp commented 6 years ago

I propose we add support for consumption and production of methods on interfaces with concrete implementations to enable the following:

The existing way of approaching this problem in F# is to either:

Additionally, it is important to call out that this requires a runtime change for .NET. The change for the runtime is currently being driven and coordinated by the C# and .NET runtime teams.

Description

The syntax for interfaces would be extended to permit the following:

❗️ The following syntax and concepts are a starting point, and absolutely need discussion. ❗️

Concrete member bodies

To begin, there needs to be a way to specify a concrete body for a member in an interface:

type IA =
    default M() = printfn "IA.M()"

A class that implements this interface need not implement its concrete method:

type C() =
    interface IA

let ia: IA = C()
ia.M() // Prints "IA.M()"

The same would apply for object expressions:

type IA =
    default M() = printfn "IA.M()"

type IB =
    inherit IA
    abstract N: unit -> unit

let x =
    { new IB with
          member __.N() = () }

x.M() // Prints "IA.M()"

Note that a class does not inherit members from the interface, so the following code would be illegal:

C().M() // ERROR, 'C' does not contain member 'M'

Member modifiers

The default modifier for interface members is public. That is, the following member is public:

type IA =
    default M() = printfn "IA.M()"

The following access modifiers could be added: public, internal, and private:

type IModifiers =
    abstract internal Doop: unit
    abstract private Hoopty: unit

This is keeping in line with access modifiers being added in C#.

Overriding members

Interfaces could override default members from other interfaces:

type IA
    default M() = printfn "IA.M()"

type IB
    inherit IA
    override IA.M() =  printfn "IB.M()" // explicitly named

type IC
    inherit IA
    override M() = printfn "IA.M()" // implicitly named

Open question: Should explicit naming like this even be a thing?

Overrides are useful to provide more useful "versions" of methods. For example, a new First() method on IEnumerable may have a much more efficient implementation on the interface IList.

Note that no overriding would occur unless specified with the keyword override.

Reabstraction

A default member in an interface could also be overridden to be made abstract again:

type IA =
    default M() = printfn "IA.M()"

type IB =
    inherit IA
    override abstract M: unit -> unit

type C() =
    inherit IB // Error: 'C' does not implement member 'IA.M'

If permitted, the abstract keyword should be required. This is currently also an open issue for C#. We should only implement this if C# does.

Most specific override

I would imagine we do the same as C# here:

We require that every interface and class have a most specific override for every virtual member among the overrides appearing in the type or its direct and indirect interfaces. The most specific override is a unique override that is more specific than every other override. If there is no override, the member itself is considered the most specific override.

For example:

type IA
    default M() = printfn "IA.M()"

type IB
   inherit IA
   override IA.M() = printfn "IB.M()"

type IC
    inherit IA
    override IA.M() = printfn "IC.M()"

interface ID
    inherit IB
    inherit IC  // ERROR: no most specific override for 'IA.M'

type C()
    inherit IB
    inherit IC // ERROR: no most specific override for 'IA.M'

type D() // OK
    inherit IA
    inherit IB
    abstract M: unit -> unit

This rule ensures that ambiguities from diamond inheritance are resolved by the programmer.

static and private members

Because interfaces could have concrete default implementations, common code could be abstracted into static and private members. There are some open issues for C# here today.

This could, in effect, turn interfaces into something in between what we use today and abstract classes. It's a bit iffy to me. I wouldn't want something like let-bound values in them, either.

Base interface invocation

Code in a type that derives from an interface with a default method can explicitly invoke that interface's "base" implementation:

type IA
    default M() = printfn "IA.M()"

type IB
    inherit IA
    override M() = printfn "IB.M()"

type IC
    inherit IA
    override M() = printfn "IC.M()"

type ID
    inherit IB
    inherit IC
    override IA.M() = IB.base.M()

Note that the syntax is Type.base.Member. This is, in keeping with the theme of this suggestion, in line with the C# spec for the feature.

Additionally, the C# team has verified that these changes would not affect existing programs. I would expect anything we do for F# also not affect existing programs.

Pros and Cons

The advantages of making this adjustment to F# are:

The disadvantages of making this adjustment to F# are

Extra information

Estimated cost (XS, S, M, L, XL, XXL): XL

Related information:

Affidavit (please submit!)

Please tick this by placing a cross in the box:

Please tick all that apply:

dsyme commented 6 years ago

I'll come clean and say I do not like this feature. If it is part of .NET interopabliity we may have to consume it, but I have little appetite to surface it directly in the F# language.

I recorded some of my feedback here: https://github.com/dotnet/csharplang/issues/288#issuecomment-288114545, copied below.


I have a bit to say about trait programming in general from the F# perspective - since this feature is really a .NET feature and the F# programmer will be exposed to functionality implemented as traits. The POV may also be useful to C# programmers.

The basic idea of traits is “Traits as a unit of composition” with “required” and “provided” methods. As mentioned in the paper Neal linked, the intention is that “required” methods are parameters, in turn “provided” by other traits or the final class.

"A trait requires a set of methods that serve as parameters for the provided behaviour"

I first want to mention that the technique “encoding parameters as required methods” is anathema In F#. In F#, parameters are parameters, much more so than in C#. I can’t emphasize that strongly enough so I’ll put it in bold again :) In the F# language design we work very, very hard to make sure parameters are just parameters :) :) This has many advantages and some disadvantages.

That is, in F# you almost never encode a parameter as an unfulfilled abstract method within a hierarchy. I’ve think I’ve seen that done like 3 times in my entire F# programming life.

That is, in F# you don’t do this:

    [<AbstractClass>]
    type C() = 
        abstract SomeParameterMethod : int -> int   /// this is not how we do it
        …

Instead in F# you almost always write something like this:

    type C(someParameterFunction: int -> int) =  // this is how we do it….
        …

In F# you instantiate the parameter function when creating the object:

    let c = C(someParameterFunction = someLambda)

A further example is in a comment below.

To be honest, this approach generally turns out to be

Crucially, the focus in F# OO is not at all on “hierarchies of types” – the F# programmer almost never thinks about “the class hierarchy” (which is mentioned often in that linked paper). That’s a very, very good thing and central to the productivity benefits of F#. Java/C# programmers who don't understand this tend to produce bad F# code and/or generally don't grok why F# is asking them to forget/unlearn/disavow/repent what they have spent so long learning. Hierarchy-oriented thinking is “horrible OO” and is the kind of thinking we really want people to avoid in F#. (I think it’s fair to say that C# very much eschews “hierarchy-oriented thinking” relative to Java - and I do fear a little that adding default interface implementations to C# might change that - but that’s up to the C# designers to decide)

Importantly, if a function parameter is encoded as a true function parameter – as in F# - then it need never be revealed, so need not appear in the “surface area” of the type at all, apart from the (internal) constructor. This encourages the F# programmer to make simple things from relatively complex compositions. As described in my book (plug! plug! buy it now!) Expert F# 4.0, making simple things from complex compositions in a key technique that F# encourages.

In the world of traits, from the F# perspective it would be ideal if traits were commonly authored in a way where F# use them to capture some of this spirit. That is a C# trait declared as

interface IMyTrait1 { 
    abstract SomeParameterFunction(int x, int y);
    member SomeProvidedMethod() =>  SomeParameterFunction(3) + SomeParameterFunction(4)
}

declared in C# might be usable in F# as if it had been declared like this:

type IMyTrait1(someParameterFunction) = 
    override SomeProvidedMethod() =  someParameterFunction(3) + someParameterFunction(4)
    …

And used like this:

type CompositionClass() = 
    inherit MyTrait1(fun x -> x + 1)

In the example, MyTrait1 would still at runtime presumably be encoded as an interface with one implementation method and one required abstract-internal method (possibly with a compiler-generated name if declared in F#, derived from the parameter name), fulfilled by the given parameter lambda in the class. So I'm not saying the .NET/CLR mechanism has to be different.

The downside to the F# approach is that “wiring” between mutually-referential traits must be done explicitly. But mutually referential traits are, I think, relatively rare, and frankly deserve to be wired explicitly.

I guess that highlights the other thing I dislike about mixin-style traits: the names of the “parameter methods” are part of the APIs of the types. Sure, they can be given a unique name, and made assembly internal, but that’s a lot of extra keywords and a basically very odd and contorted way to declare what is really just a parameter.

Anyway, from the F# perspective it would be awesome if traits were declared so that parameter “required” methods were easily distinguished from published “provided” methods, and the F# language could reveal traits as units composed using parameter instantiation.

Mantras:

To follow up, a C# example like this becomes something like this in F# thinking (I've taken some liberties with this - this is using the class inheritance features available in F# - but since the example is single inheritance this is enough to indicate the points made above

type Semigroup<'A>(append: 'A -> 'A -> 'A) =  // note 'append' is a function parameter
    member __.Append(x, y) = append x y 

type Monoid<'A>(empty, append) = 
    inherit Semigroup<'A>(append) // this construct is class inheritance in F#, there are no traits yet, nor trait inheritance
    member __.Empty = empty
    member __.Concat(xs) = Seq.fold append empty xs

type MString() = 
    inherit Monoid<string>("", (+)) // again this is class inheritance

type MArray<'A>() = 
    inherit Monoid<'A[]>(Array.empty, Array.append)

type MEnumerable<'A>() = 
    inherit Monoid<seq<'A>>(Seq.empty, Seq.append)

Note the systematic distinction between parameters (required operations) and published methods (provided operations). In this case, each parameter also happens to be published, but that is most definitely not always the case in practice - this symmetry typically only applies for simple examples.

Note also that the use of parameters means that many lines disappear in F# when connecting exiting functionality (Array.empty, Array.append) into the implementation.

dsyme commented 6 years ago

The listed benefits are these:

  1. Help API authors version interfaces
  2. Aid interoperation with Java and iOS for Xamarin programming scenarios
  3. Act as a basis for trait-like programming, potentially using this as a vehicle for further abstractions

Of these, only (2) is of interest for me, i.e. this feature may be necessary for interoperating with .NET code. This requires consumption only.

I simply don't believe (1) and (3) are real benefits in the overall scheme of things vis.a.vis the massive complexifying costs of software designs based around implementation inheritance, especially in a language like F# that seeks to re-orient programmers minds away from this horror. An over-focus on implementation inheritance has cost the software industry billions and billions of dollars in lost productivity and frankly is has created a lost generation of programmers who obsessively make complex things out of complex things. .NET made the right decision in version 1 in not allowing interfaces to have implementations, and to be honest I think it should stick with it, placing a feature like this at the outer fringes of the .NET universe like COM interop, dynamic and that whacky stuff the .NET team did with interfaces for Office interop in C# 5 which everyone has forgotten.

Additionally, the feature is massively controversial in the C# world, with https://github.com/dotnet/csharplang/issues/288 having the highest proportion of downvotes I've seen for a C# language design proposal that's being taken forward.

So, in short, this feature would be interop-only, placing it in the same class as "protected" (which again encourages implementation inheritance).

cartermp commented 6 years ago

@dsyme Just to say (not sure if this is clear), I wanted to capture all possibilities with this feature in the issue. My primary concern is interoperability, but I wanted to "open the floor" to what others thing about this as more than just as an interop feature.

Horusiath commented 6 years ago

This also aligns with the conclusions made in other languages - namely Scala - which have traits from day 1, then have seen that parametric traits are also important (see: SIP 25).

dsyme commented 6 years ago

@cartermp yes totally understood, we have to track the feature for interop purposes, and I know you're not championing it in full as such :) Thanks for doing it and adding the detail :)

dsyme commented 6 years ago

@Horusiath thanks for that info. The C# folk should really be looking very closely at the overall Scala experience with this feature.

tpetricek commented 6 years ago

Just to derail the discussion a bit, I would really love to be able to implement interfaces by saying interface ISomething = <expr> where <expr> is an expression returning the interface implementation.

I think that might actually cover some of the use cases of the C# proposal, because you could do:

type IA = 
  abstract M : unit -> unit
  abstract N : unit -> unit

let makeIA f = 
  { new IA with 
      member x.M() = f ()
      member x.N() = printfn "default" }

type C() = 
  interface IA = makeIA (fun () -> printfn "C")

And you could do this without introducing any more complex object-oriented concepts...

dsyme commented 6 years ago

@tpetricek I really like that suggestion - there is a separate suggestion for it somewhere. TBH it is vastly more "in tune" with F# methodology (delegation over inheritance) than the default interfaces feature.

cartermp commented 6 years ago

Agreed, that does feel much more natural than the override/default goop that would come with a direct port of the C# feature set.

wallymathieu commented 6 years ago

If you have the ability to specify static for interface members (in c# and f#), you could perhaps add module interfaces.

zpodlovics commented 6 years ago

Probably also worth exploring and discussing this (here or as a new issue) C# lang issue to have a proper concepts and mappings in F# (it also explores interface based generic numeric code as an example usage similar to @tpetricek http://tomasp.net/blog/fsharp-generic-numeric.aspx/):

"Roles, extension interfaces and static interface members

This is an attempt to address the scenarios targeted by my previous "shapes" investigation (which was in turn inspired by the "Concept C#" work by Claudio Russo and Matt Windsor), but in a way that leverages interfaces rather than a new abstraction mechanism. It is not necessary to read the previous proposals in order to understand this one."

https://github.com/dotnet/csharplang/issues/1711

cartermp commented 6 years ago

Just to call this out, default interface methods do give us a .NET runtime-supported way to support #243 (to some degree). However, this change would only be available in .NET Core, since it is highly unlikely that the desktop CLR would be modified to support the underlying runtime feature. So just like C#, F# would have a feature that only works if you're using CoreCLR.

robkuz commented 6 years ago

@cartermp did I read this correctly: Default Interfaces wont be available on the Desktop? Why is this?

StevenRasmussen commented 6 years ago

@robkuz - I guess one thing to consider is that desktop support is coming to DotNet core 3.0, and so perhaps it will be less of an issue since you could just target DotNet core at that point: https://blogs.msdn.microsoft.com/dotnet/2018/05/07/net-core-3-and-support-for-windows-desktop-applications/

jnm2 commented 6 years ago

You'd never be able to use Default Interface Methods in a library that targets .NET Standard, though, because it would blow up at runtime on .NET Framework and UWP. In order to use DIM, a library would be forced to single-target .NET Core. (As a library author, that's a hard sell.)

cartermp commented 6 years ago

@robkuz Default interface methods require a runtime change. This means that there is also a check for seeing if the feature is supported by a given runtime: https://github.com/dotnet/csharplang/blob/master/proposals/default-interface-methods.md#clr-support-api

No shipped .NET Framework version supports this today, and it's highly unlikely that they ever will due to the risk of breaking existing apps that are so widespread. .NET Core will eventually have this in its runtime, but it's not completely resolved if it will also be in some future .NET Framework, mono, or UWP runtime. And as @jnm2 mentioned, unless every runtime that supports a .NET Standard also has this feature, then you wouldn't be able to use them in .NET Standard. It is not in the upcoming .NET Standard 2.1 plan either.

The question in my mind, from a long-term planning perspective, is what we do beyond simply ensuring that we don't blow up in the face of such a construct. Is the feature a copy of C#? Probably not. A fully-fledged traits/typeclasses system? That would need a proper design that would take time. How would it be rationalized with existing things like SRTP? How to think about interfaces today vs. interfaces tomorrow vs. functions as interfaces vs. normal generics vs. SRTP vs. {insert thing here}? But at least in my mind, the mechanism for implementing something is coming, so it would be good to think about what that something is at a high level, what sort of behaviors it could have, and how to rationalize it with existing features in this space.

zpodlovics commented 6 years ago

Before this feature improvement / high level design planning starts, I would like to suggest to add the indirect calls supports to this list. Probably also worth to create an independent proposal/design document to the indirect calls for the details.

"We use ldftn + calli in lieu of delegates (which incur an object allocation) in performance-critical pieces of our code where there is a need to call a managed method indirectly. This change allowed method bodies with a calli instruction to be eligible for inlining. Our dependency injection framework generates such methods." [1] [2] [3]

"This proposal provides language constructs that expose low level IL opcodes that cannot currently be accessed efficiently, or at all: ldftn, ldvirtftn, ldtoken and calli. These low level op codes can be important in high performance code and developers need an efficient way to access them." [2] [3]

The high level design is fine as long as if it's done "without regret" (indirection / abstraction overhead).

"It has been said that all problems in computer science can be solved by adding another level of indirection, except for performance problems, which are solved by removing levels of indirection."

Compilers should be our tools for removing levels of indirection automatically.

[1] https://blogs.msdn.microsoft.com/dotnet/2018/08/20/bing-com-runs-on-net-core-2-1/ [2] dotnet/coreclr#13756 [3] https://github.com/dotnet/csharplang/blob/master/proposals/intrinsics.md [4] https://github.com/dotnet/csharplang/blob/3b43266cea6012b8964cb6ebc0e913ae7d6abd12/meetings/2017/LDM-2017-01-11.md [5] https://github.com/dotnet/csharplang/blob/master/meetings/2018/LDM-2018-09-05.md [6] https://github.com/dotnet/csharplang/issues/191

HaloFour commented 6 years ago

@zpodlovics

191

zpodlovics commented 6 years ago

@HaloFour You probably mean this: https://github.com/dotnet/csharplang/issues/191

HaloFour commented 6 years ago

@zpodlovics

Oh yes, sorry, I mistakenly thought your comment was posted to that repo.

In that case I might suggest you open a new issue specifically to look into that same support for F#.

MichalStrehovsky commented 5 years ago

You'd never be able to use Default Interface Methods in a library that targets .NET Standard, though, because it would blow up at runtime on .NET Framework and UWP. In order to use DIM, a library would be forced to single-target .NET Core. (As a library author, that's a hard sell.)

Default interface methods are now being added to the .NET Standard: dotnet/standard#1019 :/

abelbraaksma commented 5 years ago

The original post here says that reabstraction is an open question and should only be implemented if C# does.

@cartermp, it has meanwhile been answered and the C# team considers it a requirement to allow reabstraction. See the same link, section on Resolved Questions. Perhaps we can update the OP?

cartermp commented 5 years ago

@abelbraaksma We'll have to go over this one in more detail if it's to be a thing in .NET Standard 2.1. However, reabstraction isn't essential to interoperation. So far, my stance is that we need to be able to consume DIMs without blowing up. Implementing the same set of features is different, IMO.

robkuz commented 5 years ago

@cartermp I would like to understand how the stance on this feature synchronizes with seamless 2-way integration which is so often espoused as one of the highest goods/goals in F#. What are the (objective) reasons for not implementing this feature on the producing side? I must admit I find this troubling.

cartermp commented 5 years ago

What do you fine troubling?

robkuz commented 5 years ago

@cartermp many interesting language features have been marked as "awaiting C#" or "CLR adjustment needed" and therefore been rejected for now (?).

So all hope for any significant language improvements are being pushed to C#/CLR. However if it is possible to knockout the "seamless-2-way-integration" mantra and deny the (producing) implementation of Default Interfaces because of subjective preferences then the same could easily be done for GADTs, HKTs and TCs (or anything that might come in the future).

I find this troubling and the message it sends is inconsistent to say the least.

cartermp commented 5 years ago

So all hope for any significant language improvements are being pushed to C#/CLR.

This is not true. It may be true for those you care about, but certainly not for others.

robkuz commented 5 years ago

Actually those 3 I specifically named are among the top 6 most upvoted and also oldest. So it seems that this isn't exactly only me.

But then again your reply isn't answering my question. So again:

What are the objective criteria to ditch the 2-way-seamless integration of a C# feature?

This concerns this specific feature but also potentially other (future) features.

cartermp commented 5 years ago

There is little objective criteria in language design. That's why it's design, not engineering. Nonetheless, I will give my criteria, which likely differs a bit from @dsyme's:

Lastly, the feature hasn't even shipped in a preview of C# yet. Given the controversy around it, I'd much rather wait until there is consensus that it's okay before we think of proceeding with anything other than not blowing up when we see it in metadata.

Lanayx commented 4 years ago

Wanted to leave comment here, I write the F# library which is used by both F# and C# developers. And I have to give users ability to provide their classes for routing, authentication, interceptors. Without support of DIM I'll be making breaking changes with new releases, so if I want to avoid it I need to extract this logic to C# code which is undesirable.

cartermp commented 4 years ago

Thanks for the feedback @Lanayx. I've come around to thinking that we should also just support producing DIMs, even though it's an inheritance-oriented feature. It feel useful enough (versioning interfaces) and interop with Java/iOS is valuable.

cartermp commented 3 years ago

Closing out as completed for F# 5. We may revisit creating these. The work to create them isn't terrible large, but we'd need to figure out a good design.

realvictorprm commented 3 years ago

I'm meanwhile in favour of supporting the creation of them in F# too.

abelbraaksma commented 3 years ago

Just summing up for visitors late to the party: currently consuming default interfaces is possible, creating them is not. Considering the above comments, perhaps we need a new language suggestion for DIM creation from F#.

PR (in F# 5.0): https://github.com/dotnet/fsharp/pull/8628 RFC FS-1074: https://github.com/fsharp/fslang-design/blob/master/FSharp-5.0/FS-1074-default-interface-member-consumption.md

Serentty commented 3 years ago

Just summing up for visitors late to the party: currently consuming default interfaces is possible, creating them is not. Considering the above comments, perhaps we need a new language suggestion for DIM creation from F#.

PR (in F# 5.0): dotnet/fsharp#8628 RFC FS-1074: https://github.com/fsharp/fslang-design/blob/master/FSharp-5.0/FS-1074-default-interface-member-consumption.md

I'd be happy to see such a proposal. I've come to really like this feature in Rust. Passing in lambdas really doesn't seem like the same thing at all, because it means allowing variation at the instance level, not just the type level. Plus it means carrying all that extra data around in the instance.