MadsTorgersen commented 3 years ago

Static abstract members in interfaces

[x] Proposed
[ ] Prototype: Not Started
[ ] Implementation: Not Started
[ ] Specification: Not Started

Speclet: https://github.com/dotnet/csharplang/blob/main/proposals/csharp-11.0/static-abstracts-in-interfaces.md

Summary

An interface is allowed to specify abstract static members that implementing classes and structs are then required to provide an explicit or implicit implementation of. The members can be accessed off of type parameters that are constrained by the interface.

Motivation

There is currently no way to abstract over static members and write generalized code that applies across types that define those static members. This is particularly problematic for member kinds that only exist in a static form, notably operators.

This feature allows generic algorithms over numeric types, represented by interface constraints that specify the presence of given operators. The algorithms can therefore be expressed in terms of such operators:

// Interface specifies static properties and operators
interface IAddable<T> where T : IAddable<T>
{
    static abstract T Zero { get; }
    static abstract T operator +(T t1, T t2);
}

// Classes and structs (including built-ins) can implement interface
struct Int32 : …, IAddable<Int32>
{
    static Int32 I.operator +(Int32 x, Int32 y) => x + y; // Explicit
    public static int Zero => 0;                          // Implicit
}

// Generic algorithms can use static members on T
public static T AddAll<T>(T[] ts) where T : IAddable<T>
{
    T result = T.Zero;                   // Call static operator
    foreach (T t in ts) { result += t; } // Use `+`
    return result;
}

// Generic method can be applied to built-in and user-defined types
int sixtyThree = AddAll(new [] { 1, 2, 4, 8, 16, 32 });

Syntax

Interface members

The feature would allow static interface members to be declared virtual.

Today's rules

Today, instance members in interfaces are implicitly abstract (or virtual if they have a default implementation), but can optionally have an abstract (or virtual) modifier. Non-virtual instance members must be explicitly marked as sealed.

Static interface members today are implicitly non-virtual, and do not allow abstract, virtual or sealed modifiers.

Proposal

Abstract virtual members

Static interface members other than fields are allowed to also have the abstract modifier. Abstract static members are not allowed to have a body (or in the case of properties, the accessors are not allowed to have a body).

interface I<T> where T : I<T>
{
    static abstract void M();
    static abstract T P { get; set; }
    static abstract event Action E;
    static abstract T operator +(T l, T r);
}

Open question: Operators == and != as well as the implicit and explicit conversion operators are disallowed in interfaces today. Should they be allowed?

Explicitly non-virtual static members

Todau's non-virtual static methods are allowed to optionally have the sealed modifier for symmetry with non-virtual instance members.

interface I0
{
    static sealed void M() => Console.WriteLine("Default behavior");

    static sealed int f = 0;

    static sealed int P1 { get; set; }
    static sealed int P2 { get => f; set => f = value; }

    static sealed event Action E1;
    static sealed event Action E2 { add => E1 += value; remove => E1 -= value; }

    static sealed I0 operator +(I0 l, I0 r) => l;
}

Implementation of interface members

Today's rules

Classes and structs can implement abstract instance members of interfaces either implicitly or explicitly. An implicitly implemented interface member is a normal (virtual or non-virtual) member declaration of the class or struct that just "happens" to also implement the interface member. The member can even be inherited from a base class and thus not even be present in the class declaration.

An explicitly implemented interface member uses a qualified name to identify the interface member in question. The implementation is not directly accessible as a member on the class or struct, but only through the interface.

Proposal

No new syntax is needed in classes and structs to facilitate implicit implementation of static abstract interface members. Existing static member declarations serve that purpose.

Explicit implementations of static abstract interface members use a qualified name along with the static modifier.

class C : I<C>
{
    static void I.M() => Console.WriteLine("Implementation");
    static C I.P { get; set; }
    static event Action I.E;
    static C I.operator +(C l, C r) => r;
}

Open question: Should the qualifying I. go before the operator keyword or the operator symbol + itself? I've chosen the former here. The latter may clash if we choose to allow conversion operators.

Semantics

Operator restrictions

Today all unary and binary operator declarations have some requirement involving at least one of their operands to be of type T or T?, where T is the instance type of the enclosing type.

These requirements need to be relaxed so that a restricted operand is allowed to be of a type parameter that is constrained to T.

Open question: Should we relax this further so that the restricted operand can be of any type that derives from, or has one of some set of implicit conversions to T?

Implementing static abstract members

The rules for when a static member declaration in a class or struct is considered to implement a static abstract interface member, and for what requirements apply when it does, are the same as for instance members.

TBD: There may be additional or different rules necessary here that we haven't yet thought of.

Interface constraints with static abstract members

Today, when an interface I is used as a generic constraint, any type T with an implicit reference or boxing conversion to I is considered to satisfy that constraint.

When I has static abstract members this needs to be further restricted so that T cannot itself be an interface.

For instance:

// I and C as above
void M<T>() where T : I<T> { ... }
M<C>();  // Allowed: C is not an interface
M<I<C>>(); // Disallowed: I is an interface

Accessing static abstract interface members

A static abstract interface member M may be accessed on a type parameter T using the expression T.M when T is constrained by an interface I and M is an accessible static abstract member of I.

T M<T>() where T : I<T>
{
    T.M();
    T t = T.P;
    T.E += () => { };
    return t1 + T.P;
}

At runtime, the actual member implementation used is the one that exists on the actual type provided as a type argument.

C c = M<C>(); // The static members of C get called

Drawbacks

"static abstract" is a new concept and will meaningfully add to the conceptual load of C#.
It's not a cheap feature to build. We should make sure it's worth it.

Alternatives

Structural constraints

An alternative approach would be to have "structural constraints" directly and explicitly requiring the presence of specific operators on a type parameter. The drawbacks of that are:

This would have to be written out every time. Having a named constraint seems better.
This is a whole new kind of constraint, whereas the proposed feature utilizes the existing concept of interface constraints.
It would only work for operators, not (easily) other kinds of static members.

Default implementations

An additional feature to this proposal is to allow static virtual members in interfaces to have default implementations, just as instance virtual members do. We're investigating this, but the semantics get very complicated: default implementations will want to call other static virtual members, but what syntax, semantics and implementation strategies should we use to ensure that those calls can in turn be virtual?

This seems like a further improvement that can be done independently later, if the need and the solutions arise.

Virtual static members in classes

Another additional feature would be to allow static members to be abstract and virtual in classes as well. This runs into similar complicating factors as the default implementations, and again seems like it can be saved for later, if and when the need and the design insights occur.

Unresolved questions

Called out above, but here's a list:

Operators == and != as well as the implicit and explicit conversion operators are disallowed in interfaces today. Should they be allowed?
Should the qualifying I. in an explicit operator implenentation go before the operator keyword or the operator symbol (e.g. +) itself?
Should we relax the operator restrictions further so that the restricted operand can be of any type that derives from, or has one of some set of implicit conversions to the enclosing type?

Design meetings

tannergooding commented 3 years ago

As for the interoperability part, while I appreciate the C# improvements of late, they seem to come to the expense of turning CLR into the C# Language Runtime, with VB, F#, C++/CLI getting 2nd or 3rd place treatment, if at all, as some newer frameworks become unaccessible to them other then we having to write wrapper libraries in C#.

@pjmlp At the end of the day, C# is by far the most used language in .NET and so its natural that it also drives many of the bigger .NET features and that other languages end up adding support for consuming those features.

There are likewise some features, both new and existing, from other languages (such as F#) that get improvements or that already exist and they are often considered when C# implements new features, when we expose new .NET APIs, and when new runtime features get created.

The same is true in other languages and frameworks. One example is LLVM where it is primarily driven by C++, but where some other languages (such as Rust) help influence it.

masonwheeler commented 3 years ago

As for the interoperability part, while I appreciate the C# improvements of late, they seem to come to the expense of turning CLR into the C# Language Runtime, with VB, F#, C++/CLI getting 2nd or 3rd place treatment, if at all, as some newer frameworks become unaccessible to them other then we having to write wrapper libraries in C#.

Yes, this is what I've been saying for years now! Anytime the team implements some new feature that will be visible to external libraries as a franken-hack in the C# compiler, rather than putting it in the runtime, it makes things exponentially harder for all the rest of the CLR languages to keep up.

HaloFour commented 3 years ago

@masonwheeler

Yes, this is what I've been saying for years now! Anytime the team implements some new feature that will be visible to external libraries as a franken-hack in the C# compiler, rather than putting it in the runtime, it makes things exponentially harder for all the rest of the CLR languages to keep up.

Whereas the complaint here is that the feature is being implemented in the runtime, and that the other languages are going to be forced to be changed in order to interop with it.

CyrusNajmabadi commented 3 years ago

Anytime the team implements some new feature that will be visible to external libraries as a franken-hack in the C# compiler,

How is this a hack in the compiler? The language feature is just directly exposing and using the exact corresponding runtime feature. What would your have preferred we do here?

hez2010 commented 3 years ago

As for the interoperability part, while I appreciate the C# improvements of late, they seem to come to the expense of turning CLR into the C# Language Runtime, with VB, F#, C++/CLI getting 2nd or 3rd place treatment, if at all, as some newer frameworks become unaccessible to them other then we having to write wrapper libraries in C#.

Yes, this is what I've been saying for years now! Anytime the team implements some new feature that will be visible to external libraries as a franken-hack in the C# compiler, rather than putting it in the runtime, it makes things exponentially harder for all the rest of the CLR languages to keep up.

To clarify, this is totally a runtime feature, not a compiler feature. They introduced changes on CoreCLR directly to support this, and it's not a compiler hack at all. To verify this, even if you successfully compile statics in interfaces, you cannot run it against older version of .NET and get TypeLoadFailure from runtime.

Other languages on CoreCLR which cannot consume this feature for now because their compilers don't recognize the signature change on types from IL level, and they need update before they can consume abstract statics in interfaces.

pjmlp commented 3 years ago

@tannergooding

All fine and good, except that wasn't as the Common Language Runtime was announced to the world back in 2002.

Do you need references from Microsoft's own marketing materials?

In any case that was just a piece of feedback, which won't change anything on that current course, so I am stopping here, as it doesn't belong in this issue discussion.

asv7c2 commented 3 years ago

@xoofx

But static function without body in interface is compiler error. Why then abstract word needs?

https://sharplab.io/#v2:C4LgTgrgdgNAJiA1AHwAICYAMBYAUBgRjz1QIE4AKAIkB4NwEH2qBKAbmNwEspgBTMAMwCGAY24ACAJKiA3nlFzRpAGwKALKIBiAe00UWeAL5A==

333fred commented 3 years ago

But static function without body in interface is compiler error. Why then abstract word needs?

How would we later enable you to provide a virtual static (ie, with a DIM and overridable)? We'd need to require a virtual keyword, and we feel the inconsistency among static members would be far worse than the inconsistency between instance and static.

byme8 commented 3 years ago

static means that the member is available at the type level rather than at the instance level.

abstract means that the defining type does not provide the implementation, a deriving type provides it instead.

virtual means that the defining type does provide an implementation, but a deriving type may also override that implementation with its own.

The first one is fine.

The second one is fine if we are talking about the abstract class because the class is an implementation. If we are talking about the interface then it looks confusing for me. I think so because the interface starts to look like a contract and an implementation at the same time. That's weird.

The third one has the same issue. The interface starts to look like an implementation.

I think the shapes feature looks significantly better in this case. It provides all required behaviours and doesn't break existing one.

HaloFour commented 3 years ago

@byme8

The second one is fine if we are talking about the abstract class because the class is an implementation. If we are talking about the interface then it looks confusing for me. I think so because the interface starts to look like a contract and an implementation at the same time. That's weird.

We've crossed that bridge with default implementations. The only difference between "interfaces", "traits" or "shapes" is what we happen to call them.

acaly commented 3 years ago

The only difference between "interfaces", "traits" or "shapes" is what we happen to call them.

In my previous understanding, shapes should be contracts of types. That means two thing:

While you have contracts of types, it does not allow contracts of contracts (and contracts of contracts of contracts). However if interfaces are both types and contracts, will there be contracts of interfaces? What should they be? How do we distinguish contract of an interface itself (static abstract does not meet the requirement) and contract of types that implement the interface (static abstract does meet the requirement as all implementing type must provide an implementation)? If in the future we can have static virtual in interfaces, how can we express the contract/constraint that a type has a static method that is not abstract?
It allows C++-like duck typing. We can claim that a specific external type meets the requirement of an interface without having to explicitly declare that type to implement the interface. Based on how numeric interfaces are implemented in runtime now, it seems that this kind of duck typing won't be allows in C#. There are discussions whether duck typing is good or not, but it's definitely useful in some cases.

tannergooding commented 3 years ago

All of this ends up being words/semantics and differences in how people interpret it. At the end of the day everything has to fit into the constraints of the underlying runtime, including how it requires concepts to be represented. This further has constraints against the underlying "Application Binary Interface" of the runtime, architecture, or platform you are running against.

Even in C/C++, Rust , or other languages where structural typing (or similar features) exist, there is still a contract for how a given T is made to look and act like a U, this is often by a zero or low cost abstraction secretly emitted by the compiler.

In .NET, interfaces are the thing we have that can be implemented by both reference and value types. Value types are then the thing that do not have to allocate (there is no requirement to be GC tracked). Generics are the thing that allow you to delay providing a "concrete" type until runtime.

In .NET (at least for RyuJIT), value types get specialized by the JIT and so two different types get distinct method bodies. This allows the JIT to optimize for a given type and combined with value types being sealed, allows the runtime to devirtualize and produce often "ideal" (or near ideal) codegen when compared to manually exposing a method supporting the concrete type directly.

Functionally speaking, that means static abstract in interfaces are the "best way" in the .NET type system to expose these concepts and to form the low level building blocks on which any future thing, such as structural typing, shapes, traits, etc could be built. This is because they represent the closest to a zero or low cost abstraction and what the underlying type system will actually require to use and expose such features in metadata.

HaloFour commented 3 years ago

@acaly

Based on how numeric interfaces are implemented in runtime now, it seems that this kind of duck typing won't be allows in C#. There are discussions whether duck typing is good or not, but it's definitely useful in some cases.

The interface defines the contract. Roles define how an existing type can satisfy the contract without the existing type having to explicitly do so, supported by expanding extension members and supporting extension implementation. Whether or not the compiler will emit a role "automatically" when a type can be duck-typed into the interface is still up for debate, and all of that design may change.

"Shapes" more or less defines the entire project. One of the early proposals was to add a new construct which would live alongside interfaces and supported the additional functionality necessary for generic arithmetic and the like. The argument against adding a new construct was that it would split the ecosystem, and that interfaces were already 90% of the way there. So that's why the teams are exploring adding virtual static members to the runtime and language, to position them as the contracts suitable for the scenarios to be addressed by shapes. It enables shapes to be both less messy and more efficient as the runtime will provide the generic specialization. It positions the languages to be able to expand further into other structural typing design such as shapes or extension implementation.

masonwheeler commented 3 years ago

All of this ends up being words/semantics and differences in how people interpret it. At the end of the day everything has to fit into the constraints of the underlying runtime, including how it requires concepts to be represented. This further has constraints against the underlying "Application Binary Interface" of the runtime, architecture, or platform you are running against.

...

In .NET, interfaces are the thing we have that can be implemented by both reference and value types.

No, no, please no. That is a completely wrong way to look at Shapes, and will end up with one of the worst possible implementations. If it's going to be done, it needs to not be kludged into working with the existing runtime with a bunch of Roslyn compiler magic turning it into yet another frankenhack. Shapes need to be a fundamentally new "type of type" in the CLR's type system, for reasons that have already been discussed at length in the various threads about shapes.

CyrusNajmabadi commented 3 years ago

and will end up with one of the worst possible implementations.

The work with static abstracts show this to not be the case. It slots in incredibly well and provides a ton of the value and need here with just a tiny augmentation of what we've had since the beginning. You'll need a lot more information provided to show why we'd have to go an entirely different direction from top to bottom as we expand this out further in the future.

quixoticaxis commented 3 years ago

Sorry for breaking in, but following the discussions in this repo and in runtime repo I've been constantly bugged by the question: is there a list of explicit goals and explicit non-goals of C# language as a project? I could not find one, but I suppose having one would be a great help, because people tend to have and express completely different assumptions about what should be achieved by the language, the runtime, and the libraries.

For example, the current proposal lacks power for coding the most of the mathematics I deal with at work in an easy-to-reason-about way (I miss the currently unimplemented "type classes", "roles", "shapes", or anything else that would allow me to define different ways to multiply the same type ad-hoc, for example, cross product and scalar product of vectors. The currently defined interfaces lack precision that is needed to make my programs more verifiable: no partial ordering interface, no not zero integer types, no interfaces for different kinds of integer machine arithmetic, no ability to define multiple identity entities for a single type, and the new interfaces are, well, limiting, for example, the complex numbers cannot be INumber. Everything listed makes the proposal almost useless for my tasks. ) I wouldn't call it "wrong", probably the feature is simply not targeted at the class of problems I have to solve. But having explicit goals and non-goals, in my humble opinion, would definitely help in straitening the discussions.

CyrusNajmabadi commented 3 years ago

there a list of explicit goals and explicit non-goals of C# language as a project?

I don't believe such a thing exists. And even if it did, it would never be discussing enough to definitely answer if those use cases you mentioned sound be addressed by this proposal.

333fred commented 3 years ago

Your feedback on the interface designs should probably be directed to the runtime repo so @tannergooding sees it.

quixoticaxis commented 3 years ago

@CyrusNajmabadi

I don't believe such a thing exists. And even if it did, it would never be discussing enough to definitely answer if those use cases you mentioned sound be addressed by this proposal.

Sure, and it should not, as I believe, but it would clearly help by providing people with the description of what the general direction is.

I'm fairly certain that prioritizing multilevel metaprogramming technics is a non-goal of C#. It's cool you can do a lot with generics with some help from reflection API, but that's it.
I suppose expressiveness and type-safety come second to ease-of-use and easiness-to-jump-in for the language.
I assume creating a stricter type system to achieve better compile-time verifiability is also a non-goal for the language.
I tend to think that being modular and interchangeable is a non-goal for the runtime and libraries (most of the libraries can be customized as a solution or a tool, but most facilities I know cannot be reliably re-purposed to become an independent building block outside of the current solutions developed by Microsoft and/or cannot be easily disconnected from the unified (and sometimes limited) interface framework).

P.s. Sorry for the off-topic, maybe I'll raise the question of the list's existence as a separate issue/discussion later.

tannergooding commented 3 years ago

Shapes need to be a fundamentally new "type of type" in the CLR's type system, for reasons that have already been discussed at length in the various threads about shapes.

Even if the runtime ended up with some new "type", I would very much expect that the underlying implementation and ABI of such a type would be similar to the interface approach.

You have to fit into the platform application binary interface at some point and at the ABI level you need some metadata to represent the shape/trait/contract (e.g. interfaces) and something tying an implementation of a shape/trait/contract to a type so that a consumer can successfully invoke it. The latter ends up fundamentally being something like directly implementing the interface (either the direct type or some helper type that shims the contract) which the compiler can then elide at emit time to be zero or low cost.

No matter which way you look at it, we have to represent these "shim" types somewhere in order to interact and be compatible with the platform calling convention and there will not be a huge difference between some new "shape type" and an interface, as functionally they are representing the same things (a type and a list of members available to that type which a deriving type implements).

acaly commented 3 years ago

Even if the runtime ended up with some new "type", I would very much expect that the underlying implementation and ABI of such a type would be similar to the interface approach.

I agree that using the current interface to represent such information is probably the easiest way to go for the runtime, but that doesn't change the fact that type of type should be different from type. This can be achieved by representing by a metadata (e.g. attribute or modifier) and forbid invalid use from the language side. But the current method to implement arithmetic interfaces is not following this path. Those interfaces are exactly the same as normal interfaces. No additional attributes. Unless they are changed later, we are very unlikely to have another concept in runtime or language for specifically "type of type" besides normal interfaces, which are just types. This brings some questions:

Numberic types must declare to implement those interfaces in order to be considered to belong to that type. In this reply (https://github.com/dotnet/csharplang/issues/4436#issuecomment-898935675) it seems this has not been decided yet.
Interfaces itself can have static non-virtual members today, which must not be there if the interface will be used as "type of type". There has not been a mechanism to forbid this yet.
Today, for an interface I to match T in where T : I, I must contain no static abstract members. This rule makes no sense for interfaces and should only be applied to "type of type" interfaces. Again, there is no mechanism to do this today.

CyrusNajmabadi commented 3 years ago

but that doesn't change the fact that type of type should be different from type.

Why is this a fact? I'm not really understanding why these need to be different.

acaly commented 3 years ago

I'm not really understanding why these need to be different.

Second-order logic is different from first-order logic.

You can continue reading the rest of my comment. I wrote 3 points. At least point 2 and 3 are outcomes of conflating the two.

tannergooding commented 3 years ago

Numberic types must declare to implement those interfaces in order to be considered to belong to that type. In this reply ([Proposal]: Static abstract members in interfaces #4436 (comment)) it seems this has not been decided yet.

A type must implement an interface to be that interface, if it doesn't it can't be used in a method constrained to that interface. We don't have structural typing in IL and so even if C# had structural typing, there would be some shim type in metadata that does implement the interface/shape/constraint for everything to work.

Interfaces itself can have static non-virtual members today, which must not be there if the interface will be used as "type of type". There has not been a mechanism to forbid this yet.

This is a needless limitation. If someone declares an interface and wants to provide a non-abstract/virtual helper method for all INumber, there is no reason that should be prevented. It only means a deriving type can't provide additional or overriding functionality for such method. Its not "ideal" for every scenario but neither is forcing everything to be "virtual".

Today, for an interface I to match T in where T : I, I must contain no static abstract members. This rule makes no sense for interfaces and should only be applied to "type of type" interfaces. Again, there is no mechanism to do this today.

I'm not sure what you're saying here. The compiler allows where T : INumber<T> and it works for some T that implements INumber<T>. The compiler will prevent the use of INumber<T> in a similar fashion to abstract classes in general, you must have some concrete T (such as via a constrained generic) for everything to work and INumber<T> itself (if you managed to get one via boxing) isn't valid since it doesn't itself implement INumber<T> (being the type and implementing the type are "different").

masonwheeler commented 3 years ago

You have to fit into the platform application binary interface at some point and at the ABI level you need some metadata to represent the shape/trait/contract (e.g. interfaces)

Sure, at the metadata level it would probably look a whole lot like classes or interfaces, and could probably even reuse some of the same tables. I've never disagreed with that part.

and something tying an implementation of a shape/trait/contract to a type so that a consumer can successfully invoke it.

When you say "an implementation of a shape" it makes it sound like you're still thinking of it as an interface rather than a shape. Let's break that way of thinking with one simple rule: if it is possible to have these as the declared type of a field, property, or method parameter, it is not a shape. It's a frankenhack that should not exist.

The latter ends up fundamentally being something like directly implementing the interface (either the direct type or some helper type that shims the contract) which the compiler can then elide at emit time to be zero or low cost.

Ugh, now you're neck-deep in frankenhacks, creating multiple layers of new types under the hood. 🤢 This is specifically the idea I was referring to when I said "worst possible implementation" above. It will be a mess for anything outside of C# to interoperate with!

No matter which way you look at it, we have to represent these "shim" types somewhere in order to interact and be compatible with the platform calling convention and there will not be a huge difference between some new "shape type" and an interface, as functionally they are representing the same things (a type and a list of members available to that type which a deriving type implements).

No. There are no shim types. There are no "shape types." A shape is not a type; it's a structural description that a type can match. It can be represented in metadata using the same mechanisms as types, but it's not a type, and trying to make it into one will be a disaster for interoperability, suboptimal for performance (grandiose promises of low/no-cost shims notwithstanding), and limiting in power.

I know that some people are reluctant to expand the CLR's IL and metadata specifications, because Turing equivalence means it's always possible to implement new features as kludges in Roslyn, but the CLR 2.0 architecture is growing long in the tooth. The community has been asking for "operator constraints" since generics first came out, and when we finally get them, they should be implemented in the CLR type system the same as any other generic constraints.

acaly commented 3 years ago

A type must implement an interface to be that interface, if it doesn't it can't be used in a method constrained to that interface. We don't have structural typing in IL and so even if C# had structural typing, there would be some shim type in metadata that does implement the interface/shape/constraint for everything to work.

Yes, I was talking about language feature, not runtime feature for that point. I was not asking for representing structural types in IL. That would not be easy, not only for the runtime, but also for tools targeting IL.

If someone declares an interface and wants to provide a non-abstract/virtual helper method for all INumber, there is no reason that should be prevented.

That can easily be done with extension methods. Today extension methods are already stronger than DIMs in C#. We sometimes are forced to write static class XXXExtensions instead of directly writing a method into the type itself.

For the third point, I mean that the interface itself does not meet the requirement of itself if it contains a static abstract method. This is to solve the problem of the language/runtime don't have a method to call on the interface's abstract method. Note that there are corner cases that the language and runtime don't agree with each other: the language compiles, but the type cannot be loaded: https://github.com/dotnet/csharplang/discussions/4981#discussioncomment-1088206.

Note that "an interface does not meet the requirement of itself" is a proper example of conflating type and type of type. A type of type cannot be used on another type of type, only on types.

CyrusNajmabadi commented 3 years ago

Second-order logic is different from first-order logic.

I don't know what this means.

CyrusNajmabadi commented 3 years ago

if it is possible to have these as the declared type of a field, property, or method parameter, it is not a shape. It's a frankenhack that should not exist.

Why? You continue to assert things without stating why. That seems like it would just be a needless (and confusing) limitation.

tannergooding commented 3 years ago

There are no "shape types."

Sure on paper and in "type theory" that might be true, but in the real world (e.g. actual code executing on your hardware) there is something implementing the "contract" exposed by a shape so that different APIs can communicate with each other.

If you want .NET to call into C, C++, Java, Rust, Go, Swift, etc, you have to understand these details even if the languages don't directly expose it in code in this exact manner. To this extent, exposing this through interfaces is maybe exposing a bit much of how things actually work, but its how things actually work and so everything else can functionally be built on top of it and abstracted later.

masonwheeler commented 3 years ago

I was not asking for representing structural types in IL. That would not be easy, not only for the runtime, but also for tools targeting IL.

It gets a lot easier if you don't try to make it a general-purpose type. This is where the rule I described above comes to the rescue: if it is possible to have these as the declared type of a field, property, or method parameter, it is not a shape.

We already have two informal shapes defined in the C# compiler, which I informally call "SEnumerable" (the shape of any type that foreach accepts as its collection type) and "STask" (the shape of any type that can be awaited). These exist in local variable scopes only; you can't pass one around because it's not a formal type, just a description of how some part of the type works, and the compiler uses that information to use the type in the appropriate way.

Shapes should be a formalization of this concept, with one added ability: in addition to being usable as local variables, they should be usable as generic type constraints. Here, it would more likely be the JIT, rather than the language-level compiler, that figures out how to use the type members appropriately, but it would be essentially the same principle: substitute in any type that fits the shape, without any explicit interface matching (or shim types to shoehorn it into interface matching!) and get the behavior you want.

acaly commented 3 years ago

if it is possible to have these as the declared type of a field, property, or method parameter, it is not a shape. It's a frankenhack that should not exist.

Why? You continue to assert things without stating why. That seems like it would just be a needless (and confusing) limitation.

That's basically what I have said, second-order logic is different from first-order logic. It would be long and off-topic to expand it here and I suggest you just google it.

acaly commented 3 years ago

There are no "shape types."

Sure on paper and in "type theory" that might be true, but in the real world (e.g. actual code executing on your hardware) there is something implementing the "contract" exposed by a shape so that different APIs can communicate with each other.

If you want .NET to call into C, C++, Java, Rust, Go, Swift, etc, you have to understand these details even if the languages don't directly expose it in code in this exact manner. To this extent, exposing this through interfaces is maybe exposing a bit much of how things actually work, but its how things actually work and so everything else can functionally be built on top of it and abstracted later.

Note that this is C# repo and we are not only discussing how the runtime implements it, but more importantly, how they are exposed to programmers through C# language. I agree that under the hood it can be interface, but the language should forbid some use cases as I have described some above.

masonwheeler commented 3 years ago

There are no "shape types."

Sure on paper and in "type theory" that might be true, but in the real world (e.g. actual code executing on your hardware) there is something implementing the "contract" exposed by a shape so that different APIs can communicate with each other.

If a shape type appears in an API as anything other than a generic type parameter, you've already broken the rule.

CyrusNajmabadi commented 3 years ago

It will be a mess for anything outside of C# to interoperate with!

Why? I see lots of assertions, but no clarification on why this would be. Of all the .net parts, this seems like it would be really easy to incorporate into any language. Why would it not be?

CyrusNajmabadi commented 3 years ago

It would be long and off-topic

It's your claim. Asserting it and and then stating others have to go and research to understand what you mean isn't a reasonable way to support your case.

tannergooding commented 3 years ago

Note that this is C# repo and we are not only discussing how the runtime implements it, but more importantly, how they are exposed to programmers through C# language.

Yes, but this proposal is also not about shapes/traits/roles. Its about a different concept.

It just happens to be that shapes/traits/roles could be implemented on top of this feature because it has everything (or mostly so) that would be actually required for the runtime to successfully emit code for a shape/role/trait like feature.

acaly commented 3 years ago

It just happens to be that shapes/traits/roles could be implemented on top of this feature because it has everything (or mostly so) that would be actually required for the runtime to successfully emit code for a shape/role/trait like feature.

I think that this feature is introduced to prepare for "shape" (instead of "just happen to"). I don't think anyone has asked to have such a feature before the language team thought about how to implement shape (correct me if I am wrong). What ends up in the runtime today with generic arithmetic interfaces should indeed be shapes. I am not saying they should not be interfaces, given interfaces will become "shapes" (or have everything of a shape).

CyrusNajmabadi commented 3 years ago

but the language should forbid some use cases as I have described some above.

Perhaps. It seems like a minor part of the design here.

masonwheeler commented 3 years ago

It will be a mess for anything outside of C# to interoperate with!

Why? I see lots of assertions, but no clarification on why this would be. Of all the .net parts, this seems like it would be really easy to incorporate into any language. Why would it not be?

Everything seems like it should be really easy until you actually try to do it.

The problem is, when you actually do it, it has to be done in the same way as C# does it in order to preserve interoperability. If it's implemented as a frankenhack in Roslyn, this means that you have to reverse-engineer Roslyn code in order to build a compatible implementation. On the other hand if there's one definitive implementation in the runtime and all Roslyn does is write a thin wrapper around that, it's an order of magnitude simpler to get an additional language working.

This is coming from experience. I implemented C#'s last big frankenhack (async/await) in another compiler, and it was a months-long, painstakingly laborious process to get all the fiddly little semantics to work the way third-party libraries written in C# expected them. If some notion of a coroutine existed at the IL level, it would have been exponentially simpler. I'm trying to keep from having to go through that ever again, for me and for all the other language maintainers who have to keep up with the feature treadmill.

CyrusNajmabadi commented 3 years ago

Everything seems like it should be really easy until you actually try to do it.

What would not be easy here?

CyrusNajmabadi commented 3 years ago

If it's implemented as a frankenhack in Roslyn,

Nothing about the design so far requires any hacks afaict (and I haven't seen any hacks shown so far), so I really don't know what you're referring to.

CyrusNajmabadi commented 3 years ago

This is coming from experience.

Static abstracts are not at all similar to async/await. I have no idea why your experience there would have any bearing on this feature.

masonwheeler commented 3 years ago

I really don't know what you're referring to.

Referring to the concept of "witnesses" (the shims @tannergooding mentioned above) which are a terrible way to implement shapes, by people who don't understand that they serve a completely different purpose from interfaces.

tannergooding commented 3 years ago

What ends up in the runtime today with generic arithmetic interfaces should indeed be shapes.

Structural typing itself has many issues including that changing a generic constraint is a binary breaking change and that a given contract may not be intentional (you could expose a + operator that doesn't do addition like expected, see also << being used to output text or / being used to concat paths in C++).

Likewise, simply allowing something similar to where T : T operator +(T, T) is really no different (or more readable than) where T : IAdditionOperators<T, T, T> and both restrict the method now and forever (without a binary breaking change) to only support the most basic of addition operations available. In practice, operator + ends up being a primitive building block on which more usable contracts (like INumber) are built.

CyrusNajmabadi commented 3 years ago

If some notion of a coroutine existed at the IL level, it would have been exponentially simpler.

As I've mentioned, the feature here is literally just exposing the same concept at the runtime level directly through c#. Can you explain what about that you think it's a hack?

CyrusNajmabadi commented 3 years ago

which are a terrible way to implement shapes

Why?

masonwheeler commented 3 years ago

Structural typing itself has many issues including that changing a generic constraint is a binary breaking change

You say that like it's a bad thing.

In a statically typed system like .NET, that's the entire point. That's a feature, not a bug. When you make changes, you hit Build, you get errors, and you know exactly where your changes impacted so that you can fix them too. That's exactly what we want; why are you talking about it like it's scary?

acaly commented 3 years ago

What ends up in the runtime today with generic arithmetic interfaces should indeed be shapes.

Structural typing itself has many issues including that changing a generic constraint is a binary breaking change and that a given contract may not be intentional (you could expose a + operator that doesn't do addition like expected, see also << being used to output text or / being used to concat paths in C++).

Likewise, simply allowing something similar to where T : T operator +(T, T) is really no different (or more readable than) where T : IAdditionOperators<T, T, T> and both restrict the method now and forever (without a binary breaking change) to only support the most basic of addition operations available. In practice, operator + ends up being a primitive building block on which more usable contracts (like INumber) are built.

When I say "should indeed be shapes" I mean named shapes, not just structural. C++ has been using duck-typing for long, producing notoriously unhelpful compiler error messages. That is not something C# should repeat. However, the concept in C++ is a good design because it provides each duck-type a name. The programmer can then claim that a type matches a shape and if not, the compiler can produce error messages accordingly.

In case of breaking change, whenever you remove something from the public type, that's already a breaking change, no matter what you use (interfaces or shapes or just where operator +).

tannergooding commented 3 years ago

In a statically typed system like .NET, that's the entire point. That's a feature, not a bug. When you make changes, you hit Build, you get errors, and you know exactly where your changes impacted so that you can fix them too. That's exactly what we want; why are you talking about it like it's scary?

And that's exactly what you get from interfaces. A contract exposing static and instance members which can then be explicitly implemented by other types.

CyrusNajmabadi commented 3 years ago

That is not something C# should repeat.

I don't see why the impl would be unable to give good error messages here. Can you give an example of a case where you think that would be hard?

dotnet / csharplang

[Proposal]: Static abstract members in interfaces #4436

Static abstract members in interfaces

Summary

Motivation

Syntax

Interface members

Today's rules

Proposal

Abstract virtual members

Explicitly non-virtual static members

Implementation of interface members

Today's rules

Proposal

Semantics

Operator restrictions

Implementing static abstract members

Interface constraints with static abstract members

Accessing static abstract interface members

Drawbacks

Alternatives

Structural constraints

Default implementations

Virtual static members in classes

Unresolved questions

Design meetings