C# Design Notes for May 10, 2016

[MadsTorgersen]

C# Language Design Notes for May 10

In this meeting we took a look at the possibility of adding new kinds of extension members, beyond extension methods.

Extension members

Ever since we added extension methods in C# 3.0, there's been a consistent ask to add a similar feature for other kinds of function members. While "instance methods" are clearly the most important member kind to "extend with" from the outside, it seems arbitrary and limiting not to have e.g. properties, constructors or static members.

Unfortunately, the declaration syntax for extension methods is sort of a local optimum. The static method with the extra this parameter works great for methods: It is low cost, and it comes with an obvious disambiguation syntax. However, this approach doesn't carry over well to other kinds of members. A property with an extra this parameter? And how would extension static members attach to their type - cannot be through a this parameter representing an instance! As we were designing C# 4 we tried very hard to answer these questions and design an "extension everything" feature that was a syntactic extension (no pun intended) of extension methods. We failed, creating a succession of monsters until we gave up.

The other option is to take a step back, and do a different style of design. Most proposals nowadays do this, and the idea actually goes all the way back to debates we had when adding extension methods in the first place: instead of the extension members saying what they extend, maybe they should be grouped into "extensions"; class-like groups of extension methods that all extend the same type. With the enclosing "extension class" doing all the extension, the members can just be declared with their normal syntax:

class Person
{
    public string Name { get; }
    public Person(string name) { Name = name; }
}

extension class Enrollee extends Person
{
    static Dictionary<Person, Professor> enrollees = new ();                                            // static field
    public void Enroll(Professor supervisor) { enrollees[this] = supervisor; }                          // instance method
    public Professor Supervisor => enrollees.TryGetValue(this, out var supervisor) ? supervisor : null; // instance property
    public static ICollection<Person> Students => enrollees.Keys;                                       // static property
    public Person(string name, Professor supervisor) : this(name) { this.Enroll(supervisor); }          // constructor
}

Issue #11159 summarizes the recent proposals along these lines.

This syntactic approach has clear advantages - writing an extension member is as simple as writing a normal member. It does have some challenges, too. One is, what does it compile into? Another is, how do you disambiguate when several extension members are in scope? Those are trivially answered by the current extension methods, but need more elaborate answers here.

Looking at our current BCL libraries, it is clear that there are many places where layering is poor. Essentially, there's a trade off between layering (separating dependencies) and discoverability (offering members on the important types). If you have a file system, you want to offer a constructor overload that takes a file name. Extension constructors (for instance) address this.

Also, extension members allow putting concrete functionality on interfaces, and on specific instances of generic types.

We think that the most useful and straightforward kinds of extension methods to add would be instance and static versions of methods, properties and indexers, as well as static fields. We don't think instance fields make sense - where would they live? The answer to that is at best complicated, and probably not efficient.

Extension constructors would also be useful. There's a bit more of a design space here, depending on whether you want them to be just like instance constructors (which would require the specific extended type to already have a constructor to delegate to), or more like factories (which could be added to interfaces etc. and produce instances of more derived types).

Extension operators are in a bit of a gray zone. Some of them, like arithmetic operators, probably make sense. Equality operators are hard, because everyone's already got one, so the "built-in" would always shadow. Implicit conversion operators are especially problematic, because their invocation doesn't have an associated syntax, so it's very subtle when they are applied, and there's no easy way to choose not to do it.

Implementation-wise, the extension class would probably turn into a static class, and the extension members would probably be represented as static methods, with the most obvious mapping we can find. There's a case for implementing extension properties via "indexed properties" which are supported in IL (and VB), though maybe we'd prefer mapping to something that could be written directly in C#.

The name of the extension would be the name of the static class, and can also be used in disambiguation, for which we would need a syntax. Maybe existing syntax, such as ((Enrollee)person).Supervisor or (person as Enrollee).Supervisor could be used when disambiguating instance members or operators, whereas static members would just be accessed directly off of the generated static class, as in Enrollee.Students.

Arguably, consumption is more important than production. In this case, that is an argument both for and against introducing a new syntactic approach to declaring extension members: On the one hand, it makes it less of a problem if we effectively deprecate the existing syntax, on the other it makes it less compelling to add syntax for it, instead of maybe some attribute-based approach, or other less syntax-heavy solutions.

For extension properties, people often point to Enumerable.Count(IEnumerable src) as an example of why they want them, but that's a terrible example, because it isn't efficient. Even so, there's probably a good need for these - the Roslyn code base for instance has many little extension methods on top of interfaces, that "should" have been properties. Kind() is a perfect example.

Conclusion

We think this is worth pursuing further. We will come back to it in future design meetings to start ironing out the details.

[alrz]

Would it be a breaking change to rewrite an existing extension class, e.g. Enumerable with the new syntax? extension class Enumerable<T> extends IEnumerable<T> it seems that it would be turned to a generic class.

[jnm2]

If static extension fields can be added, static extension events seem like a logical thing to add. Actually, custom instance extension events make sense as well since they are only a pair of methods. (Similar story for static-only auto properties vs static and instance custom properties.)

Also, +1 for factories as extension constructors.

[miloush]

Can we introduce interfaces to classes this way? The class grouping syntax would be a natural fit for that.

[alrz]

@miloush You can already do that with partial classes. I doubt if implementing interfaces on external classes would be possible without CLR support.

[miloush]

@alrz yes I meant external classes. Well I guess that the extending class could be passed to the parameter of desired interface type instead (it would technically implement the interface), I don't necessarily see what would you need the CLR support for. It's the same with current extension methods - it is just a compile time thing.

[alrz]

@miloush It is not, because interfaces are dynamically dispatched. for example,

Bar bar = ... // Bar is extended to implement IFoo
M(bar);

void M(IFoo foo) {
  foo.Foo(); // how should the compiler know what method should be called? 
}

[miloush]

@alrz the compiler would just call IFoo.Foo() as usual, I would expect it to do work at call site and pass an instance of the extending class (or another form of wrapper) that implements the interface and has reference to bar.

[svick]

@miloush What about the following:

object o = new Bar();
if (o is IFoo)
   …

Would is somehow have to figure out that Bar has been extended (how?)? Or would casting to object create a wrapper? Could you then cast back to Bar?

Also see #9132 (same idea, different syntax).

[drewnoakes]

To clarify what @jnm2 said, it's not possible to allow regular extension events as they are essentially just fields (containing references to subscribers in a MulticastDelegate) and extension classes understandably can't have fields. However it should be possible to allow custom/explicit extension events:

extension class Enrollee extends Person
{
    public event EventHandler NameChanged
    {
        add { /* ... */ }
        remove { /* ... */ }
    }
}

I have difficulty coming up with a valid and useful scenario. If Person was INotifyPropertyChanged and you wanted to filter PropertyChanged events to only match nameof(Person.Name), then I don't think you could easily do that. It'd involve allocating a new delegate in the add and you'd have no way of storing the reference to it for the remove.

[Thaina]

As usual I don't like the idea of introduce new keyword. We could just reuse

static class Enrollee : Person
{
}

Since static class cannot has base class before. It could be use for telling that it is being extension class

[Thaina]

Another of my concern is generic. Is it support this syntax?

extension class Enrollee<T> extends T where T : class,IPerson,IComparable
{
}

If we has class implement IPerson and IComparable. Would the compiler be able to include this extensions class?

[jnm2]

@drewnoakes If Person was INotifyPropertyChanged and you wanted to filter PropertyChanged events to only match nameof(Person.Name), then I don't think you could easily do that. It'd involve allocating a new delegate in the add and you'd have no way of storing the reference to it for the remove.

You would be limited to forwarding the handler to other storage accessible via Person, which could be another object on Person or a service (if Person has a service provider) or even a ConditionalWeakTable if worst came to worst. I think the service provider scenario is very likely. I would have used this already if I could.

Or in the case of e.g. Entity Framework or any library- Mads mentioned the ideal of better layering- an object might have an interface that exposes the object's EventHandlerList. This would be in exactly the same vein as how all windows forms controls store their events (and all Components; see Component.Events). That enables all kinds of rich layering scenarios for those particular objects because your extension class could additionally create a new delegate pointing to your own behavior and store it with your unique object key in the target object's event handler list.

[MgSam]

Is writing Kind rather than Kind() really worth all this design effort?

Still believe it's not worth doing unless adding instance state is put on the table. #extensionfieldsorbust

[mharthoorn]

Is an extension class really a class? And does it then become a synonym of the extended class? It's better to call it what it is: a class extension, or just extension. The main reason to give an extension a name probably is to be able to find it back in your code. You're not going to reference it by name. The static modifier has no meaning either.

public static extension class Enrollee extends Person // neh...

// more concise
public extension Enrollee extends Person 
public extension Enrollee for Person 

[dfkeenan]

I don't think you need the class keyword there. extension should be the keyword like class, struct etc. Also I think a : would be better than the word keyword extends.

[miloush]

I do occasionally call extension methods explicitly on the declaring type.

I see the point of static modifier in the current model, as the methods are static. If state can be introduced using extensions, a non-static extending class feels natural way to express it, and can later go with the other familiar syntax (such as inheritance, generics).

Don't forget about accessibility via reflection. If you use extension instead of class and remove static, it is a bit confusing to ask for "static" methods on a "class" if you want to inspect the "extension". Or would there be a special treatment for extending objects and discovering what objects they extend?

I am also in favor of : as opposed to extends.

[Thaina]

C# have so many keyword that was not used on the class before. Please just reuse it

https://msdn.microsoft.com/en-us/library/x53a06bb.aspx?f=255&MSPPError=-2147217396

There are many keyword it could be use such as static class Enrolee : Person static extern class Enrolee : Person static implicit class Enrolee : Person static explicit class Enrolee : Person static virtual class Enrolee : Person static override class Enrolee : Person static this class Enrolee : Person

Please consider one of these before introduce another one like extension

[qrli]

@Thaina Here extension is just a contextual keyword, which means it is treated as keyword only before class. It will not break anything. All the combinations you listed are confusing, if not misleading. I'd prefer simple obvious contextual keyword instead.

[Thaina]

@qrli I know but even it not break anything, add more keyword is not so good in my opinion With the same reason we could reuse the old one as contextual keyword too but we don't need to introduce new one

I think extern class is not so misleading And to just make it static class S : A can be read as static class S extend A also intuitively without any keyword

[miloush]

@Thaina well extern class particularly looks like something you would want to have free in case you introduce actual extern classes.

I admit that I wasn't very happy seeing two new keywords either, but there is also cost in making existing keywords mean different things in different contexts.

Another option would be to use metadata, say ExtensionAttribute .. oh, wait.. 😉

[qrli]

@miloush Or even: ;-P

class Enrollee: ExtensionsOf<Person> {

or

static class Enrollee: ExtensionsOf<Person> {

or

static class Enrollee: Person {

[Thaina]

or maybe

static Enrollee : Person
{
}

Yeah, anyway it is not a class, just an extension. It can be extension of struct. So don't need type at all just static to enforce staticness, name to find reference, colon to state that it extend from a thing

[miloush]

@qrli The problem with the first two is that they create inheritance expectations on the extensions itself, like the ability to override another extension.

No but seriously, if state needs to be brought in, it just seems easy to lift the static requirement and require either constructor or explicit operator, e.g.

C# [Extension] public class PersonExtensions { public PersonExtensions(Person p) { ... } }

You don't introduce any new syntax to the language, and you even continue supporting extending different types within one extending class. And you can instantiate/call it only when the extension method is used. You can keep using the extending class explicitly, too.

Because the only other option I see is to do it the dependency property way, i.e. big dictionary.

[miloush]

Except that you need to keep reference to the instance of extension somewhere and get it back on the same instance of extended object. The above suggestion is not enough indeed.

[dotnetchris]

I've been wanting this level of extension support since the day extension methods were announced. This is an absolutely amazing feature to be added to C#

This should fit fantastically with record types and pattern matching. Long live functional c#

[aL3891]

Would be really nice to see this! extends feels a bit like Vb though, I prefer : as in

public extension class MyExtensions : Person { ...

I also think static could be omitted here because static extension members could just be marked static and instance extension methods could be declared as they normally would. it feels a little strange to have non static members inside a static class, and the extension keyword would tell the compiler to make everything static anyway. If state is added later on it would just work syntax wise

[TKharaishvili]

+1 for extension arithmetic operators

would be nice to have the ability to write something like list += item instead of list.Add(item) for existing collection classes.

extension properties would also be very useful for adding strongly typed wrappers around dictionary-like objects, for example a Session in asp.net

I'm also in favor of extension Student : Person syntax...

[shahid-pk]

why do we need to name the extension class ? is their any language grammar barrier that i am not seeing sorry for my ignorance.

for example we can take a much cleaner approach

class Person
{
    public string Name { get; }
    public Person(string name) { Name = name; }
}

// compiler can infer from name of the extension it is extension for Person
// as well as being in the same namespace may be as class Person
// It is cleaner as well
extension Person
{
    static Dictionary<Person, Professor> enrollees = new ();                                            // static field
    public void Enroll(Professor supervisor) { enrollees[this] = supervisor; }                          // instance method
    public Professor Supervisor => enrollees.TryGetValue(this, out var supervisor) ? supervisor : null; // instance property
    public static ICollection<Person> Students => enrollees.Keys;                                       // static property
    public Person(string name, Professor supervisor) : this(name) { this.Enroll(supervisor); }          // constructor
} 

i know it is exactly like how swift does it but still it looks more clean and not alien to C# and besides their is no shame in copying good stuff from other languages as other languages have copied stuff from C#

edit: sorry for the edit notification :smile:

[HaloFour]

@shahid-pk

why do we need to name the extension class ?

The name of the extension class becomes a part of the public contract. If the name of the class were to change existing consumers would need to be recompiled or they would break. Rather than having to come up with rules for how the compiler would derive a name it's probably safer to just have the developer explicitly provide one.

[shahid-pk]

@HaloFour by "if name of the class were to change" i assume you mean name of the existing class for which you wrote the extension ? But still in the current proposed syntax the name of the existing class is in the public contract

extension class Enrollee extends Person

so why would this existing contract not break if someone was to change name of the Person class ? sorry i am also trying to understand as well as sharing my view.

[shahid-pk]

and if you meant the extension class by "if name of the class were to change" then this would be breaking contract in the syntax i proposed too

extension Person

if you change the name Person , it will become extension of something else.

further more in future it could also be extended to include extensions for everything not just class for example

extension IEnumerable<T>   // extensions for Interface 

extension Point           // extensions for structure in a given context "namespace"

extension Point<T> 

and for differentiating between different extensions of a given class or type, rules for other types like structs or class should be applied to extensions.

edit : further context

[HaloFour]

@shahid-pk

I meant the name of the extension class. That extension class becomes a normal CLR class that exposes the extension members as static methods that accept the extended class as the first parameter.

public extension class PersonExtensions extends Person {
    public void SayHello() {
        Console.WriteLine($"Hello {this.Name}!");
    }
}
// is really
public static class PersonExtensions {
    public static void SayHello(Person @this) {
        Console.WriteLine($"Hello {@this.Name}!");
    }
}

When you invoke the extension methods the compiler rewrites the call into a static method call on the extension class:

void Main() {
    var person = new Person("Bill Gates");
    person.SayHello();
}
// is really
void Main() {
    var person = new Person("Bill Gates");
    PersonExtensions.SayHello(person);
}

As the extension class is a part of the static method call, the name of that class is actually a part of the public contract. If I were to change the name to something other than PersonExtensions, any existing libraries that consumed my extensions would fail to load.

This is an unavoidable problem. The extension methods have to exist in a static class with a predictable name. The easiest way to do that is to just make the name the responsibility of the developer. There have been a few comments in this thread about ways to perhaps auto-generate the name so that it could be emitted but those strategies often have different issues. If you think that you have a novel approach to generate the name I'm sure that the team would be interested in hearing it.

[shahid-pk]

@HaloFour thanks for clearing it up i understand it now. And about if i have a novel idea i think their is choice between verbosity "making the developer do more work and be explicit" and simplicity "making the compiler do more work", i like the latter but i know and understand their are constraints for making things simple.

I think the names for the extension class can be generated like how names for anonymous types are generated if we want/expect extension types for types in a same namespace or it could have simple rules if we want/expect/enforce similar rules as are for class types for example

public extension Person {
    public void SayHello() {
        Console.WriteLine($"Hello {this.Name}!");
    }
}

// can really be a compiler generated class by combining 
// (name of the extension) + the word Extensions/ or anything else
// if  we want single extension in a namespace for a type.
// The compiler will generate exactly the same thing for the above extension syntax

public static class PersonExtensions {
    public static void SayHello(Person @this) {
        Console.WriteLine($"Hello {@this.Name}!");
    }
}

// The name generation will have similar constraints as constraints for similar names 
// for two class have in a similar namespace

The rest of the work should be exactly like how you explained it :smile:

And beside all this i also don't see how is this

public extension class PersonExtensions extends Person {
    public void SayHello() {
        Console.WriteLine($"Hello {this.Name}!");
    }
}

better than this

public static class PersonExtensions {
    public static void SayHello(Person @this) {
        Console.WriteLine($"Hello {@this.Name}!");
    }
}

are we just avoiding having to write @this at the expense of writing extension, extend and Person ? or is their more to it that i don't understand ?

this excerpt from the design team seems to show the problem with the latter approach

Unfortunately, the declaration syntax for extension methods is sort of a local optimum. The static method with the extra this parameter works great for methods: It is low cost, and it comes with an obvious disambiguation syntax. However, this approach doesn't carry over well to other kinds of members. A property with an extra this parameter? And how would extension static members attach to their type -

But this can easily be deal with the syntax i proposed as well as making it more simple.

[aluanhaddad]

@shahid-pk If the name is synthesized based on the name of extended type, how would multiple extension classes that target that type work? Currently it's very useful to define extension methods on types from different namespaces and across different assemblies. Even if the extension classes were somehow implicitly partial and were required to be in the same namespace, how would this work across assemblies? There are other issues as well...

[HaloFour]

I guess in theory you could dump all of the extension members into a single class defined within the namespace since they could be overloaded by their "this" parameter:

namespace Foo {
    public extension Person {
        public void SayHello() => Console.WriteLine($"Hello {this.Name}!");
    }

    public extension Student {
        public bool IsPassing => this.Grade >= 65;
    }
}

// becomes

namespace Foo {
    public static partial class Extensions {
        public static void SayHello(Person @this) => Console.WriteLine($"Hello {@this.Name}!");

        public static bool get_IsPassing(Student @this) => @this.Grade >= 65;
    }
}

That way you don't have to worry about generating a name and you'd retain the ability to include extensions via using.

[shahid-pk]

@aluanhaddad How would multiple extension classes that target a type work ?

You can have one extension type per namespace , you can have another extension for the same type in a different namespace. i assume one would rarely need two different extension types for single type in a same namespace. if one do need to one always have the old syntax :smile:

Currently it's very useful to define extension methods on types from different namespaces and across different assemblies

i don't see how what i proposed stop you from doing that. could you elaborate ?

Even if the extension classes were somehow implicitly partial and were required to be in the same namespace

same namespace as in the type the extensions are extending and the actual type being extended ? no i did not propose that limitation, you can have extension for any type in any namespace as long as that namespace don't have another extension for the same type already extended.

for example

namespace foo {
   class bar { 
   }
}

can have

namespace foo {
    extension bar {
    }
}

and can also have in the same assembly or different assembly

namespace too {
    extension bar {
    }
}

you could also extend two different types in the same namespace

namespace too {
    extension bar {
    }
    extension jar {
    }
}

the only constraint is single extension for a given type per namespace. keep in mind the compiler will generate static class with a name generated through (type being extended) + the word extensions , so our only constraint is we can not have two similar named classes in the same namespace.

[aluanhaddad]

@HaloFour That makes sense, but we would lose the granular extension method importing we gained from the recent addition of using static.

@shahid-pk I see I misunderstood what you were proposing. I thought you were saying that the extension class would need to be logically defined alongside the extended class, which didn't make a lot of sense.

For what it's worth, I say just require a name for extension classes, it's easy to come up with names for them. This feature proposal offers a lot of value regardless.

[Thaina]

That not seem right. Given difference namespace in the same module may contain class with same name. To have automatically guess class may be ambiguous

[aluanhaddad]

@Thaina That's a good point. Synthetic names would likely need to be mangled in some way to help avoid this, and this would complicate tooling and documentation scenarios. I think they should be required to have programmer specified names.

[shahid-pk]

@Thaina This name generation was just an example you can always generate the name like how names are generated for anonymous type, or any other way because developer will never have to deal with the names directly. You could have C>>CCC + class name for example which will reduce the chances of name collision with user defined classes.

for example actual types are generated for anonymous types and the compiler takes care of name collision with existing types or with one another. It has already been done

[miloush]

If the name was autogenerated, how would you disambiguate between static methods in your extension and instance methods of the original class?

[shahid-pk]

@miloush the auto generated static class will have a different name which will include name of the original class+ something random/the word extensions/or anything. or the auto generated class does not even have to have a logical name as the developer is never supposed to deal with them, only the compiler should be aware that the static class is supposed to be an extension for a certain type.

[aluanhaddad]

@miloush He means the name of the extension class, not its members. But I still think this poses a problem for tooling and documentation. What exactly is gained by auto generating the names anyway? It seems like at adds an extraneous source of complexity to this very worthwhile, but undoubtedly complex proposal.

@shahid-pk Firstly, anonymous types are not shared across assemblies. Secondly, there are plenty of reasons to refer to the class name directly. A few of them area:

1. <see cref="SomeExtensionClass"/>
2. [Obsolete(nameof(Old.TypeExtensions) + " is deprecated. Use " + nameof(New.TypeExtensions))]
3. Referring to the class in find references, go to definition.
4. Importing a specific extension class via using static.
5. Passing extension methods defined in extension classes to other methods.

new [] { new [] { 1, 2, 3 }, new [] { 6, 3, 9 } }.Aggregate(Enumerable.Concat);

Again, what exactly is the benefit of anonymity that outweighs all of these problems?

[shahid-pk]

@aluanhaddad i agree with your points that it does poses problems for the 4 points you pointed out.

the effort i was putting was to simplify the declaration but i see it now that it will pose many problems and it will not be worth it making efforts to save a few key strokes and have all this complexity. Thanks

[miloush]

I am aware that you mean generating the name of the extending class. You would need the name if you had e.g. a static helper method in the extending class (i.e. a method that wouldn't be extending anything). You cannot fully qualify it if you don't know the name of the class.

[omariom]

@aL3891

Not very different from deriving..

// Extending
public extension class MyExtensions : Person { }

// Deriving
public class MyExtensions : Person { }

but it may be a good thing! )

[JeffreySax]

One syntax option I haven't seen mentioned is to use the same pattern as explicit interface implementations: qualify the member name by the type it extends. As with current extension methods, this syntax would be restricted to static classes only. Within the body of instance members, this refers to the implicit instance parameter. A standard method name is generated (GetXxx/SetXxx for properties, the friendly operator name for operators), but this can be overridden with a MethodName attribute similar to howIndexerName can override the default Item property name for indexers.

public static class PersonExtensions
{
    // Instance property:
    public bool Person.Passes { get { return this.Grade >= 65; } }
    // Static extension method:
    public static Person Person.FromJson(string jsonText) { /* ... */ }
    // Static extension operator:
    [MethodName("LogicalAnd")]
    public static bool[] bool[].operator&(bool[] a, bool[] b) { /* ... */ }
}

Constructor extensions can simply be methods that return an instance of the specified type:

    public Person.Person(string fullName)
    {
        var parts = fullName.Split(' ');
        return new Person(parts[0], parts[1]);
    }

Although I can imagine a constructor-like syntax:

public Person.Person(string fullName) : Person()
{
    var parts = fullName.Split(' ');
    this.FirstName = parts[0];
    this.LastName = parts[1];
    // implicit "return this"
}

Also for constructors, there is a syntax similar to current extension methods that might work: Put this with the return type. However, it's not really obvious that this is an extension constructor:

public static this Person Create(string fullName) { /* ... */ }

[Thaina]

@JeffreySax Your idea seem great. But would it be weird if use generic?

public static class PersonExtensions
{
    public static T T.FromJson<T>(string jsonText) where T : Serializable
    { /* ... */ }
}

Well, maybe not

[iam3yal]

@JeffreySax Do you know about #11159?

[jcouv]

LDM notes for May 10 2016 are available at https://github.com/dotnet/csharplang/blob/master/meetings/2016/LDM-2016-05-10.md I'll close the present issue. Thanks