Make mutable generic collection interfaces implement read-only collection interfaces

[TylerBrinkley]

Rationale

It's long been a source of confusion that the mutable generic collection interfaces don't implement their respective read-only collection interfaces. This was of course due to the read-only collection interfaces being added after the fact and thus would cause breaking changes by changing a published interface API.

With the addition of default interface implementations in C#8/.NET Core 3.0 I think the mutable generic collection interfaces, ICollection<T>, IList<T>, IDictionary<K, V>, and ISet<T> should now implicitly inherit their respective read-only collection interfaces. This can now be done without causing breaking changes.

While it would have been nice for these interfaces to share members, I think the proposed API below is the best we can possibly do with the read-only interfaces being added after the fact.

As an added bonus, this should allow some simplification of the type checking in LINQ code to check for the read-only interfaces instead of the mutable interfaces.

Proposed API

 namespace System.Collections.Generic {
-    public interface ICollection<T> : IEnumerable<T> {
+    public interface ICollection<T> : IReadOnlyCollection<T> {
-        int Count { get; }
+        new int Count { get; }
+        int IReadOnlyCollection<T>.Count => Count;
     }
-    public interface IList<T> : ICollection<T> {
+    public interface IList<T> : ICollection<T>, IReadOnlyList<T> {
-        T this[int index] { get; set; }
+        new T this[int index] { get; set; }
+        T IReadOnlyList<T>.this[int index] => this[index];
     }
-    public interface IDictionary<TKey, TValue> : ICollection<KeyValuePair<TKey, TValue>> {
+    public interface IDictionary<TKey, TValue> : ICollection<KeyValuePair<TKey, TValue>>, IReadOnlyDictionary<TKey, TValue> {
-        TValue this[TKey key] { get; set; }
+        new TValue this[TKey key] { get; set; }
-        ICollection<TKey> Keys { get; }
+        new ICollection<TKey> Keys { get; }
-        ICollection<TValue> Values { get; }
+        new ICollection<TValue> Values { get; }
-        bool ContainsKey(TKey key);
+        new bool ContainsKey(TKey key);
-        bool TryGetValue(TKey key, out TValue value);
+        new bool TryGetValue(TKey key, out TValue value);
+        TValue IReadOnlyDictionary<TKey, TValue>.this[TKey key] => this[key];
+        IEnumerable<TKey> IReadOnlyDictionary<TKey, TValue>.Keys => Keys;
+        IEnumerable<TValue> IReadOnlyDictionary<TKey, TValue>.Values => Values;
+        bool IReadOnlyDictionary<TKey, TValue>.ContainsKey(TKey key) => ContainsKey(key);
+        bool IReadOnlyDictionary<TKey, TValue>.TryGetValue(TKey key, out TValue value) => TryGetValue(key, out value);
     }
-    public interface ISet<T> : ICollection<T> {
+    public interface ISet<T> : ICollection<T>, IReadOnlySet<T> {
-        bool IsProperSubsetOf(IEnumerable<T> other);
+        new bool IsProperSubsetOf(IEnumerable<T> other);
-        bool IsProperSupersetOf(IEnumerable<T> other);
+        new bool IsProperSupersetOf(IEnumerable<T> other);
-        bool IsSubsetOf(IEnumerable<T> other);
+        new bool IsSubsetOf(IEnumerable<T> other);
-        bool IsSupersetOf(IEnumerable<T> other);
+        new bool IsSupersetOf(IEnumerable<T> other);
-        bool Overlaps(IEnumerable<T> other);
+        new bool Overlaps(IEnumerable<T> other);
-        bool SetEquals(IEnumerable<T> other);
+        new bool SetEquals(IEnumerable<T> other);
// Adding this new member is required so that there's a most specific Contains method on ISet<T> since ICollection<T> and IReadOnlySet<T> define it too
+        new bool Contains(T value) => ((ICollection<T>)this).Contains(value); 
+        bool IReadOnlySet<T>.Contains(T value) => ((ICollection<T>)this).Contains(value);
+        bool IReadOnlySet<T>.IsProperSubsetOf(IEnumerable<T> other) => IsProperSubsetOf(other);
+        bool IReadOnlySet<T>.IsProperSupersetOf(IEnumerable<T> other) => IsProperSupersetOf(other);
+        bool IReadOnlySet<T>.IsSubsetOf(IEnumerable<T> other) => IsSubsetOf(other);
+        bool IReadOnlySet<T>.IsSupersetOf(IEnumerable<T> other) => IsSupersetOf(other);
+        bool IReadOnlySet<T>.Overlaps(IEnumerable<T> other) => Overlaps(other);
+        bool IReadOnlySet<T>.SetEquals(IEnumerable<T> other) => SetEquals(other);
+    }
 }

Binary Compatibility Test

I was able to test that this change doesn't break existing implementers with the following custom interfaces and by simply dropping the new interfaces dll to the publish folder without recompiling the consuming code, the IMyReadOnlyList<T> interface was automatically supported without breaking the code.

Original Interfaces DLL code

namespace InterfaceTest
{
    public interface IMyReadOnlyList<T>
    {
        int Count { get; }
        T this[int index] { get; }
    }

    public interface IMyList<T>
    {
        int Count { get; }
        T this[int index] { get; set; }
    }
}

New Interfaces DLL code

namespace InterfaceTest
{
    public interface IMyReadOnlyList<T>
    {
        int Count { get; }
        T this[int index] { get; }
    }

    public interface IMyList<T> : IMyReadOnlyList<T>
    {
        new int Count { get; }
        new T this[int index] { get; set; }
        int IMyReadOnlyList<T>.Count => Count;
        T IMyReadOnlyList<T>.this[int index] => this[index];
    }
}

Consuming Code

using System;
using System.Collections.Generic;

namespace InterfaceTest
{
    class Program
    {
        static void Main()
        {
            var myList = new MyList<int>();
            Console.WriteLine($"MyList<int>.Count: {myList.Count}");
            Console.WriteLine($"IMyList<int>.Count: {((IMyList<int>)myList).Count}");
            Console.WriteLine($"IMyReadOnlyList<int>.Count: {(myList as IMyReadOnlyList<int>)?.Count}");
            Console.WriteLine($"MyList<int>[1]: {myList[1]}");
            Console.WriteLine($"IMyList<int>[1]: {((IMyList<int>)myList)[1]}");
            Console.WriteLine($"IMyReadOnlyList<int>[1]: {(myList as IMyReadOnlyList<int>)?[1]}");
        }
    }

    public class MyList<T> : IMyList<T>
    {
        private readonly List<T> _list = new List<T> { default, default };

        public T this[int index] { get => _list[index]; set => _list[index] = value; }

        public int Count => _list.Count;
    }
}

Original Output

MyList<int>.Count: 2
IMyList<int>.Count: 2
IMyReadOnlyList<int>.Count:
MyList<int>[1]: 0
IMyList<int>[1]: 0
IMyReadOnlyList<int>[1]:

New Output

MyList<int>.Count: 2
IMyList<int>.Count: 2
IMyReadOnlyList<int>.Count: 2
MyList<int>[1]: 0
IMyList<int>[1]: 0
IMyReadOnlyList<int>[1]: 0

Moved from dotnet/runtime#16151

Updates

• Added ISet<T> implementing IReadOnlySet<T> since https://github.com/dotnet/runtime/issues/2293 has been implemented.

[safern]

@terrajobst I would like to have your input on these w.r.t new default implementations and adding new interfaces or new members to interfaces.

[GrabYourPitchforks]

For folks confused as to why this would be a breaking change without default interface implementations, consider three separate assemblies defined as such:

namespace MyClassLib
{
    public interface IFoo
    {
        void PrintHello();
    }
}

namespace MyOtherClassLib
{
    public interface IBar
    {
        void PrintHello();
    }
}

namespace MyApplication
{
    class Program
    {
        static void Main(string[] args)
        {
            object myFoo = MakeNewMyFoo();

            IFoo foo = myFoo as IFoo;
            if (foo is null)
            {
                Console.WriteLine("IFoo not implemented.");
            }
            foo?.PrintHello();

            IBar bar = myFoo as IBar;
            if (bar is null)
            {
                Console.WriteLine("IBar not implemented.");
            }
            bar?.PrintHello();
        }

        [MethodImpl(MethodImplOptions.NoInlining)]
        private static object MakeNewMyFoo()
        {
            return new MyFoo();
        }

        class MyFoo : IFoo
        {
            void IFoo.PrintHello()
            {
                Console.WriteLine("Hello from MyFoo.IFoo.PrintHello!");
            }
        }
    }
}

Compile and run the main application, and you'll see the following output.

Hello from MyFoo.IFoo.PrintHello!
IBar not implemented.

Now change the assembly which contains IFoo to have the following code, and recompile and redeploy that assembly while not recompiling the assembly which contains MyApplication.

namespace MyClassLib
{
    public interface IFoo : IBar
    {
    }
}

The main application will crash upon launch with the following exception.

Unhandled exception. System.MissingMethodException: Method not found: 'Void MyClassLib.IFoo.PrintHello()'.
   at MyApplication.Program.Main(String[] args)

The reason for this is that the method that would normally be at the slot IFoo.PrintHello was removed and instead there's a new method at the slot IBar.PrintHello. However, this now prevents the type loader from loading the previously-compiled MyFoo type, as it's trying to implement an interface method for which there's no longer a method entry on the original interface slot. See https://stackoverflow.com/a/35940240 for further discussion.

As OP points out, adding default interface implementations works around the issue by ensuring that an appropriate method exists in the expected slot.

[TylerBrinkley]

@terrajobst any input on this collection interface change as well?

[TylerBrinkley]

@safern @GrabYourPitchforks @terrajobst Is this something we could get triaged soon? I'd love for this to make it into .NET 5.

[safern]

@eiriktsarpalis and @layomia are the new System.Collections owners so I'll defer to them.

[TylerBrinkley]

@terrajobst

We can't make ISet<T> extend IReadOnlySet<T> (for the same reason that we couldn't do for the other mutable interfaces).

Is this still true even with default interface methods? Does that mean dotnet/corefx#41409 should be closed?

We discussed this. We used to think that that DIMs would work, but when we walked the solution we concluded that it would result commonly in a shard diamond which would result in an ambiguous match. However, this was recently challenged so I think I have to write it down and make sure it's actually working or not working.

Here's a comment @terrajobst had about this in a separate issue, https://github.com/dotnet/runtime/issues/2293#issuecomment-579524652

Hoping for some more insight.

[TylerBrinkley]

I updated the proposal to add a Contains DIM to ISet<T> as the current Contains method is defined on ICollection<T> and so if you tried to call Contains on an ISet<T> the call would be ambiguous with the ICollection<T> and IReadOnlySet<T> versions.

The new method would take precedence in this case and it's implementation if not overridden would delegate to ICollection<T>'s implementation as IReadOnlySet<T>'s does. It's not ideal but this is the only incidence of such a solution where this is required for the proposal.

[TylerBrinkley]

As an alternative since IReadOnlySet<T> hasn't released yet, instead of adding another Contains method to the list including ICollection<T>, IReadOnlySet<T>, and now ISet<T> we could instead add an invariant read-only collection interface as proposed in https://github.com/dotnet/runtime/issues/30661 and have it implement a Contains method and then have IReadOnlySet<T> implement this new invariant interface and remove it's Contains method.

With that change then ICollection<T> could implement this new invariant read-only collection interface.

So instead of this.

namespace System.Collections.Generic {
    public interface ICollection<T> : IReadOnlyCollection<T> {
        bool Contains(T value);
        ...
    }
    public interface IReadOnlySet<T> : IReadOnlyCollection<T> {
        bool Contains(T value);
        ...
    }
    public interface ISet<T> : IReadOnlySet<T> {
        new bool Contains(T value) => ((ICollection<T>)this).Contains(value);
        bool ICollection<T>.Contains(T value);
        bool IReadOnlySet<T>.Contains(T value) => ((ICollection<T>)this).Contains(value);
        ...
    }
}

it would be this.

namespace System.Collections.Generic {
    public interface IReadOnlyCollectionInvariant<T> : IReadOnlyCollection<T> {
        bool Contains(T value);
        ...
    }
    public interface ICollection<T> : IReadOnlyCollectionInvariant<T> {
        new bool Contains(T value);
        bool IReadOnlyCollectionInvariant<T>.Contains(T value) => Contains(value);
        ...
    }
    public interface IReadOnlySet<T> : IReadOnlyCollectionInvariant<T> {
        ...
    }
}

[mikernet]

As a follow-up to this:

Perhaps LINQ methods that have optimized execution paths on interfaces like IList<T> could be updated to optimize on IReadOnlyList<T> instead since all IList<T> will now implement IReadOnlyList<T> and then new code can begin moving to a sane development model where custom read-only collections don't need to extraneously implement the writable interfaces as well. I recall the justifications for optimizing LINQ on writable interfaces instead of read-only interfaces were:

1. IReadOnlyList<T> is covariant and thus slower.
2. IReadOnlyList<T> hasn't been around as long as List<T> so some code may not implement it and thus not get the optimizations.
3. It is "standard" to implement both IList<T> and IReadOnlyList<T> on all list-like collections anyway, I believe mostly because of the perf/optimization issues above?

I could be wrong but I believe there have been runtime updates to address point 1. Point 2 would be addressed by this issue. Point 3 is something that we could move away from with these changes to make read-only collection interfaces real first-class citizens in the framework that are actually widely usable.

A remaining sore point would be the annoying lack of support for read-only interfaces in many places where they should be supported as inputs and adding overloads that accept read-only interfaces is problematic because many collections currently implement both IList<T> and IReadOnlyList<T> directly so overload resolution would fail. This could be addressed in the BCL by updating writable collections to only directly implement the writable interface but might be confusing if people's code no longer compiles after updating to a new runtime version when using third-party collections that haven't been updated as such.

The above issues combined with the inability to pass an IList<T> into a method that accepts an IReadOnlyList<T> almost aways prevents me from using read-only collection interfaces in my code. Collections are easily one of the most poorly designed areas of the BCL and so fundamental to development that cleaning this up as much as possible with a bit of help from DIMs would be really nice.

[vpenades]

I would like to propose an alternative solution:

For example, for a List we have this:

IList<T>
IReadOnlyList<T>
List<T> : IList<T> , IReadOnlyList<T>

What I propose is this:

IList<T>
IReadOnlyList<T>
IWriteableList<T> : IList<T>, IReadOnlyList<T>
List<T> : IWriteableList<T>

So, from the point of view of the runtime, it would just require introducing a new interface, so I don't think it would break any compatibility. And developers would progressively move from using IList to IWriteableList

[jhudsoncedaron]

So it turns out this is a very peanut-buttery performance optimization as well. There are a number of places in Linq that check for IEnumerable for ICollection or IList for performance improvements and the report is that these can't be changed to check both for performance reasons; yet I found place after place in my own code where only IReadOnlyCollection/List is provided and a mutable collection would be meaningless. I also found many places where ICollection/List is dynamically downcast to its ReadOnly variant with an actual synthesis if the downcast fails.

Particularly to the implemenation of .Select, I found that making this change to that all ICollection/List also provide their IReadOnly variants yields 33-50% performance improvement at only the cost of JIT time (to provide the interfaces where they are not). I also found that adding an IReadOnlyCollection.CopyTo() is a possible improvement but the gains aren't needed because implementing IListSource on the projection returned by Select() on an IReadOnlyCollection yields more gain than that ever could.

Synthetic benchmark attached showing the component changes. The first number is garbage (ensures the memory is paged in); the second and third numbers show the baseline, and the last number shows the gains. (It's approximately equal to the second number, which means I found all the performance gains). I found on my machine about every third run is garbage due to disturbance, so run it three times and keep the closest two numbers. selectbenchmarks.zip

[jhudsoncedaron]

Update: Saying that IReadOnlyCollection.CopyTo() was not needed because of a better implementation involving IListSource() proved to be overfitting to the test.

It turns out that there is more cheap (but not free) performance improvements lying around for direct invocation of .ToList() and .ToArray() (commonly used for shallow copies of arrays), List.AddRange(), List.InsertRange(), and List's own Constructor to the point where adding IReadOnlyCollection<T>.CopyTo(T array, int offset); is worthwhile. There is no gain for the default implementation but a custom implementation providing .CopyTo() yields cheap performance gains.

The default implementation would be:

public partial interface IReadOnlyCollection<T> {
    void CopyTo(T[] target, int offset) {
        if (this is ICollection<T> coll)  { coll.CopyTo(target, offset); return ; }
        foreach (var item in this) target[offset++] = item;
        return;
    }
}

[eiriktsarpalis]

Potential alternative design: #23337.

Please also see this comment demonstrating the potential for compile-time and runtime diamond errors when adding DIMs to existing interfaces. Would need to investigate if the current proposal is susceptible to the same issue (I suspect it might not be as long as we're not adding new methods).

[jhudsoncedaron]

@eiriktsarpalis : I found that to be not an alternative design but a different design defect that could be fixed independently. ICollection<T> : IReadOnlyCollection<T> is not the heart of this one but rather IList<T> : IReadOnlyList<T>, IDictionary<T> : IReadOnlyDictionary<T>, and ISet<T> => IReadOnlySet<T>.

[jhudsoncedaron]

Not implementing this is slowly causing us more and more problems. The root is IList<T> and IReadOnlyList<T> are conflicting overloads (along with ICollection<T> and IReadOnlyCollection<T>). We have a large number of implementations of these interfaces and a decent number of extension methods on lists. The conflicting overloads are causing chronic problems.

It almost feels like a dumb compiler, but it's not. It doesn't matter which way half of the the overloads are resolved. The implementations are copy/paste of each other.

Latest trigger. One of our programmers really likes calling .Any() for readability as opposed to .Count > 0. In theory I could go and make some stupidly-fast extension methods but I actually can't because of the overload conflict, so we're stuck with Any<T>(IEnumerable<T>) and a dynamic downcast and as often as not an enumeration.

[mikernet]

@jhudsoncedaron This could potentially alleviate some of those issues:

https://github.com/dotnet/csharplang/discussions/4867

Would still be nice to fix collections though.

[jhudsoncedaron]

@mikernet : It won't. To work I need it to do exactly what it says it won't do: "Overload priorities should only be considered when disambiguating methods within the same assembly". I need to win (or in one case lose) against overloads shipped with .NET itself.

[mikernet]

Not for the example you showed you don't. Your ICollection or IReadOnlyCollection overloads WILL beat the IEnumerable overload from the BCL.

[jhudsoncedaron]

@mikernet : Breaking example: IReadOnlyDictionary<TK, TV>.GetValueOrDefault(TK key, TV default). Discovered in our codebase where these two had been hanging out for some time until somebody tried to call one of them directly on a Dictionary<TK, TV>. The one on IDictionary<TK, TV> is stock.

[mikernet]

I didn't say it will solve all your problems. I said it could alleviate some of the ones you showed.

Also, regarding:

One of our programmers really likes calling .Any() for readability as opposed to .Count > 0

There are several analyzers that detect this. I don't know off hand which rule we use - probably style cop or roslynator. Just set a company-wide analyzer rule to warn when someone does that and then he can't merge code that violates the analyzer rule - problem solved :P It's fairly well known that LINQ methods shouldn't be preferred over faster alternatives when available.

[Joe4evr]

There are several analyzers that detect this. I don't know off hand which rule we use - probably style cop or roslynator.

FWIW, the one in-box with the SDK is: CA1826 - Do not use Enumerable methods on indexable collections (I think)

[Bip901]

This is a highly requested feature not only for performance reasons but also for object oriented-ness and greatly simplified code. It has been requested in tens of issues in this repository, some of them mentioned here.

Will we be seeing this feature in .NET 7?

[joshudson]

@Bip901: I could make a PR immediately; but the API review team is unwilling to approve the API change.

[eiriktsarpalis]

It's not an issue of unwillingness. As it stands, it seems unlikely we would consider augmenting existing interfaces with DIMs, because this is known to create both compile-time and runtime breaking changes in downstream consumers due to the potential for diamond ambiguities. See https://github.com/dotnet/runtime/issues/52775#issuecomment-843446017 for an example.

Related to #55106.

[Joe4evr]

@eiriktsarpalis As an alternative, is it worth shipping an analyzer that detects types implementing ICollection<T>/IList<T> without the ReadOnly versions, and suggests to add them to the type declaration? (If I'm not mistaken, the members in read-only interfaces are a strict sub-set of the writable versions, so it wouldn't require adding new members that the type didn't already have (explicit implementation notwithstanding).)

[Timovzl]

As an alternative, is it worth shipping an analyzer that detects types implementing ICollection/IList without the ReadOnly versions, and suggests to add them to the type declaration?

I love this idea as an alternative. While it does not make for a foolproof solution, it should decrease the chance of encountering missing readable interfaces over time.

It would be especially great if .NET's own classes started adhering to the proposed analyzer. For example, consider the concrete Grouping type used by LINQ's ToLookup(). It implements IList<T> but not IReadOnlyList<T>. For one thing, this hinders optimizations by library authors. As an example, one might optimize for IReadOnlyList<T> (can index instead of enumerating) or IReadOnlyCollection<T> (can use multiple enumeration if necessary). However, separately optimizing for readable and writable interfaces tends to introduce an overgrowth of special cases and branches.

The effects of such missing interfaces keep spreading, as seen in this example. Out of performance considerations, the optimizations are often made only on writable types like ICollection<T>. As a result, concrete readonly types tend to miss the benefits.

[Bip901]

That's a good alternative. I'd still prefer a real solution. These interfaces are so widespread that they deserve a breaking change (then again, you can use the same fact to argue against breaking them).

[joshudson]

@eiriktsarpalis Take your breaking change already before someone pushes a fork of the framework to nuget and causes chaos.

I was actually plotting long ago to write up a broad-stroke solution involving patching it in the loader before someone came along and found that this specific change of adding a base interface almost never causes binary level breaking interfaces. In this case, someone would have to have IReadOnlyCollection at the root of their own diamond. Using ICollection as one of the legs won't do it.

[terrajobst]

Analyzers aren't a viable alternative for breaking changes in highly used APIs because it would likely immediately break the entire library ecosystem.

[jhudsoncedaron]

"it would likely immediately break the entire library ecosystem."

With this specific change, I'm anticipating few if any breaking changes in practice from the added default methods. While trying to add new methods to IEnumerable and overriding them below does not work, the tight-in checks that mimic this sequence of changes don't fail in the same way.

The next-best alternative is adding IList2 : IList, IReadOnlyList and changing hundreds of properties over to using it thus taking hundreds of breaking changes. The weight of this design defect does not stop growing.

[terrajobst]

The problem, and it has been a while that I looked, was that there were diamond dependencies in how customers implement collection interfaces which would result in TypeLoadExceptions because the runtime can't decide which default implementation to use. At compile time, it would fail to compile, forcing the developer to pick, but at runtime it's basically a fatal crash.

[TylerBrinkley]

It's not an issue of unwillingness. As it stands, it seems unlikely we would consider augmenting existing interfaces with DIMs, because this is known to create both compile-time and runtime breaking changes in downstream consumers due to the potential for diamond ambiguities. See #52775 (comment) for an example.

Related to #55106.

@eiriktsarpalis I do not think your diamond problem example is analagous to what's proposed here. In your example you added a completely new member and defined multiple DIM's for it across different interfaces that do not inherit from each other. That is quite different to this proposal which does not add any new members and only has a single DIM defined for each member and the interfaces do inherit from each other.

Everytime I look at this I fail to see how a diamond could occur. Please if anyone has a true example of how a diamond would occur with this proposal I'd love to know.

[jhudsoncedaron]

I too have utterly failed to reproduce the binary load problem. Here is a much more complete demo of the changes proposed and I could not construct a reasonable failure case.

Attached is a sample set of projects. To test, build CollectionDemo, Beta\CollectionInterfaces and Gamma\CollectionInterfaces. Run the demo binary, then copy the dll from Beta or Gamma over the dll in the name name in CollectionDemo and run. CollectionDemo contains some tests for List and Dictionary to force them to be loaded and at the very end a precise check to see if the change was applied correctly.

CollectionInterfaces.zip

Changeset BETA is a mirror of the cautious proposal here. It applied without load errors as expected. Changeset GAMMA is an aggressive change hunting out performance problems in Linq and List.InsertRange. To my surprise, it also applied without load errors.

Summary code:

    public interface IReadOnlyCollection<out T> : IEnumerable<T>
    {
        int Count { get; }
    }

#if GAMMA
    public interface IReadOnlyCollectionExact<T> : IReadOnlyCollection<T>
    {
        bool Contains(T item) { /* snip */ }
        void CopyTo(T[] array, int arrayIndex) { /* snip */ }
    }
#endif

    public interface IReadOnlyList<out T> : IReadOnlyCollection<T>
    {
        T this[int index] { get; }
    }

    public interface IReadOnlyDictionary<TK, TV> : IReadOnlyCollection<System.Collections.Generic.KeyValuePair<TK, TV>>
#if GAMMA
        , IReadOnlyCollectionExact<System.Collections.Generic.KeyValuePair<TK, TV>>
#endif
    {
        bool TryGetValue(TK key, out TV value);
        TV this[TK key] { get; }
        IEnumerable<TK> Keys { get; }
        IEnumerable<TV> Values { get; }

#if GAMMA
        void IReadOnlyCollectionExact<System.Collections.Generic.KeyValuePair<TK, TV>>.CopyTo(System.Collections.Generic.KeyValuePair<TK, TV>[] array, int arrayIndex)
            => ((ICollection<System.Collections.Generic.KeyValuePair<TK, TV>>)this).CopyTo(array, arrayIndex);

        bool IReadOnlyCollectionExact<System.Collections.Generic.KeyValuePair<TK, TV>>.Contains(System.Collections.Generic.KeyValuePair<TK, TV> item)
            => this.TryGetValue(item.Key, out var value) && object.Equals(item.Value, value);
#endif
    }

    public interface ICollection : IEnumerable
    {
        int Count { get; }
        bool IsSynchronzied { get; }
        object SyncRoot { get; }
    }

    public interface ICollection<T> : IEnumerable<T>
#if BETA
        , IReadOnlyCollection<T>
#endif
#if GAMMA
        , IReadOnlyCollectionExact<T>
#endif
    {
        int Count { get; }
        void Add(T item);
        void Clear();
        bool Contains(T item);
        void CopyTo(T[] array, int arrayIndex);
        bool Remove(T item);

#if BETA
        int IReadOnlyCollection<T>.Count => Count;
#endif
#if GAMMA
        bool IReadOnlyCollectionExact<T>.Contains(T item) => Contains(item);
        void IReadOnlyCollectionExact<T>.CopyTo(T[] array, int arrayIndex) => CopyTo(array, arrayIndex);
#endif
    }

    public interface IList<T> : ICollection<T>
#if BETA
        , IReadOnlyList<T>
#endif
    {
        T this[int index] { get; set; }
        void Insert(int index, T item);
        void RemoveAt(int index);

#if BETA
        int IReadOnlyCollection<T>.Count => Count;
        T IReadOnlyList<T>.this[int index] => this[index];
#endif
    }

    public interface IDictionary<TK, TV> : ICollection<System.Collections.Generic.KeyValuePair<TK, TV>>
#if BETA
        , IReadOnlyDictionary<TK, TV>
#endif
    {
        TV this[TK key] { get; set; }
        ICollection<TK> Keys { get; }
        ICollection<TV> Values { get; }
        void Add(TK key, TV value);
        bool ContainsKey(TK key);
        bool TryGetValue(TK key, out TV value);
        bool Remove(TK key);

#if BETA
        int IReadOnlyCollection<System.Collections.Generic.KeyValuePair<TK, TV>>.Count => Count;
        TV IReadOnlyDictionary<TK, TV>.this[TK key] => this[key];
        IEnumerable<TK> IReadOnlyDictionary<TK, TV>.Keys => Keys;
        IEnumerable<TV> IReadOnlyDictionary<TK, TV>.Values => Values;
        bool IReadOnlyDictionary<TK, TV>.TryGetValue(TK key, out TV value) => TryGetValue(key, out value);
#endif
    }

If there is a failure mode it should be possible to mutate CollectionDemo into triggering it while still keeping CollectionInterfaces matching this proposal. (I'm operating on a cut-down version to save time. I should have all cases covered, but I might not.)

There's a guy who wants a non-generic IReadOnlyCollection inserted directly beneath IEnumerable. After seeing changeset GAMMA work I'm convinced even that could be made to work. That causes the diamond to appear and the load to finally fail.

[MichalStrehovsky]

Consider following valid user code:

interface IMyDefaultReadOnlyCollection<T> : IReadOnlyCollection<T>
{
    int IReadOnlyCollection<T>.Count => 0;
}

class MyCollection<T> : ICollection<T>, IMyDefaultReadOnlyCollection<T>
{
    // Adding implementation of ICollection<T> left as an exercise
    // IReadOnly is left default-implemented from IMyDefaultReadOnlyCollection
}

Now suppose we add the IReadOnlyCollection<T> implementation to ICollection<T>. The implementation of IReadOnlyCollection<T>.Count on MyCollection<T> is now ambiguous between the implementation provided by IMyDefaultReadOnlyCollection<T> (old) and ICollection<T> (added in the breaking change). The class won't compile or work at runtime (if compiled against old BCL).

There are very few scenarios where default interface methods are non-breaking. The rules are basically captured in https://github.com/dotnet/corefx/pull/41949.

[Bip901]

It's true that this is a breaking change, but the benefits far outweigh the relatively rare breaking cases. Better fix this API late than never.

[vpenades]

It's true that this is a breaking change, but the benefits far outweigh the relatively rare breaking cases. Better fix this API late than never.

What about nuget packages that are no longer maintained but are critical for a number of projects? unless the "relatively rare cases" can be quantified, I think this change might be beneficial for some developers, and extremely harmful for others.

Also, this change would be added to the newer frameworks, most probably Net7, so it would introduce a behavior conflict between net7 and older frameworks, specially netstandard2.0 which is widely used as the baseline by the majority of nuget packages.

[Bip901]

What about nuget packages that are no longer maintained but are critical for a number of projects?

Then those projects shouldn't migrate to .NET 7 until they've migrated their dependencies, just like projects couldn't migrate to .NET Core while they had .NET Framework dependencies. Microsoft is known as a company that excels in backwards compatibility, but sometimes it's beneficial to make bold decisions.

unless the "relatively rare cases" can be quantified

Perhaps we could search a repository of libraries for explicit implementations of readonly interfaces (e.g. regex IReadOnlyCollection<.+>\.Count).

[stephentoub]

Perhaps we could search a repository of libraries for explicit implementations of readonly interfaces (e.g. regex IReadOnlyCollection<.+>.Count).

https://grep.app/search?q=IReadOnlyCollection%3C.%2B%3E%5C.Count&regexp=true So... not so rare.

[jhudsoncedaron]

So.. not so rare.

I went through all 11 pages of that and found only 1 instance: https://github.com/statiqdev/Statiq.Framework/blob/main/src/core/Statiq.Common/Execution/IExecutionContext.cs ; almost all of the matches are classes not interfaces.

So, to break it you need another derived interface with a default implementation. That could not have happened until default implementations were released, and the ability to default-implement other interfaces and have it work at load time feels taylor-made to fix this design flaw. Time to admit you blew it but good.

I'm going to be the most radical here. If it takes patching the loader to auto-resolve these is what it takes to get ICollection : IReadOnlyCollection and IList : IReadOnlyList then that's what it takes and it still should be done.

Every .NET Runtime release I end up having to deal with half a dozen breaking changes. What's one more?

The point of shipping the runtime with the applications that use it is we can take breaking changes. The equivalent of the what the bleep breaking changes from .NET Framework 2.0 to 3.5 or 4.0 to 4.5 can't happen because the applications get the version of the runtime they were shipped with. In this case, failures happen reliably early on testing of a build targeting a new runtime.

If you compile against netstandard packages, you won't see ICollection : IReadOnlyCollection at compile time and get compilation errors if you try to cast; however if you do something that truly brakes it it won't load on your own test platform, so the only issue there is if you assume the cast works you will get strange problems when passed a collection from application code written in .NET Framework that doesn't implement it.

[stephentoub]

Time to admit you blew it but good.

Excuse me? Please be respectful.

[Bip901]

Problematic wording aside, @jhudsoncedaron is correct: the regex matched 109 files, but they also need to be an interface/base class of another class and that other class needs to implement ICollection as well. Overall, very rare breaking cases.

[astrowalker]

It would be good to take historic view on this as well. There is some present codebase for sure and breaking change will have additional cost for it. But will new codebase be added or not? I guess it will, so every day of postponing fixing the issue (after all it should be done from the very beginning) is making all incoming codebase developers pay the price. It is another "make" history "I already have 3 users" only with turning point shifted forward.

[fitdev]

I hope it gets implemented ASAP, especially now that there have been DIMs for a few years.

[Daniel-Svensson]

@terrajobst @stephentoub

Would you consider making this change given that the type loader problem is solved ?

If so do you have any thoughts on what would be an acceptable solution (for example using som kind of internal mechanism such as a special attribute to give internal overloads lower priority)

[zms9110750]

If don't want to modify the code, how about adding a new namespace directly?

internal interface IReadOnlyCollection<out T> : IEnumerable<T>, IEnumerable
{
    int Count { get; }
    bool IsReadOnly { get; }
}

internal interface ICollection<T> : IReadOnlyCollection<T>
{
    void Add(T item);

    void Clear();

    bool Contains(T item);

    void CopyTo(T[] array, int arrayIndex);

    bool Remove(T item);
}

Compared to having a bool member in the Readonly collection, I am even more surprised why I am not allowed to use Add in an interface with an Add method.

In our language philosophy, interfaces only specify what the minimum can do. I haven't seen any definition and decided what an interface is not allowed to do.In terms of responsibility extension, read write is a perfect extension for read-only.

And, in our actual use, why do we give someone a read-only collection? Because I want him to see my changes and passively accept them when I make changes, but he can observe the changes.But today, either we use a specific type. Either create a new instance. The former does not conform to abstract programming, while the latter cannot meet the requirements.Compared to these two, I think the 'only' of readonly can be ignored.

[jhudsoncedaron]

@zms9110750 : The problem with this idea is the shear number of breaking changes required to switch almost everything over to the new interfaces. Same reason I didn't just make a parallel interface set.

[huoyaoyuan]

We should also include the non-generic interfaces (#91043) in discussion. Basically they are suffering the same risk of breaking.

[Daniel-Svensson]

Great to se this being reviewed for NET9.

I have a working example of solving the diamond problem making it possible to add DIM to already defined interface methods without making it a breaking change at https://github.com/dotnet/runtime/compare/main...Daniel-Svensson:runtime:default_interface_method_fallbacks

The POC reuse an existing attribute for simplicity, it should be replaced with something new instead.

The POC makes the following solution based on @MichalStrehovsky example run as expected (with current runtime it throws).

@terrajobst do you want to handle the runtime problem as part of this issue or as a separate issue or pr?

There are several possible such as a a new simplw atrribute (internal to runtime) , attribute with priority and/or maybe inferring some priority from where the DIM is defined (in assembly as interfacemethod overridden, in separate assembly "3rs party" , in same assembly as the concrete class). The POC is based on the first simple possible fix I could think of, where an attribute marks a method as "low priority"/fallback to use id no other implementation is found).

To keep being able to use the solution in future NET versions it might make sense to make it a non public type and maybe even enforce that it is only used in the same assembly as the interface method being implemented (to prevent other libraries for using it on runtime types).

[Daniel-Svensson]

@TylerBrinkley maybe you want to just add a short section to the issue mentioning that the diamond DIM problem need to be solved. You don't have to mention my POC or possible solutions to that problem, but it would be good to mention that it should be solvable and not block the suggestion.

[terrajobst]

I have created a repo trying to replicate the shared diamond problem and it doesn't seem to exist:

https://github.com/terrajobst/ReadOnlyCollectionInterfaces

I suggest we put this work on early for .NET 9.0 and see whether anything shows up. So far, it looks promising.