Add (Group)Join overload for working with IEnumerable<T>

[WhatzGames]

Good Day to you all.

While using the latest 6.0.1 Version of System.Linq.Async I noticed that there is no method overload for joining IEnumerables to an IAsyncEnumerable.

I propose to add the following overloads to the library in order to improve development experience working across asynchronous and synchronous data

public static partial class AsyncEnumerable{
    public static IAsyncEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner, TResult> resultSelector);
    public static IAsyncEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner, TResult> resultSelector, IEqualityComparer<TKey>? comparer);
+   public static IAsyncEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner, TResult> resultSelector);
+   public static IAsyncEnumerable<TResult> Join<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, TInner, TResult> resultSelector, IEqualityComparer<TKey>? comparer);

    public static IAsyncEnumerable<TResult> GroupJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, IAsyncEnumerable<TInner>, TResult> resultSelector);
    public static IAsyncEnumerable<TResult> GroupJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IAsyncEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, IAsyncEnumerable<TInner>, TResult> resultSelector, IEqualityComparer<TKey>? comparer)
+   public static IAsyncEnumerable<TResult> GroupJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, IEnumerable<TInner>, TResult> resultSelector);
+   public static IAsyncEnumerable<TResult> GroupJoin<TOuter, TInner, TKey, TResult>(this IAsyncEnumerable<TOuter> outer, IEnumerable<TInner> inner, Func<TOuter, TKey> outerKeySelector, Func<TInner, TKey> innerKeySelector, Func<TOuter, IEnumerable<TInner>, TResult> resultSelector, IEqualityComparer<TKey>? comparer);
}

Adding overloads to (Group)JoinAwait and (Group)JoinAwaitWithCancellation may have to get their corresponding overloads as well, but I'm not to sure how the Api should be shaped as an inner keyselector is possible for both Func<TInner, ValueTask<TKey>> or Func<TInner, TKey>.

[idg10]

I need to tread carefully with this.

Since taking over this project at the start of the year, our focus has been on Rx. The System.Linq.Async bits of Ix are not something I've delved into deeply, and this project also has a slightly different status: it gets treated as the de facto async counterpart of LINQ to Objects. So although it's not a Microsoft-funded project, a lot of people using it might be under the impression that it is. (And @davidfowl has suggested in the past that this might be code that could move into the .NET runtime class libraries precisely because it is very much something people are likely to assume is a .NET feature just like LINQ to Objects is.)

Since this would be the first time we've made any modifications to this code, and since people may have expectations of this code that are based on incorrect assumptions about how it's funded, I think this needs some careful thought before proceeding. Here are some things I want to consider:

• are there other examples in this library of hybrid IEnumerable<T>/IAsyncEnumerable<T> operators?
• are there similar hybrid examples in Rx that might be useful exemplars?
• does the fact that iterating through an IEnumerable<T> is a blocking operation risk rendering this non-useful? (Would we making promises of async behaviour that we can't actually fulfil? Will consumers of this new API be angry when they eventually discover that despite us returning an IAsyncEnumerable<T> we sometimes block their thread?) And if so is that only the case for certain implementation strategies?
• is the lack of symmetry a problem? (this makes a hybrid sync/async extension method available for IAsyncEnumerable<T>, but not the obvious equivalent one for starting with an IEnumerable<T> and joining it to an IAsyncEnumerable<T>) What would a symmetric solution look like?
• are we opening a combinatorial can of worms here by stepping into a hybrid world where we mix sequence types? (and even if the answer is "no" for this particular project, is this setting a precedent which would be impossible to follow in the Rx and AsyncRx projects?)
• are there other operators we should be doing this for (e.g., SelectMany)?

I'm not to sure how the Api should be shaped as an inner keyselector is possible for both Func<TInner, ValueTask> or Func<TInner, TKey>

AsyncRx.NET has a similar problem with its Defer operator. It needs to work with either. The solution currently in place there is that we have a DeferAsync that only accepts the ValueTask<T> form, but this is effectively just an alias for one of the Defer overloads. I don't know the history behind that design choice, but my guess is that it might be useful in situations where the compiler overload resolution would otherwise have been unable to resolve ambiguity.

That's an experimental project for now, so it's not proven that its solution to this problem is necessarily the right one. (For one thing, I don't think anyone was using it until very recently, because there was a bug that made DeferAsync unusable...) But it's one possible exemplar to consider when trying to work out the right shape for this.

[mwadams]

My main concern is that we understand what the semantics could/should be for this kind of "sync-under-async" pattern.

It feels like the "desired" behaviour might be quite use-case dependent and people's (false) assumptions might lead to significant issues.

However, I'd also recognize that this is a real world situation and strategies for dealing with it are important. Is it perhaps more like education materials (the "patterns and practices" examples we've discussed for RX as a whole) illustrating different approaches and their trade-offs?

[idg10]

It feels like the "desired" behaviour might be quite use-case dependent

This has made me think of an important point to add to my list:

• is this a problem that has to be solved in this particular library?

Since people might want different things in different scenarios, there might not be one right answer here, in which case this library is probably the wrong place to try to offer a solution to this.

Bear in mind the Rx had to invent a mechanism, (IScheduler, and a whole host of code associated with that type) to deal with what happens when worlds collide in this way. Its mechanism is opinionated, which is one reason the Rx libraries arguably belong in a separate library even though the core IObservable<T>/IObserver<T> types are in the .NET runtime libraries. IScheduler is definitely one way to solve those problems, but it's also definitely not the only reasonable solution. Reaqtor has its own slightly different solution to this exact problem, for example.

[WhatzGames]

just found #280 as stated, a (temporary?) solution for it might just be using ToAsyncEnumerable.