Variadic Generics

[avdosev]

lack of variadic generics (like variadic templates in C ++ / D / etc) provokes writing a lot of useless code.

example:

class Tuple2<T1, T2> {
// ...
}

class Tuple3<T1, T2, T3> {
// ...
}

like C++ solution:

template <class... U>
class Tuple {
// ...
}

I see two possible syntax options first:

class Tuple<...T> {
// ...
}

second:

class Tuple<T[]> {
// ...
}

the second option is better in my opinion

Here is a basic example of variadic generics usage:

class Tuple<T[]> {
  final T item;

  Tuple(...T this.item);

  List<dynamic> toList() => ...item;

  T[comptimeIndex] getItem<int comptimeIndex>() => item[comptimeIndex];

  dynamic getItem(int runtimeIndex) => item[runtimeIndex];

  Tuple<...T> withItem<int i>(T[i] item) => Tuple(...item.sublist(0, i), item, ...item.sublist(i+1));
}

// Usage

void main() {
  final tuple = Tuple(10, 'str', false);
  final tuple2 = tuple.withItem<1>('str2');
  print(tuple2.getItem<1>()); // output: 'str2'
}

[Levi-Lesches]

The whole point of a Tuple2 or Tuple3 is that it has fields for its items: myTuple.item2, myTuple.item3, etc. If you're just going to make a collection, use a list:

void main() {
  final List<dynamic> tuple = [10, "str", false];
  final List<dynamic> tuple2 = List.of(tuple);
  tuple2 [1] = "str2";
  print(tuple2 [1]);  // "str2"
}

I get what you're trying to do, but it seems like you're confusing runtime values with static analysis. In most cases of using Tuple, the values come in after the program has already compiled. There's no way Dart can do static analysis on that. The best it can do is dynamic runtime checks, which means you're better off using dynamic and regular parameters. Unless all your values are known ahead of time, this won't have the level of safety you're probably looking for.

[avdosev]

Although the values come in at runtime, the types are known even at compile-time, so we can infer the required types at compile time. The feature of variadic generics is that we work with them at compile time without losing type checks (unlike List in your example where we lose all type checks). Variadic generics will simplify the development of libraries by eliminating duplication of code or code generation. For example, in the reference implementation Tuple, developers must support 7 versions of the same class. This problem exists not only in the tuple implementation, there are many ideas that would be more convenient to implement using Variadic Generics

[Levi-Lesches]

Okay, I get what you're saying. I guess the syntax you used confused me and I didn't understand your example, but I do recall seeing this pattern in package:provider and wondering why there isn't a better solution. Upvoted.

[lrhn]

As I read the example,

class Tuple<T[]> {
  final T item;

declares a type parameter which is an arbitrary tuple type, and then a field with that tuple type. Which basically means that Tuple is a slightly confusing example because the language must already has native "tuples" for this code to be possible, and the object wrapper then seems unnecessary.

It's interesting that you need compile time integers too for this to really work (the tuple arities). I can see how it makes more sense for something like C++ templates than the current Dart.

[Levi-Lesches]

Which basically means that Tuple is a slightly confusing example because the language must already has native "tuples" for this code to be possible, and the object wrapper then seems unnecessary.

Consider instead package:provider, which defines:

• ChangeNotifierProxyProvider0 through ChangeNotifierProxyProvider6
• Consumer through Consumer6
• ListenableProxyProvider0 through ListenableProxyProvider6
• ProxyProvider0 through ProxyProvider6
• Selector0 through Selector6
• ProxyProviderBuilder0 through ProxyProviderBuilder6

I assume 6 is just arbitrary here. I also don't really know how often these variants are used (I never use more than one), but it looks like a prime candidate for this type of syntax.

That being said, since the number of type arguments is known at compile-time, it's not that hard to incorporate meta-programming into this:

@MultipleGenerics(6)
class Tuple<T> { /* ... */ }

// generates: 
class Tuple2<T1, T2> { /* ... */ }
class Tuple3<T1, T2, T3> { /* ... */ }
class Tuple4<T1, T2, T3, T4> { /* ... */ }
class Tuple5<T1, T2, T3, T4, T5> { /* ... */ }
class Tuple6<T1, T2, T3, T4, T5, T6> { /* ... */ }

[mateusfccp]

@Levi-Lesches said:

That being said, since the number of type arguments is known at compile-time, it's not that hard to incorporate meta-programming into this:

@MultipleGenerics(6)
class Tuple<T> { /* ... */ }

// generates: 
class Tuple2<T1, T2> { /* ... */ }
class Tuple3<T1, T2, T3> { /* ... */ }
class Tuple4<T1, T2, T3, T4> { /* ... */ }
class Tuple5<T1, T2, T3, T4, T5> { /* ... */ }
class Tuple6<T1, T2, T3, T4, T5, T6> { /* ... */ }

Meta-programming, in this case, may solve the maintainer issue by easing the maintainability cost. However, the API for the consumer will still be bloated and ugly.

[Levi-Lesches]

True, and I fully agree that there should be a cleaner way to do this. I just brought up the metaprogramming example since @lrhn said this would need to wait for some other language features first.

[avdosev]

@lrhn, Yes, my example can't work without compile-time integers. Maybe we can achieve them without breaking the Dart style. Zig has comptime arguments: https://ziglang.org/documentation/master/#Compile-Time-Variables

fn performFn(comptime prefix_char: u8, start_value: i32) i32 {
  // ...
}

If we rename comptime to const (while keeping its semantics) and add it to Dart, we'll get:

class Tuple<T[]> {
  final T item;

  Tuple(...T this.item);

  List<dynamic> toList() => ...item;

  T[comptimeIndex] getItem(const int comptimeIndex) => item[comptimeIndex];

  dynamic getItem(int runtimeIndex) => item[runtimeIndex];

  Tuple<...T> withItem(const int i, T[i] item) => Tuple(...item.sublist(0, i), item, ...item.sublist(i+1));
}

So we will not go too far into C++ syntax.

[mateusfccp]

I would love to have this, as well as things like inlined loops. I though we would have this solved with static meta programming, but the approach elected don't deal with expression expansion...

For example, today I can't do something like this, even though all elements here are static and only depends on compile-time data:

  static const __environments = {
    for (int i = 1; i < 20; i ++)
      'uat${_getServerNumber(i)}': EnvironmentConfig.uat(i), // Here, `uat` is a static method that could be inlined
    'prod_beta': EnvironmentConfig.prod_beta,
    'stress_test': EnvironmentConfig.stress_test,
  };

 static String _getServerNumber(int number) => '${number + 1}'.padLeft(2, '0');

_{Source: adapted from a real code from an app from my company.}

So I have to change it to be static final instead or manually/explicitly type all uat items ("uat01" ... "uat20").

[Levi-Lesches]

Yes, my example can't work without compile-time integers

Perhaps #1684 can solve that issue, and @lrhn specifically mentions this in his comment:

Can you abstract over const in generics?

typedef F<T> = int Function(T); 
// ... 
F<const int> f = ...;

If not, why not? (Because it's not a type, but then you can't abstract over all unary functions any more using:

typedef F<R,P> = R Function(P);

).

Maybe this is why const parameters would be useful. Although, again, another dependency sadly means this will be even further down the todo list.

[pchasco]

C# has extended its grammar to support tuples as a special case. This support in the grammar provides an elegant workaround to most of the cases where a variadic generic might be desired.

The C# implementation of the tuple is not implemented with a variadic generic. During compilation, C# will transform a tuple literal expression into the appropriate version of tuple.

// This code creates a Tuple2<string, int>
(string, int) t = ("Count: ", 1);

// Equivalent to the following tuple
Tuple2<string, int> t = new Tuple2("Count: ", 1);

One interesting application for a tuple literal would be when dealing with streams. In TypeScript, one can write the following and it will be entirely type safe (as much as a language built on JavaScript can be):

Observable<int> intObservable;
Observable<string> stringObservable;

combineLatest([
    intObservable, 
    stringObservable
])
    .subscribe(([intValue, stringValue]) => console.log(`${stringValue} $(intValue.toString()}`);

We can achieve something similar already with Dart and the Tuple package, but it isn't quite as intuitive as TypeScript or C#'s implementations. The primary drawback is that the developer cannot easily change the number or items in the tuple without being required to change the tuple type to match its arity, and the lack of descructuring requires the developer to use the itemX properties rather than more meaningful identifiers.

Stream<Tuple2<T1, T2>> combineLatest<T1, T2>(Tuple2<Stream<T1>, Stream<T2>> streams) {
  // Stub
  return Stream.empty();
}

void main() {
  final Stream<String> stringStream = Stream.empty();
  final Stream<int> intStream = Stream.empty();
  combineLatest(Tuple2(stringStream, intStream))
    .listen((streams) {
        print("${streams.item1}: ${streams.item2}");
    });
}

[avdosev]

With the upcoming release of Dart 3.0 and the introduction of Records, the Dart programming language has taken a significant step forward. It is worth mentioning, however, that while Records offer a new approach to handling tuples in Dart, the issue of generic variables remains unresolved. It would be interesting to find out how close we are to implementing Variadic generics.

[munificent]

Record types do help, yes. But there is still a lot of other language features we'd need to add in order to have useful variadic generics. We'd need (at the least, and off the top of my head):

• A way to spread a variadic type parameter in a type annotation in a place where a list of types is expected. That means, at least, inside a type argument list and inside a record type. Possibly inside a function type's parameter list.
• A way to spread a variadic type parameter in a function parameter list. This in turn suggests that variadic parameters would require the compiler to specialize the function for each instantiation since now the signature would be different for each arity. Or maybe we just require arguments to always be passed as an explicit record? That would probably be really annoying for users.
• A way to capture a list of variadic function parameters into a single value. We could probably build this on records.

Probably other stuff I'm forgetting. There's a lot of machinery to make this work in C++, and a lot of that machinery presumes C++'s compilation model where templates are specialized at compile time.

[lrhn]

Variadic type arguments only makes sense when combined with something that applies those types to values of a similar structure.

So let's just say that there are no variadic type arguments, only abstraction over record types.

A type parameter of the form ... R must have a record type (not Record itself, so it's not just R extends Record). Then it's possible to accept values of that type, and to spread those into other records, which has a type which is also found by spreading the types. Say:

(...R1, ...R2) concat<...R1, ...R2>(R1 r1, R2 r2) => (...r1, ...r2);

This concatenates two records into one, with abstraction over the record structure. (We have to define what happens if both types has a member with the same name. Let's say the last one wins and the first one's value is ignored.) Then you can write:

var all = concat((1, "a"), (true, null)); // (1, "a", true, null)  : (int, String, bool, Null)

That's potentially useful, but you can't do anything with the record other than spreading it, because you don't actually know anything about its shape.

Consider now the ability to map things onto every field. Let's say:

R::map((X) -> X?)  // for types
  nullIfEmpty<...R>(R values) => values::map<T>((T x) => (x is String && x.isEmpty || x is Iterable && x.isEmpty) ? null : x);

The ::map applies an operation to every field of the record or record type. For values, the argument is a normal function. For types, it's a type function, which must be a constant-computable and apply to every type. The compiler can then check that for all types <T>(T x) => ... ? null : x will have type T?, which is the type function (X) -> X? applied to T, so the static types match up. (Then we can start adding ways to restrict the individual types, so you can have a record of types that are all Futures, but not necessarily of the same thing.)

(It's not going to be easy to make that all work. And this is fairly low-tech type abstraction.)

[mrwonko]

For an example use case in the standard library, you can look to FutureRecord2, FutureRecord3, FutureRecord4 etc.

A good variadic generic implementation would make it possible to have only one definition that supports records of an arbitrary number of fields.