Support declaration of generic type for methods

[alxhub]

Dart should support generic methods/functions, similar to Java:

class Foo<E> {   E _item;

  / ... /

  <T> T transform(T fn(E)) {     return fn(_item);   } }

Currently this is supported only by omitting the type information. Additionally, some core Dart code includes the <T> annotation within comments: /<T>/. In order for proper tool/IDE support, Dart should support explicitly declaring generic types for a method.

[DartBot]

This comment was originally written by drfibonacci@google.com

---

Removed Type-Defect label. Added Type-Enhancement, Area-Language, Triaged labels.

[gbracha]

We've tried to keep things very simple. but this is obviously a possible enahncement.

---

Set owner to @gbracha. Added Accepted label.

[anders-sandholm]

Added apr30-triage label.

[anders-sandholm]

Removed apr30-triage label.

[anders-sandholm]

Added triage1 label.

[anders-sandholm]

Added this to the Later milestone. Removed triage1 label.

[peter-ahe-google]

I think type inference and generic functions go hand in hand.

Since I would prefer not having type inference in Dart, I would prefer to not have generic functions.

[DartBot]

This comment was originally written by vva...@gmail.com

---

I want to write top-level function with generics.

for example, I create this top-level function. Expection<T> expect(T obj){   return new _ExpectionImpl.expect(obj); }

use this function invalid case. and raise a type mismatch error on IDE. expect("foobar").toBe(1);                                  ↑ warning. message "int is not assignable to String"

(google group log) https://groups.google.com/a/dartlang.org/d/topic/misc/6eTB2Vrrwh0/discussion

[DartBot]

This comment was originally written by Konstantin.Solo...@gmail.com

---

This would be especially helpful in Futures API. There are transform methods which would be improved if they had generics.

[gbracha]

Issue #7099 has been merged into this issue.

[gbracha]

Issue #11033 has been merged into this issue.

[justinfagnani]

Issue #11033 has been merged into this issue.

[lrhn]

Issue #11689 has been merged into this issue.

[lrhn]

I could easily live with generic methods without type inference. As long as you don't specify the type, it'll just default to "dynamic" anyway, but it would still improve documentation, and you can use it if you want. It's better than just using dynamic, which is the other option.

 T id<T>(T value) => value;

 int x = id<int>("not an int"); // Static type warning! Checked mode error!

So, can we have it, please?

[polux]

Same here. It would improve documentation and completion so much... (plus it would help the editor's inference for providing completion even when you don't specify the type explicitly).

[munificent]

It seems to be tribal knowledge among the language designers that "specifying type inference is bad". Has anyone spelled out why this is bad? I don't seem to recall ever seeing an explanation here. The trend for other statically-typed OOP languages (Java, C#, Scala, TypeScript, et. al.) is towards (local, bottom-up) type inference, so I'm interested to know the rationale behind our desire to go the other direction.

[polux]

I don't know what the exact motivations are but I can see one reason why it would be bad: type inference in presence of subtyping is hard (or sometimes impossible). So inference would have to sometimes bail out (as does javac with generic methods).

If it bails out by issuing an error message that's not dart-ey. If it silently bails out and passes dynamic instead, the the meaning of a program with an is-check would depends on whether type inference succeeds or not. That sound pretty bad.

Even if there wasn't the bailout issue, using type inference for generics means you trust type annotations (for performing inference) to give a meaning to the program (because of is-checks). So that alone violates dart principle of type annotations not affecting a program's semantics.

[DartBot]

This comment was originally written by @simonpai

---

The motivation to have method scope generic type parameter is exactly the motivation to have class scope generic type. Of course adding a feature will add complexity to a language and its implementation, but the arguments about challenges brought by type inference also applies to class scope generic type as well. If it's really that bad, why doesn't Dart just remove all generics features anyway?

[justinfagnani]

Simon, the difference between generic constructors and generic methods is that people generally expect that you have to specify the type parameters on a constructor and they don't expect to always have to specify the type parameters on a generic method, thus more pressure a type inference.

[lrhn]

Closed by mistake.

[jodinathan]

Hi. Is this ever going to happen?

[munificent]

Yes, it's in progress.

[zoechi]

@jodinathan that's working since quite some time already by adding generic type parameters in comments and is supported by dartanalyzer

dynamic /*=T*/ someFunc/*<T,U>*/(dynamic /*=U*/ p1) {
  return someCalc/*<MyType,T>*/() as dynamic /*=T*/;
}

where dynamic is a type that works without using generic type parameters (runtime) and /*=T*/ is the type the analyzer uses to check for type conflicts instead.

There are some issues here (some might be closed) that discuss more details that should help to figure out how to use it. There seems to be a way to use it without the comments, perhaps a flag, but I don't know details. Just saw it mentioned in some comments.

[jodinathan]

@zoechi, thanks for the response, however, it seems a bit too hacky for me. We are still deciding if we will use dart or not in a big project we are projecting. It is important to know where dart is going, and generics is so basic with modern languages that we really expect dart to have it soon. We were thrilled when studying dart couple days ago, but now there are some down lows that we didn't expect.

[simonpai]

@munificent, any information on its schedule? Just being curious.

[zoechi]

@jodinathan I wasn't aware you are new to Dart. In this case I can understand :D I'm using it since several months (with comment syntax) and it's working great but it's not yet released.

What about discussing your issues in https://groups.google.com/a/dartlang.org/forum/#!forum/misc ?

[astashov]

Last time I tried (a couple months ago), generic methods worked for me in dart2js, in Atom with Dart plugin (but IDEA with Dart plugin still was giving me errors), analyzer supported them, but DartVM still didn't. So, worked everywhere except in DartVM. I could do things like:

T id<T>(T arg) {
  return arg;
}

To enable it for analyzer, so Atom wouldn't highlight them as syntax errors, add to your .analysis_options file:

analyzer:
  language:
    enableGenericMethods: true

And to use them with dart2js, use it with --generic-method-syntax flag, like: dart2js --generic-method-syntax bin/main.dart

[eernstg]

Right, Anton just described the options you need to use, so you can use the regular syntax (not the one using magic comments). As long as you do not run in checked mode, you should also be able to use --generic-method-syntax with the virtual machine dart.

However, you should note that the implementations in dart2js and in dart will give you the ability to parse introduction and usage of method type parameters and passing of actual method type arguments, but the runtime semantics associated with these constructs are incomplete:

In dart2js, actual type arguments are parsed but then ignored (nothing is passed at runtime, bounds are not checked), and usages of formal type parameters in the body of a generic function or method are replaced by dynamic or a malformed type (so if T is a method type parameter then T t = 42 will not fail, even in checked mode, no matter whether T == int; similarly List<T> will be a List<dynamic>; and 42 is T is a runtime error whereas <int>[] is List<T> is true). In dart you get a similar treatment in production mode, but it fails with 'malformed type' in checked mode as soon as you encounter a check that involves a formal type parameter (I suspect this will be changed, because it stops almost all programs using generic functions immediately; but for now you can just use production mode).

This means that you can get the static checks (from the analyzer) and the "erased" semantics, which may or may not suffice for your purposes. In Java, for instance, type arguments are always erased, so it might sound fine at first to have this level of support; but as soon as you do things like return new C<T>(..) where T is a method type argument, Dart will reify T in the new object, so you will be able to detect that you got a C<dynamic>, not a C<T>.

And, as Günter mentioned, it's not yet released. ;-)

[munificent]

@munificent, any information on its schedule? Just being curious.

Sorry, no. In general we don't tend to commit to schedules for things since we tend to be pretty fluid about organizing our work and we don't want to set expectations and then annoy people when schedules change.

[sethladd]

Dart 1.21 now has generic methods. (There's an informal specification linked from https://github.com/dart-lang/sdk/blob/master/CHANGELOG.md)

Should we close this?

[munificent]

Let's leave it open until the runtime support is there in dart2js and the VM too.

[sethladd]

Is there a meta bug that is tracking implementation scope?

What does "runtime support in VM" mean? I thought the VM supports generic methods?

[jmesserly]

What does "runtime support in VM" mean? I thought the VM supports generic methods?

it means reified types:

class C<T> { get t => T; }
f<T>() => T;
main() {
  print(new C<int>().t); // int
  print(f<int>()); // dynamic, but should be int
}

[munificent]

What does "runtime support in VM" mean? I thought the VM supports generic methods?

It parses the syntax but treats all type arguments as dynamic. This does not do anything useful in the VM right now:

isOfType<T>(Object obj) {
  print(obj is T);
}

Is there a meta bug that is tracking implementation scope?

I couldn't find one. I emailed the language team to see what the plan for that is.

[sethladd]

Ah, gotcha. Thanks for the clarification.

[brad-jones]

Any update on this? Is the plan to eventually fully support reified generics on methods?

[lrhn]

The plan is to fully support reified generics on methods in strong mode and in Dart 2 (which will be similar to strong mode). I'm not absolutely sure where strong mode is right now - it already supported parameters written in comments, and I think it works correctly with the real syntax too.

We support the syntax, but ignore the actual parameters, in Dart 1 implementations to make the transition easier.

/L

On Tue, Jan 3, 2017 at 7:10 AM, Brad Jones notifications@github.com wrote:

Any update on this? Is the plan to eventually fully support reified generics on methods?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/dart-lang/sdk/issues/254#issuecomment-270056514, or mute the thread https://github.com/notifications/unsubscribe-auth/AEo9B_7w8Nq9d8nIfr88ubTDZ5c746LFks5rOeZSgaJpZM4E39PU .

-- Lasse R.H. Nielsen - lrn@google.com 'Faith without judgement merely degrades the spirit divine' Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K

• Denmark - CVR nr. 28 86 69 84

[ScottPierce]

@lrhn This is the first I'm hearing of Dart 2. Is there any place I can get some information on it?

[brad-jones]

Maybe @Irhn is referring to V1.22 ???

[lrhn]

On Tue, Jan 3, 2017 at 7:42 AM, Scott Pierce notifications@github.com wrote:

@lrhn https://github.com/lrhn This is the first I'm hearing of Dart 2. Is there any place I can get some information on it?

Currently Strong Mode is not really officially Dart (it's not in the spec), but it is quite successful, so eventually we will nail it down and make it officially official. We're reserving the version number 2.0 for that time :)

Adding the syntax for generic methods (which was released in v.1.21), without actually implementing them except in strong mode, is a step towards supporting strong mode.

/L 'When its ready!' -- Lasse R.H. Nielsen - lrn@google.com 'Faith without judgement merely degrades the spirit divine' Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K

• Denmark - CVR nr. 28 86 69 84

[jodinathan]

So is it safe to say that if you want to catch up with Dart 2 it is recommended for you to already use strong mode once you can?

@lrhn https://github.com/lrhn This is the first I'm hearing of Dart 2. Is there any place I can get some information on it?

Currently Strong Mode is not really officially Dart (it's not in the spec), but it is quite successful, so eventually we will nail it down and make it officially official. We're reserving the version number 2.0 for that time :)

Adding the syntax for generic methods (which was released in v.1.21), without actually implementing them except in strong mode, is a step towards supporting strong mode.

-- Jonathan Rezende

[munificent]

So is it safe to say that if you want to catch up with Dart 2 it is recommended for you to already use strong mode once you can?

Yes, definitely. :)

[jodinathan]

any news on this? generic method with dynamic as the type is not very useful

[eernstg]

The limited level of support for generic methods that we call generic method syntax is what we plan to support for Dart 1.x. For full support of generic functions you'd need to switch to strong mode, including the strong mode run-time semantics, or wait for Dart 2.0 (and strong mode is essentially a preview of Dart 2.0).

Note, however, that a language like Java has had generic methods for many years at this level (because type arguments are never reified in Java), and it's not useless:

The static checks that you get with strong mode static analysis (e.g., dartanalyzer --strong ...) will reveal any inconsistencies in your invocations of generic functions/methods, and in the usage of any returned results. However, we can do things like e is T or e is List<T> in Dart, and that won't return the correct result when T is a formal type parameter of an enclosing function/method, unless and until the actual type arguments of invocations get reified. In Java you also wouldn't get the correct result, you'd get an "unchecked cast" warning, and the non-throwing evaluation of (T)e wouldn't provide any guarantees at all about the actual type of e. So in that sense it's "no worse than Java", but, of course, we want fully reified type arguments for Dart. It just won't happen in Dart 1.x.

Edit Aug 29, 2017: Updated link to current version of informal spec of generic method syntax.

[jodinathan]

@eernstg, can I have full generics with strong mode? Because I am using it with "analyzer: strong-mode: true" and T is still dynamic.

About Java... Personally, I try to compare Dart to more modern language like C#.

[lrhn]

You have full generics with strong mode, but the only compiler that can compile strong mode is DDC.

If you run with the VM or compile with dart2js, then you will get non-strong mode semantics and no reified method type parameters.

/L

On Wed, Apr 5, 2017 at 4:17 PM, jodinathan notifications@github.com wrote:

@eernstg https://github.com/eernstg, can I have full generics with strong mode? Because I am using it with "analyzer: strong-mode: true" and T is still dynamic.

About Java... Personally, I try to compare Dart to more modern language like C#.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/dart-lang/sdk/issues/254#issuecomment-291875557, or mute the thread https://github.com/notifications/unsubscribe-auth/AEo9B2ZG65JJFUvgeqrdx7YZCvYHm8Skks5rs6KDgaJpZM4E39PU .

-- Lasse R.H. Nielsen - lrn@google.com 'Faith without judgement merely degrades the spirit divine' Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K

• Denmark - CVR nr. 28 86 69 84

[eernstg]

Right, and I'll make Lasse's explanation a bit more explicit, because we need to make some distinctions that may not be entirely obvious.

With analyzer: strong-mode: true you'll get the strong mode static analysis; here, a formal type parameter of an enclosing generic function/method is not the same thing as dynamic, it is treated like other type arguments (so it may be known to be a subtype of a given bound, and it may have subtype relationships with other type parameters, etc). Similarly, for actual type arguments at call sites you'd get checks based on the declared bounds of the type parameters. All quite different from dynamic.

DDC will generate code that obeys the strong mode run-time semantics; e.g., it will consider a List<dynamic> to have type List<Object> but not List<int>, and it will consider int foo(Object x); to have type Object Function(int) but not int Function(int), because function type subtyping uses contravariance for parameter types. And it reifies function type parameters.

The situation where you use strong mode static analysis and then run your program using Dart 1.x semantics may seem to be a crazy hybrid that you should never even get close enough to shake a stick at. However, the situation isn't actually so bad.

There are lots of situations where strong mode static analysis will reject a program as erroneous, and Dart 1.x static analysis just emits a warning. Given that Dart 1.x warnings generally mean "this is an error, but we can fix it for you" (so you should never ignore them), this is a matter of workflow and not really language semantics. The function type subtype rules in Dart 1.x are very, very forgiving, and basically the function type subtypes in strong mode are just picking the subset of cases that will actually work. This will allow your strong-mode-checked program to continue to run in some situations where it is passing around "a bad function" (say, because an as expression succeeded, but with strong mode semantics it would have failed), but it shouldn't hurt you when running a "good" program. Finally, dynamic doesn't get to play the role as a bottom type ("the almighty type that can do all things"), and in particular you won't be able, with strong mode dynamic semantics, to pass a List<dynamic> where a List<int> is expected, and if you're actually running with Dart 1.x semantics then this won't be detected (but you may have failures, e.g., failing downcasts, later on when you are using the elements in that list).

When it comes to generic methods/functions, your strong-mode-checked program will use the value dynamic in the cases where static analysis expected the actual type argument to be available, and this may make casts succeed where they should have failed (e as T), and it may insert dynamic into newly created instances (return <T>[]; ), and downcasts may succeed where they should have failed (T x = e;), but otherwise the type parameters actually don't matter at run-time.

The List<dynamic> "will work" for all purposes except for a direct type test (it will return true for myList is List<int> in Dart 1.x), and all the other differences are strongly biased towards errors: As long as your program is "working correctly", you won't see any differences.

With this in mind, we think that it does make sense to run programs in Dart 1.x mode after checking them with strong-mode static analysis.

That said, it will be a relief and an improvement when we get strong mode semantics everywhere, too.

About Java and C#: It isn't necessarily fair to consider reification of type arguments as a modern trait, and erasure as old-fashioned. In the functional community it has been considered as an important property that polymorphic types can be handled in a parametric manner, which basically means that it must be completely impossible for the implementation of any given polymorphic programming abstraction (say, a generic function) to depend on the actual type arguments. In return for this particular restriction, we get theorems for free.

This may indeed be helpful for developers when they are reasoning about their software (e.g., about whether a particular modification preserves the meaning of the program as a whole), but it tends to conflict with another very deep force: Object orientation relies on the ability of objects to report on their type ("I'm a String!", "I'm a BankAccount!"), because this is required in order to allow method invocation to be late-bound (such that we can call the correct method implementation for the given object, which is not known statically).

An obviously correct treatment of type parameters in a strictly parametric setup is to erase them. When they've been checked (at compile-time), we don't need them any more.

Reification of type arguments is not something that was invented along with C#, it was part of BETA already in the 1970'ies (in the shape of virtual patterns, which are actually more powerful than type arguments as we know them today).

People will be quick to claim that Java uses erasure because it was impossible to reconcile reified type arguments with the installed base of Java software, and that may indeed be true, but I also think there is a connection to this ancient war between the "theorems for free" crowd and the "OO" crowd. In C#, they made a better choice at some point, at least as seen from the OO side.

In any case, we will have reified generics in Dart, and you can get the full package with strong mode and DDC, and strong-mode-checks-plus-Dart-1.x-semantics as a meaningful stepping stone.

Hope you can live with that for the limited amount of time where it's needed. ;-)

(About the notation: T Function(S) is the new function type syntax. Functional languages commonly use S -> T for the same thing.)

[zoechi]

Can this be closed?

[jmesserly]

Can this be closed?

yup! this was fixed in https://github.com/dart-lang/sdk/issues/27501