Add generic parameters to allow strongly typing for collections

[Shadowblitz16]

Describe the project you are working on: A space ship battle game

Describe the problem or limitation you are having in your project: Just the general lack of static typing features in the gdscript language for some things

Describe the feature / enhancement and how it helps to overcome the problem or limitation: I was wondering if gdscript could get generics simular to C#..

it would allow us to make classes the require a type to be passed like so.. func foo<T>(t:T)

they could also get constraints to allow static typing func foo<T>(t:T) T : int, CustomClass

this could be used to do things like so..

#only accepts numbers
var veci := Vector2<int>()
var veci : Vector2<int>
var arr  := Array<MyObject>()
var arr  : Array<MyObject>

things like so wouldn't be allowed..

var veci = Vector2<int>()

var arr  = Array<MyObject>()

but things like so would..

var veci = Vector2()

var arr  = Array()

Things to note..

• generic functions use <> to remove confusion between () which is used for calling a function
• constraints can be classes, any godot object, or any hint

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams: see above

If this enhancement will not be used often, can it be worked around with a few lines of script?: this would be used whenever someone wants static typing for a collection like a array or dictionary or numeric struct like a vector it can be worked around but it takes a bunch of type checking for everything you do and this easily because unmanageable and ugly

Is there a reason why this should be core and not an add-on in the asset library?: its another static typing feature which I think is good

[jonbonazza]

I agree this is useful for containers like Array and Dictionary, however I don't think it makes sense for Vector classes. The only two real possible types for Vector components are int or float and there is really no good reason that you cant just assign an int as a float. If you really do want to differentiate here, I'd prefer just having a second explicit type for ints, like Vector2i.

That said, I believe I read somewhere that typing for Arrays and Dictionaries was coming in gdscript 2.0 (though I don't know if it will take the form of generics or not).

[Shadowblitz16]

@jonbonazza The issue is I would like to be able to create my own generic classes and this is a good way to do it

[jonbonazza]

Unfortunately, generics are not possible to implement in a dynamically typed language. They don't really make sense either, as variables don't have an underlying type. Since types are optional in gdscript, generics are simply not something that can be added due to technical limitations.

[jonbonazza]

That said, you can utilize dynamic typing to get the same effect.

For instance:

class Foo:
    var bar
    func _init(bar):
        self.bar = ba

####

var foo := Foo.new(2)
var foo2 := Foo.new("baz")

In this example, foo is a int-based Foo, while foo2 is a string-based Foo

[Tadaboody]

Unfortunately, generics are not possible to implement in a dynamically typed language. They don't really make sense either, as variables don't have an underlying type. Since types are optional in gdscript, generics are simply not something that can be added due to technical limitations.

While it's true that it won't be enforced language-wise, it's still possible to add it to the annotation system (see python as an example).

One use-case that I keep getting is in exports, right now there are two different syntax options for exports

# Type annotation
export var color: Color
# `export` function, this case is not possible without generics
export(Array, Color) var colors

Adding generics to the annotation system would allow always using the type annotation syntax which will eliminate export being a special case

[Calinou]

@Tadaboody You should be able to export PoolColorArray (PackedColorArray in 4.0) for this particular use case.

[Tadaboody]

@Tadaboody You should be able to export PoolColorArray (PackedColorArray in 4.0) for this particular use case.

@Calinou And for the case of Array[int]? Array[vector2d]? It would be much easier to use if there were one (hopefully obvious) solution

[Calinou]

@Tadaboody There's PoolIntArray and PoolVector2Array too.

A typed "generic" array will be different from a pooled array (these were replaced with packed arrays in 4.0). Pooled arrays are mainly suited for storing large amounts of elements (thousands of them), but you can also use them for smaller arrays to get type safety. In contrast, a generic array will always hold a Variant which uses more memory. This will probably be the case even if type hints are used.

Still, GDScript will most likely feature typing for "generic" arrays in 4.0. This is mainly useful for typing arrays with user-made types.

See also the core refactoring progress report #1.

[Shadowblitz16]

so what about generic pool_arrays? I think all collections should support generics. even dictionary...

var dict := Dictionary<String,int>()

the main issue is not just lack of generics but lack of support for static typing in alot of things I actually have a issue about statically typed PackedScenes

[jonbonazza]

Static typing is just optional hints in gdscript. It never will be more than that, so you never will have all the luxuries a true statically typed language will afford.

[jonbonazza]

That said, dictionaries could support gebrics since tgey are implemented in the core, and i would agree that should be considered for gdscript 2.0, however custom type generics in gdscript simply don't make sense, so don't count on them being implement.

[Shadowblitz16]

@jonbonazza I know gdscript will never be a statically typed language but having the option is a good thing static typing ensures that your accessing the right data and allows the editor to tell you when your not. it also is good for things like the godot inspector since the godot inspector needs to know what type its serializing and what types it accepts

limiting the user to what they can do in gdscript is never a good thing. it makes it harder to acheave task the user might want to do

EDIT: btw this is what I think one of godot's biggest flaws are is the lack of support for static typing. you can basically pass anything to the inspector and it will take it. which is plain useless. so yes generics do make sense and this is why static typing is always better you have something like a inspector

[Shadowblitz16]

I just want to mention if this gets added I REALLY think it should be with <> instead of ()

[Gnumaru]

I would like to advocate in favor of generic syntax for user scripts, that is, generics implemented with type erasure, like in java and typescript, where the generic type info is lost at runtime.

See typescript: since it became such a popular language, the benefits of compile-time/parse-time static analysis of dynamic typed languages cannot be denied. I admit that in the specific case of typescript generics is such a small part of the whole package that it would still be a great language even if it wouldn't have generics at all.

But the thing is that the programming community has been taking great advantage of generic syntax since C++98, and again with java 1.5 in 2004 and again in many other languages that support generic syntax, be it as a code generation template processing engine as in C++ or only as a compile time type check in java or with full runtime with generic type introspection like in .net.

Even though runtimes like .net provide runtime type safety for generics, I argue that the runtime safety is less important than the compile-time/parse-time type safety given by the syntax alone.

Letting aside container types, with generics in gdscript we would have increased type-safety for custom user types. For example, I could model a generic weapon class where the generic type can be any ammo class, or a generic powerup class where the generic type is any behavior class that can benefit from that powerup.

Even though we could perfectly model and implement such examples without generics at all, the point is that with generic syntax we get compile-time type safety, which is the whole point. The type hints introduced in 3.1 are the same: even if there would never be runtime type safety, the bulk of the benefit for the developer is the parse-time errors he gets while developing, because if he just want runtime errors, he can already assert() conditions of arbitrary complexity, not only just type checking.

[coelhucas]

So, I really wanted to add here because I think that one of the greatest features of generics wasn't truly addressed here. Recently I really felt the need of using it for better developer experience.

To illustrate it, I'll use a generic "component system", but I can see this working for many other cases such as State machines. I'll try to illustrate this in multiple ways without being too specific:

# First scenario: how it would be done at the current state of the language
(get_component(StatusComponent) as StatusComponent).physical_damage
(get_component(HealthComponent) as HealthComponent).value

(target.get_component(HealthComponent) as HealthComponent).apply_damage(
  (get_component(StatusComponent) as StatusComponent).physical_damage
)

# Second scenario: using return type generics
func get_component<C>() -> C:
  for component_node in components:
    if component_node is C:
      return component_node

  return null

get_component<StatusComponent>().physical_damage
target.get_component<HealthComponent>().apply_damage(
  get_component<StatusComponent>().physical_damage
)

I made a point of explicitly showing how get_component would be implemented, because as is right now, we don't have any proper way of doing such a thing with such easiness. If you want something like that without something such as generic types, you'd have to:

# Component.gd
# [...]

# This would be needed for each subclass of it
static func get_component_name() -> String:
  return "Component"

# ComponentsManager.gd
# [...]

func get_component(_component: GDScript) -> Component:
  for component_node in components:
    if component_node.get_component_name() === _component.get_component_name():
      return component_node

  return null

I not only have the function, as now I need a static method on the base class that I will be overwritten every time I make a new component (as I said, this is just a generic example). And the language right now won't even be able to predict me what return I'll get when consuming it without explicitly casting it (which is bad since I could also make a wrong cast and I won't know until I run the project)

[julian-a-avar-c]

What about variance? Unless we figure out that can of worms, I don't see a reliable output out of this issue.

Now... we don't even have generics, so why worry, because, it will eventually be a problem for someone somewhere, so it's best to discuss it.

# I know a lot of you want `House<T>`, but we already have `Array[T]`.
# No need to change what ain't broken.
# Since we don't have class generics, let's use arrays for now, since those work similarly.
# Every time you see `Array`, imagine `House`)
# class House[T]: var data: T

class Animal:
    var name: String
    func _init(_name: String):
        self.name = _name
class Dog extends Animal: pass
class Cat extends Animal: pass

# Given a cat, give me their residency
func where_is(cat: Cat) -> Array[Cat]: return [cat]

func _ready():
    # This is fine of course...
    var favorite_pet : Animal = Cat.new("Felix")

    # Looking good...
    var cat_house : Array[Cat] = where_is(favorite_pet)

    # This should not compile.
    # We currently get a runtime error, which is better than nothing?
    var animal_house: Array[Animal] = cat_house as Array[Animal]
    var animal: Animal = animal_house[0]
    # But why? It doesn't look dangerous?

    # Dogs can go in animal houses...
    animal_house[0] = Dog.new("Fido")
    print(animal_house.map(func(animal): return animal.name))
    # ... Ah! we defined our `animal_house` to be a `cat_house` :facepalm:

    # If we did this...
    var housed_cat: Cat = cat_house[0]
    print(housed_cat)
    # ... we get "Fido"... He's not supposed to be a cat tho

I actually never tried this in GDScript... I should probably file a bug on as ending up a runtime error. When I need generics, I use dynamic fields + casting. Not that the usage is that common to begin with; Generics (specially in the standard library) would greatly increase the productivity of developers, similar to how lambdas did with Godot 4. But back on topic.

I'm no expert, but I am aware of this issue in other languages, and how it is usually solved: Adding a variance marker:

# How it is done in Scala, `+`/`-`
class House[+T]: var data: T
# How it is done in C#, `out`/`in`
class House[out T]: var data: T

If you don't include it, the default is invariant, which I think is nice.

I prefer the Scala version because types can get hard, and sometimes the meaning of the two markers flip, because variance is hard. That said, the best option, might just be to make all types invariant, meaning that this would never compile or run: var animal_house: Array[Animal] = cat_house as Array[Animal]. And I'm not even sure if it'd even be possible since types are dynamic.

---

By the way, if after reading this you think that we can just not have generic types House[T], and focus on methods func add[T](a: T, b: T) -> T: pass, think again, variance is even more problematic on methods, just thought starting on types might be best conceptually.

---

Adapted from https://docs.scala-lang.org/tour/variances.html

[julian-a-avar-c]

Another topic to cover in relation to generics, is how do we want to implement constraints:

class Animal:
    var name: String
    func _init(_name: String):
        self.name = _name
class Dino extends Animal: pass
class Fish extends Animal: pass
class Tuna extends Fish: pass

# Scala style
class House[T <: Fish]: var data: T
# Java style
class House[T extends Fish]: var data: T
# C# style
class House[T] where T : Fish: var data: T
# ... there are more

I think using the extends keyword instead of a symbol (:, <:) would be better for GDScript, but then again, I love the fact that in Scala I can use <: and >:, because in my personal experience, where clauses attract complexity. Not like any of these things are not complex :P

I'm not even sure if it would be worth it to add constraints, but it is something to consider.

[jonbonazza]

Another topic to cover in relation to generics, is how do we want to implement constraints:

class Animal:
  var name: String
  func _init(_name: String):
      self.name = _name
class Dino extends Animal: pass
class Fish extends Animal: pass
class Tuna extends Fish: pass

# Scala style
class House[T <: Fish]: var data: T
# Java style
class House[T extends Fish]: var data: T
# C# style
class House[T] where T : Fish: var data: T
# ... there are more

I think using the extends keyword instead of a symbol (:, <:) would be better for GDScript, but then again, I love the fact that in Scala I can use <: and >:, because in my personal experience, where clauses attract complexity. Not like any of these things are not complex :P

I'm not even sure if it would be worth it to add constraints, but it is something to consider.

I feel that introducing new operators for this is overkill. I think just using extends here is the right move to keep parser complexity down.

[AlbertoRota]

While we wait for "Generic types" to be added to GdScript, we can achieve the exact same functionality (Base class being "Type agnostic" and specific instances being "Type safe") with just a bit of boilerplate, here you have the example for a very basic implementation of Pair<K, V> in GdScript:

First we need to define a "Type agnostic" version of what we want to do:

# generic_pair.gd
class_name GenericPair

var key
var value

func _init(key, value):
    self.key = key
    self.value = value

func get_key(): return key
func get_value(): return value

This is the base class, and it is NOT meant to be used "As is", just think of it as an abstract class.

Then we have the type-safe extension of the base class:

# int_string_pair.gd
extends GenericPair
class_name IntStringPair

func _init(key:int, value:String): super._init(key, value)

func get_key() -> int: return super.get_key() as int
func get_value() -> String: return super.get_value() as String

Bear in mind that you need to create one implementation per each differently typed Pair<K, V> that you might need.

Finally here is how its usage looks like:

# We do this instead of `var int_string_pair := Pair<int, String>.new(3, "The third")`.
var int_string_pair := IntStringPair.new(3, "The third")

# typed_key is resolved as `int`, not as variant.
var typed_key := int_string_pair.get_key()

# typed_value is resolved as `String`, not as variant.
var typed_value := int_string_pair.get_value()

# Fails due to incompatible types:
var wrongly_typed_key: Vector2 = int_string_pair.get_key()
var wrongly_typed_value: Vector2 = int_string_pair.get_value()
var wrongly_typed_pair := IntStringPair.new("The third", 0)

It is a bit verbose and requires to create an extra class for every type permutation used... but you will achieve the exact same behaviour and benefits of the "Generic types".

Although it will be GREAT if we can save all that boilerplate and have native support to it so that we can save the boilerplate.

Note: How can we implement constraints with this work-around?

If the base class (GenericPair in this particular example) rather than accepting Variant, is typed to accept a defined type (Node2D for example), extension classes can only be created for types that extend that class, and it will fail at Editor/Compile time if you don't.

[DaloLorn]

The whole point, IMO, is to eliminate boilerplate code. Both to improve ease of development and reduce the possibility of bugs creeping in over time.

Gnumaru's comment in October 2021 echoes a lot of my sentiment, though if we can get more than what he's asking for, I'm obviously all for it.

[smedelyan]

Maybe we can at least start with (if I understand it correctly and not mix things) template classes used in C++?

[julian-a-avar-c]

Can we not have C++ style generics, AKA template classes? It's a hole that's difficult to climb out of.

[AThousandShips]

Do you mean the syntax or the specifics of having each template compiled only when needed and fully individually?

[smedelyan]

Can we not have C++ style generics, AKA template classes? It's a hole that's difficult to climb out of.

I don't have much experience with C++, but just thought that at least generating code, perhaps, not that hard as implementing full-blown generics, - and we would all benefit from at least this very much. However, if there are serious drawbacks, I agree that implementing templates may worsen the situation because of compatibility commitments (the engine would need to continue supporting this thing even if it's bad).

Again, this is just a cheaper (I hope) alternative. If we are choosing between no generics at all and at least something - I would prefer at least something (but it would be great to have real generics, of course)

Do you mean the syntax or the specifics of having each template compiled only when needed and fully individually?

More of the latter. I think that generating code is sort of syntactic sugar that is easier to implement, and maybe this is even easy to turn on / off with some flag in the editor. I mean, maybe this could be a small "patch" to not keep everyone waiting for ages while the Godot team works on the real good implementation of that

[julian-a-avar-c]

Sorry, let me try being clearer, I'm concerned about type safety. I'm against some simple text processing like C++ templates. @AThousandShips @smedelyan

[DaloLorn]

For my part, I think Java-like type erasure should be the bare minimum to aim for.

[shrubbgames]

For another use case, I'm commonly having to wait for parameters to be set in a networked game. I'd really like to have a generically typed container of a single value.

class_name Synced<T>

var data : T
var is_valid := false
signal is_set()

func get():
    if !is_valid:
        await is_set
    return data

func set(value: T):
    data = value
    is_valid = true
    is_set.emit()

Just wanted to throw this example out there because the word "collections" in the title might imply more than one value, although I don't think it really makes a difference.

This example would also benefit from the terrible-but-yet-useful code generation without being terribly impacted by type safety. With only one value in the container, it becomes a lot simpler to deal with the generic type. I'm also quite used to Java where I have things like Optional<T>, another use case I would appreciate in gdscript.

[FabriceCastel]

To add my own use case to the pile, I'd love to be able to do something like..

class_name Observable<T> extends RefCounted

signal on_change(v: T)

var value: T:
  set(v):
    if v != value:
      value = v
      on_change.emit(value)

func _init(v: T) -> void:
  value = v

# Assuming Callable gets more type information with an arbitrary
# number of param types and its return type specified last..
func observe(callback: Callable<T, void>) -> void:
  on_change.connect(callback)
  callback.invoke(value)

As is, I've had to implement my own IntObservable, BoolObservable, etc etc for every damn type I want to observe

[P5ina]

In my opinion, it would be preferable to utilize square brackets ([]) instead of angle brackets (<>) for generic functions. GDScript already provides a generic mechanism for typed arrays, as demonstrated by the following examples:

var arr: Array[int]

or in the 4.4 version:

var dict: Dictionary[String, int]

Therefore, a generic function could be defined as follows:

func generic_function[T](some_variable: T):
    pass

func generic_function[T extends Node](some_variable: T):
    pass

While I understand that there may be alternative perspectives, I believe this approach aligns more closely with consistency of existing features in GDScript language.

[DaloLorn]

IIRC it's not strictly generics under the hood, but syntactically, yeah, your logic checks out.

I'm largely indifferent to the bracket choice, though, as long as it gets implemented at all! :joy:

[FabriceCastel]

Yeah, I'm on the same page as @DaloLorn

I used angled brackets in my example because that's muscle memory from years of Java, but I'm indifferent to whether Godot adopts <T> or [T]. Either is fine, I just don't want to have to duplicate my utility classes a zillion times for the sake of static types (and no, I will not give in to Variant, I don't care for wasted time caused by runtime errors where I can help it)