Python-style list comprehensions in GDScript

[SwissCore92]

Issue description

Godot's GDScript team rejected python-style list comprehensions in #2972 for readability reasons.

It would be very nice to see this feature in Redot.

Steps to reproduce

doesn't work but is more readable:

var happy_users = [user for user in redot.users if user.uses_gdscript]

does work but is less readable in my opinion:

var happy_users = redot.users.filter(func(user): return user.uses_gdscript)

does work and is readable but clunky:

var happy_users = []
for user in redot.users:
    if user.uses_gdscript:
        happy_users.append(user)

[muhuk]

Please refrain using ALL CAPS in issue titles.

[nukeop]

I oppose this syntax. It could be confusing for newcomers, and the same functionality can already be achieved by using existing constructs.

[SwissCore92]

I oppose this syntax. It could be confusing for newcomers, and the same functionality can already be achieved by using existing constructs.

Why should this be more confusing than using filter(lambda) to newcomers? the comprehension can literally be read like a english scentence.

[nukeop]

I don't know, to me this is way less readable. This doesn't read like a natural language sentence nor like a logical statement. Maybe there is a better example out there, but from this one — this is confusing way to express the intent with code.

Code readability is one of the most important metrics when designing a programming language. Writing code takes way less time than reading it in one's career and day-to-day.

IMO turning nested logic into a one-liner always worsens the readability. LOCs are not a resource you can run out of, so I see no reason to try and be as terse as possible. I have no idea what the code you've shown here does.

[AlexMGitHub]

I support this feature request. I don't think it's a high priority, but the objection to it is baseless.

GDScript is openly modeled after Python, and list comprehensions are one of the most-loved features in Python.

Here's some evidence for that claim:

https://www.reddit.com/r/Python/comments/c4m1lm/whats_your_favorite_syntactic_sugar_in_python/?sort=top

The most-upvoted answer to the post "What's your favorite syntactic sugar in Python?" is list comprehensions, with 126 votes.

[cheetoray]

Just replace the entire thing with real Python. It'll work faster and will support stuff like NumPy.

[SwissCore92]

Just replace the entire thing with real Python. It'll work faster and will support stuff like NumPy.

Python is not typesave

[IllusionDX]

Honestly it would be better if the scripting language was Python and not GDScript

[SwissCore92]

Honestly it would be better if the scripting language was Python and not GDScript

GDScript is well developed and integrated tightly into the engine. And it is (semi) typesave. Me and lot of godot users like working with it. I am also a big python fan. But I oppose the idea integrating it into the engine. I dont see any benefit.

I just would like to see list comprehensions in GDScript.

Edit: And to all disliking people: Would this feature hinder you? If you don't like it, just don't use it.

[SkogiB]

I personally find the proposed 'readable' line insufferable, but if some people would prefer to use it I can see why. gdscript overhauls are on our radar, but it's very much down the line.

A good point to be made is that a lot of Python users WOULD want this, as its a syntax they're used to and would greatly ease and please their onboarding process into gdscript and the engine. As Swiss said, if we don't like it we don't need to use it. It seems like this is simply never happening in Godot, so we may be able to look at adding this functionality. It may be wise for us to poll the community to see how many Python lovers are out there that would benefit from this as well. Personally would not use it, but I see why people want it.

[nukeop]

There are many people who don't use Python in this community, myself included, so they didn't grow to learn these constructs. Through this discourse I've learned how to read it, but I still don't like the syntax. Not for the least part because sometimes you have to read it right to left (or bottom to top if pretty-formatted) to make sense. That's not how I usually read code.

GDScript aims to be easy to pick up and learn, but there is no plan or goal to make it Python-like. In fact, one of the reasons to not include something like this proposed syntax is to keep GDScript easy to pick up and learn. You just need to consider people, who are not familiar with Python or programming in general, or have little experience with it and no expectations of similarity.

And if you really want to use Python with Godot, just use Python 👀

[tokengamedev]

I think GDScript is simple and expressive enough to be easily understood by a novice user. If there is a way this can be done, without adding sugar coated syntax, then this is not required, as it will add performance overhead.

GDScript is not a GP(General Purpose) language, it can be compared to GLSL shader language as domain specific. GDScript is to Redot/Godot game engine as GLSL language is to Shader program in Graphics pipeline.

[SLimeyMC]

This is coming from someone who has never heard of list comprehensions syntax. I see it as

capture_variable for capture_variable? in list if_logical_statement

(No idea why there seem to be 2 capture variables, again I never learned the syntax)

While the pythonic code feels fine, I don't think it's worth adding. The filter function is fine as it is Cause it will look like this

list.filter(func(capture_variable): logical_statement

Or in English form

filter list of capture variable by logical statement

Even some languages especially functional ones where everything is expressive can just do something like this

list filter it.is_true

(Not relevant but I figure I might drop it here)

But the imperative way is fine and honestly the best practice for GDScript.

[SwissCore92]

This is coming from someone who has never heard of list comprehensions syntax. I see it as

capture_variable for capture_variable? in list if_logical_statement

because you can also do:

[capture_varialbe.property for capture_variable in collection if logical_statement]

it also works with dictionaries

{capture_variable.name: capture_variable.property for capture_variable in collection if logical_statement}

Edit: I know, all of this can also be done using map(), filter(), reduce() or just a loop.

I see, this is a controversial topic. So it might be better not to implement it. But i promise, everyone who is familiar with this syntax would be very happy if it was implemented. And everyone who is not familiar can learn it very quickly.

And I think noobs/newcomers are confused anyway at the beginning.

[nitori]

I thought I'd comment a small overview over Python's list comprehension features for those that don't know much about them or Python in general.

Small foreword: The list comprehension notation syntax ist not arbitrarily chosen like that.

It's actually based on the mathematical set-builder notation, and anyone that has studied a bit of mathematics would probably recognize it as such (well, at least I did):

https://en.wikipedia.org/wiki/List_comprehension

---

The Python list/set/dict comprehension and generator expression notation is a mix of a "map" and an optional "filter". Additionally, it can have nested for loops.

The simplest form is a simple "map". The left-most (left of the "for") is the expression that is added to the squares list.

numbers = [1, 2, 3, 4, 5]
squares = [x**2 for x in numbers]
print(squares)
# [1, 4, 9, 16, 25]

On the very right you can add an optional if-condition, acting as a filter. Here only even numbers are kept and squared:

numbers = [1, 2, 3, 4, 5, 6, 7, 8, 9]
squares = [x ** 2 for x in numbers if x % 2 == 0]
print(squares)
# [4, 16, 36, 64]

And regarding the nested loops. It's seen more rarily in Python, and at that point the readabilty of the syntax usually starts to break down. But the probably most common and simplest usecase is for flattening a nested list:

nested = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = [num for lst in nested for num in lst]
print(flat)
# [1, 2, 3, 4, 5, 6, 7, 8, 9]

The order in which the for-loops need to be written isn't immediately obvious. But it's actually exactly the same order you'd write a normal nested for-loop, but instead of line breaks and indentations, you have simply spaces between the loops.

The equivalent normal nested for loop:

nested = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flat = []
for lst in nested:
    for num in lst:
        flat.append(num)
print(flat)

If you take this part:

for lst in nested:
    for num in lst:

And just remove the colons and line break:

for lst in nested for num in lst

You got most of the comprehension already done. You only need to add the num output expression to the beginning and wrap it in [...]

flat = [num for lst in nested for num in lst]

Another possible use-case for nested loops is a simple product:

flat = [(x, y) for x in [1, 2, 3] for y in [4, 5, 6]]
print(flat)
# [(1, 4), (1, 5), (1, 6), (2, 4), (2, 5), (2, 6), (3, 4), (3, 5), (3, 6)]

set, dict and generators

Additionally, you can create set and dict comprehensions, as well as generator expressions in Python.

The only difference between list and set comprehension ist the set comprehensions are wrapped in {...} instead of [...].

The same goes for dict comprehensions, however, in those cases you need to specify a key:value pair as the left-most expression.

Generator expressions are like list-comprehensions, except the expression is evaluated lazily, instead of building an entire list.

Example set:

numbers = [-5, -4, -3, -2, -1, 1, 2, 3, 4, 5]
unique_squares = {x ** 2 for x in numbers}
print(unique_squares)
# {1, 4, 9, 16, 25}

Example dict:

numbers = [1, 2, 3, 4, 5]
squares_map = {x: x ** 2 for x in numbers}
print(squares_map)
# {1: 1, 2: 4, 3: 9, 4: 16, 5: 25}

Generator expressions

numbers = range(1_000_000_000_000)  # represents: 0 to 1 trillion
squares = (x ** 2 for x in numbers)
print(squares) # <generator object main.<locals>.<genexpr> at 0x000001ABDB591BE0>
print(next(squares)) # 1
print(next(squares)) # 4
print(next(squares)) # 9

Note: next() is part of Python iterator protocol. It's basically what Python's for-loop uses under the hood. range() also returns a magical object that doesn't use any memory for its items - not sure if GDScript has something similar.

---

As far as I know gdscript doesn't have any set literals, and I couldn't find anything about generators or similar (functions using the yield keyword). So those two might not be in the current scope.

But list and dict comprehension should be doable.

Greetings Nitori.

(Sani (sanisensei) in Redot Discord - if you got any questions)

[derula]

I couldn't find anything about generators or similar (functions using the yield keyword). So those two might not be in the current scope.

yield exists in GDScript, but has nothing to do with generators. Instead, it's a legacy notation for coroutines (don't ask). In general, functions in Python that also exist in GDScript (e.g. dict.keys) and return iterators / generators in Python, always return arrays (/ packed arrays) in GDScript.

Two takes from a Python dev by trade:

• IMHO, a = b if c else d isn't any less readable than list comprehensions. So the argument that one should be in GDScript, but the other shouldn't, makes no sense to me. If we want to keep the ternary abomination, might as well add list comprehensions. If we don't want list comprehensions because they're hard to read, we should also remove that other thing for consistency.
• List comprehensions are IMHO a less important side note and should take a backseat to more pressing features such as improved static typing (e.g., Dictionary[String, int] for a dictionary with string keys and int values, Array[Array[String]] for a nested array, String | int for a restricted Variant that only accepts strings or integers.)

[derula]

I thought about it a bit more.

Arguably, one reason why list comprehensions are so popular in Python (and why the syntax is so weird), is because single-line loops aren't a thing there.

var a = []
for i in range(5): a.append(i)  # <- would be a syntax error in Python

Why not use this existing syntax to create something like this?

var a = [for i in range(5): i]

Can be chained with if and operations in the same way:

var a = [for i in range(5): if i % 2: i ** 2]

Even works with while loops:

var t = true
var a = [while t: t = determine_t(); t]

And can be broken down to multiple lines:

var t = true
var a = [while t:
    t = determine_t()
    t
]

While we're at it, why not similarly allow using if / match statements as expressions, like Ruby?

var a = if b:
    c
else: d

Maybe allow single lining it like so:

var a = if b: c; else: d

GDScript is not Python, why not use the tools we have in GDScript to our advantage?

[SLimeyMC]

I thought about it a bit more.

Arguably, one reason why list comprehensions are so popular in Python (and why the syntax is so weird), is because single-line loops aren't a thing there.

var a = []
for i in range(5): a.append(i)  # <- would be a syntax error in Python

Why not use this existing syntax to create something like this?

var a = [for i in range(5): i]

Can be chained with if and operations in the same way:

var a = [for i in range(5): if i % 2: i ** 2]

Even works with while loops:

var t = true
var a = [while t: t = determine_t(); t]

And can be broken down to multiple lines:

var t = true
var a = [while t:
    t = determine_t()
    t
]

While we're at it, why not similarly allow using if / match statements as expressions, like Ruby?

var a = if b:
    c
else: d

Maybe allow single lining it like so:

var a = if b: c; else: d

GDScript is not Python, why not use the tools we have in GDScript to our advantage?

That work, it's just stream api with map/fold lambda at the right. To be fair just using map/fold method is better.

How would you treat filter though, I'm guessing the if? But wouldn't that make it hard to read? I think we should add new keyword or different syntax for filtering.

Also to add I think the code convention for code block inside bracket should be indented by 2 to separate with other code block.

[SwissCore92]

Maybe allow single lining it like so:

var a = if b: c; else: d

GDScript is not Python, why not use the tools we have in GDScript to our advantage?

GDScript is already supporting this

var some_value = a if some_condition else b

--

var a = []
for i in range(5): a.append(i)  # <- would be a syntax error in Python

This is wrong. Python supports such one liners. This code would NOT raise a syntax error. And this is also not the reason why comprehensions are so popular in python.

I recommend reading this article about the advantage of comprehensions.

[derula]

GDScript is already supporting this

var some_value = a if some_condition else b

I know about this syntax. My point was that if the argument is that a for b in c is confusing and unnecessarily different from regular syntax, then the same is true in my opinion for a if b else c, suggested one could keep the existing syntax which would arguably avoid this confusion, and pointed out that the same could be done with if/else.

This is wrong. Python supports such one liners. This code would NOT raise a syntax error.

Crazy. I guess I'm just an idiot and never tried it. I blame code formatting tools for taking that away from us anyway.

How would you treat filter though, I'm guessing the if? But wouldn't that make it hard to read? I think we should add new keyword or different syntax for filtering.

One of my examples had an if. My point being it's the same as for standalone code outside of comprehensions:

# No comprehension:
var a = []
for i in range(5): if i % 2: a.append(i ** 2)
# Comprehension:
var a = [for i in range(5): if i % 2: i ** 2]

Also to add I think the code convention for code block inside bracket should be indented by 2 to separate with other code block.

Yeah sure can format like:

var a = [
    while t:
        t = determine_t()
        t
]