GDScript 2.0 - multiline lambdas not supported

[ngotoandev]

This is flagged as error on gd-format but completely legit syntax in GDScript 2.0:

func f():
  return func():
    pass

Error:

return func():
                      ^

Unexpected token Token('_NL', '\n\t\t') at line 11, column 16.
Expected one of:
        * NOT
        * HEX
        * BANG
        * BIN
        * VAR
        * AWAIT
        * PERCENT
        * NUMBER
        * LPAR
        * LBRACE
        * REGULAR_STRING
        * CIRCUMFLEX
        * TILDE
        * LSQB
        * RETURN
        * NAME
        * LONG_STRING
        * DOLLAR
        * MINUS
        * FUNC
        * PLUS
        * AMPERSAND
        * PASS

[Scony]

Multiline lambdas are currently not supported. They won't be anytime soon as it requires writing a custom lexer to avoid ambiguity on the parser level.

[halcaponey]

They won't be anytime soon as it requires writing a custom lexer to avoid ambiguity on the parser level.

Just out of curiosity, what is the ambiguity?

[Scony]

As far as I remember the clash was on newlines/indent/dedent. The key thing is to get proper indent/dedent when in ([{.

[ogrady]

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

[Scony]

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd

As for exempting blocks of code from the whole linting process - it's not possible atm.

[ogrady]

Thanks for letting me know, and great work providing this toolkit btw. 👍

[iandoesallthethings]

Any movement on this bug? I've got a working 2d grid iterator pattern going, but formatting breaks and throws errors any time I use it, which is a bummer.

func damage_all(attack):
    for element in attack.keys():
        var elemental_damage = func(space, enemy):
                if enemy == null: return
                enemy.damage(attack[element], element)
                if enemy.current_health <= 0:
                    $enemies.destroy_entity_at(space)
        $enemies.for_each(elemental_damage)

[Scony]

Not yet.

[m21-cerutti]

Same problem here on a maybe more tricky problem

var l_add_and_print := func(root_path, name, is_file):
        if is_file:
            var path = root_path+"/"+name
            gut.p(path)
            paths.append(path)

Error:

print := func(root_path, name, is_file):
                                        ^

Unexpected token Token('_NL', '\n\t\t') at line 8, column 56.
Expected one of: 
    * DOLLAR
    * VAR
    * PERCENT
    * NAME
    * FUNC
    * NUMBER
    * PLUS
    * AMPERSAND
    * RETURN
    * HEX
    * NOT
    * PASS
    * LPAR
    * BIN
    * LONG_STRING
    * TILDE
    * LSQB
    * AWAIT
    * CIRCUMFLEX
    * REGULAR_STRING
    * LBRACE
    * BANG
    * MINUS

Disable the linter is ok but we can't disable the formatter for this problem. Also if we have a "if", can't make the trick to do all in the same line Unexpected token Token('COLON', ':') at line 11, column 66.

[tavurth]

    camera.tween.finished.connect(func():
        camera.tween.stop()

        # Switch to linear tween for a smoother follow
        camera.tween\
            .tween_method(set_pos, 0.0, 1.0, time)\
            .set_trans(Tween.TRANS_LINEAR)

        camera.tween.play()
    )

[Skyway666]

I can understand not wanting to support Multiline Lambdas if it implies too much work (I've never build a linter or formatter) since a simple workaround is creating a separate function for the callback.

However, would it be much harder to work on handling the exception? When parsing multiple files the user doesn't even know which one is causing the error, and it's hard to deduce why it is happening. An error message as the following would be helpful:

Error in "sample.gd" line 32: gdscript toolkit doesn't support Multiline Lambda functions, please create a separate function to assign as callback.

At least it will prevent users from writing more duplicate issues about the matter :)

[Scony]

@Skyway666 that's a good point - I can try exploring that. It's definitely not as simple as writing a single if statement or so, but maybe with some more conditions, it will be possible to give such helpful error in, say, 80% of situations. That would help a lot, I agree.

[tavurth]

@Scony this would help so much!

Currently it can be hard to figure out even what type of error is going on.

You have to delete the file in halves to try and figure out where the error is coming from.

[MikeSchulze]

Hi, i run into same issue today, but i got a more hard error. Using Python 3.12.1 and gdlint latest master

$ gdlint example.gd
Traceback (most recent call last):
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 590, in lex
    yield lexer.next_token(lexer_state, parser_state)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 528, in next_token
    raise UnexpectedCharacters(lex_state.text, line_ctr.char_pos, line_ctr.line, line_ctr.column,
lark.exceptions.UnexpectedCharacters: <exception str() failed>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__main__.py", line 134, in _lint_file
    problems = lint_code(content, config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__init__.py", line 118, in lint_code
    parse_tree = parser.parse(gdscript_code, gather_metadata=True)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\parser\parser.py", line 62, in parse
    self._parser_with_metadata.parse(adjusted_code)
  File "D:\development\tools\pyton\Lib\site-packages\lark\lark.py", line 645, in parse
    return self.parser.parse(text, start=start, on_error=on_error)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parser_frontends.py", line 96, in parse
    return self.parser.parse(stream, chosen_start, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 41, in parse
    return self.parser.parse(lexer, start)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 171, in parse
    return self.parse_from_state(parser_state)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 193, in parse_from_state
    raise e
  File "D:\development\tools\pyton\Lib\site-packages\lark\parsers\lalr_parser.py", line 183, in parse_from_state
    for token in state.lexer.lex(state):
  File "D:\development\tools\pyton\Lib\site-packages\lark\indenter.py", line 45, in _process
    for token in stream:
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 599, in lex
    raise UnexpectedToken(token, e.allowed, state=parser_state, token_history=[last_token], terminals_by_name=self.root_lexer.terminals_by_name)
lark.exceptions.UnexpectedToken: <exception str() failed>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\common\exceptions.py", line 14, in lark_unexpected_token_to_str
    return f"{exception.get_context(code)}\n{exception}"
                                            ^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 256, in __str__
    % (self.token, self.line, self.column, self._format_expected(self.accepts or self.expected)))
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 142, in _format_expected
    expected = [d[t_name].user_repr() if t_name in d else t_name for t_name in expected]
                ^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 124, in user_repr
    return self.pattern.raw or self.name
           ^^^^^^^^^^^^^^^^
AttributeError: 'PatternStr' object has no attribute 'raw'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\development\tools\pyton\Scripts\gdlint.exe\__main__.py", line 7, in <module>
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__main__.py", line 66, in main
    problems_total += _lint_file(file_path, config)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\linter\__main__.py", line 148, in _lint_file
    lark_unexpected_token_to_str(exception, content),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\gdtoolkit\common\exceptions.py", line 16, in lark_unexpected_token_to_str
    return f"{exception}".strip()
             ^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 256, in __str__
    % (self.token, self.line, self.column, self._format_expected(self.accepts or self.expected)))
                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\exceptions.py", line 142, in _format_expected
    expected = [d[t_name].user_repr() if t_name in d else t_name for t_name in expected]
                ^^^^^^^^^^^^^^^^^^^^^
  File "D:\development\tools\pyton\Lib\site-packages\lark\lexer.py", line 124, in user_repr
    return self.pattern.raw or self.name
           ^^^^^^^^^^^^^^^^
AttributeError: 'PatternStr' object has no attribute 'raw'

The example.gd

func contains_keys(expected :Array):
    var keys_not_found :Array = expected.filter(func(key):
        prints("key:", key)
        return true
    )

The only workaround is to move the inline lambda to a function and use it. The question remains, will this bug be fixed or will there at least be a flag to exclude such code blocks from the linter like # gdlint:disable=lambda

[Scony]

@MikeSchulze since it does happen on the parser level, the # gdlint:disable= is not possible. I'll try to fix it at some point probably. Anyway, it's still a low prio as for fairly small lambdas it can be worked around by turning multiline lambda into single-line one: func(key):prints("key:", key);return true

[lethefrost]

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd

As for exempting blocks of code from the whole linting process - it's not possible atm.

Sorry, this page is gone now... Do you mind explaining a little more about how to workaround?

Also, I am wondering if it would be able to just add some try/catch block to this error, so that at least the formatter can continue and finish formatting the file regardless of the error, instead of crashing at the point it encounters a lambda function and doesn't format at all.

[Scony]

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd As for exempting blocks of code from the whole linting process - it's not possible atm.

Sorry, this page is gone now... Do you mind explaining a little more about how to workaround?

Also, I am wondering if it would be able to just add some try/catch block to this error, so that at least the formatter can continue and finish formatting the file regardless of the error, instead of crashing at the point it encounters a lambda function and doesn't format at all.

As for the workaround, lambda may be written in one line where statements are separated by a semicolon like this:

var f = func(key):print("key:", key);return true

also, if the lambda has some expression inside, it can be extended to multiple lines like:

var f = func(x): return (
    x is int
    and x > 0
    and x < 10
)

As for the try-catch - it's not possible without multiline lambdas support.

[lethefrost]

So what is the current way to work around this? As this seems to be a lexer error, we can't really gdlint: disable, I assume. Is there a way to exempt blocks of code from the whole linting process?

You can workaround it by having lambda returning one bing expression, or you can use ; to separate statements. See: https://github.com/Scony/godot-gdscript-toolkit/blob/master/tests/formatter/input-output-pairs/long-inline-lambdas.out.gd As for exempting blocks of code from the whole linting process - it's not possible atm.

Sorry, this page is gone now... Do you mind explaining a little more about how to workaround?

Also, I am wondering if it would be able to just add some try/catch block to this error, so that at least the formatter can continue and finish formatting the file regardless of the error, instead of crashing at the point it encounters a lambda function and doesn't format at all.

As for the workaround, lambda may be written in one line where statements are separated by a semicolon like this:

var f = func(key):print("key:", key);return true

also, if the lambda has some expression inside, it can be extended to multiple lines like:

var f = func(x): return (
    x is int
    and x > 0
    and x < 10
)

As for the try-catch - it's not possible without multiline lambdas support.

Thanks for the clarification, and looking forward to a authentic solution. Just by the way thank you for all the efforts bringing this amazing toolkit to us.

[Bloodyaugust]

@Scony I wholly understand that this issue is very tricky to fix, but a callout in the main documentation that multiline lambdas are currently causing gdformat to error out would be much appreciated. Finding this issue was a little tricky, and for a while I was tearing my hair out.

[Scony]

@Scony I wholly understand that this issue is very tricky to fix, but a callout in the main documentation that multiline lambdas are currently causing gdformat to error out would be much appreciated. Finding this issue was a little tricky, and for a while I was tearing my hair out.

That's a very good point - I'll address that, thanks.

[oxeron]

Any news on this subject ?

[Scony]

Any news on this subject ?

It's WIP, you can track progress here: https://github.com/Scony/godot-gdscript-toolkit/commits/multiline-lambdas/

TL;DR the lexer and parser support is done, the formatter support is like 5% done.

[JohnDevlopment]

Hi, I wrote var cb := func(button: Button): button.release_focus() but still got an error. I'm using the latest version. This code is all on one line.

EDIT: Nevermind, there was a multiline lambda later on in my code.

[Jack-023]

TL;DR the lexer and parser support is done, the formatter support is like 5% done.

Does this mean that the linter works? Would it be possible to update gdlint to support multi-line lambdas without gdformat? I have some pretty long lambdas so it is difficult to work with them if they are in a single line but I like having a precommit hook run gdlint. I am working around it at the moment but by keeping the non-minified version in a comment but it is pretty hacky.

[Scony]

TL;DR the lexer and parser support is done, the formatter support is like 5% done.

Does this mean that the linter works? Would it be possible to update gdlint to support multi-line lambdas without gdformat? I have some pretty long lambdas so it is difficult to work with them if they are in a single line but I like having a precommit hook run gdlint. I am working around it at the moment but by keeping the non-minified version in a comment but it is pretty hacky.

Yes, linter should work - but making gdformat (even broken) aligned to that requires effort, therefore I can only suggest you to try https://github.com/Scony/godot-gdscript-toolkit/tree/multiline-lambdas branch yourself.

P.S. I'm not sure but there may be some regression in the parser so if you get parser errors then it won't work for you anyway.

[Jack-023]

That makes sense, thanks!

[Scony]

The support for multiline (multi-statement) lambdas has been merged into master now.

There are few not implemented problems, but IMO 99.9% of users should not notice them, so I decided to proceed.

Now, I'd like the change to sink in for few days/weeks before I release new gdtoolkit version.

If someone here would like to help, I'd appreciate it if someone could install gdtoolkit from master using either:

pip3 install git+https://github.com/Scony/godot-gdscript-toolkit.git
# or
pipx install git+https://github.com/Scony/godot-gdscript-toolkit.git

and write a comment here stating that "in my project it works!"

[tavurth]

"In my project it works!"

[lethefrost]

The support for multiline (multi-statement) lambdas has been merged into master now.

There are few not implemented problems, but IMO 99.9% of users should not notice them, so I decided to proceed.

Now, I'd like the change to sink in for few days/weeks before I release new gdtoolkit version.

If someone here would like to help, I'd appreciate it if someone could install gdtoolkit from master using either:

pip3 install git+https://github.com/Scony/godot-gdscript-toolkit.git
# or
pipx install git+https://github.com/Scony/godot-gdscript-toolkit.git

and write a comment here stating that "in my project it works!"

Finally!! Thank you so much. It's my hugest problem with this formatter. ❤️

Would you mind elaborating what the few remaining issues are? So that people might help on it.

[plink-plonk-will]

In my project it works! This is fantastic - thank you!

[Scony]

The support for multiline (multi-statement) lambdas has been merged into master now. There are few not implemented problems, but IMO 99.9% of users should not notice them, so I decided to proceed. Now, I'd like the change to sink in for few days/weeks before I release new gdtoolkit version. If someone here would like to help, I'd appreciate it if someone could install gdtoolkit from master using either:

pip3 install git+https://github.com/Scony/godot-gdscript-toolkit.git
# or
pipx install git+https://github.com/Scony/godot-gdscript-toolkit.git

and write a comment here stating that "in my project it works!"

Finally!! Thank you so much. It's my hugest problem with this formatter. ❤️

Would you mind elaborating what the few remaining issues are? So that people might help on it.

Problem one: var foo = func(): if true: return 1 won't parse

Problem two: https://github.com/Scony/godot-gdscript-toolkit/commit/89b8200f403fcd1364bbcc8ae4cf80597b8243b1#diff-deb96115f719c01f21e10dcc674db18d6be7d759a1a3badaf506c9c9f6625a0fR216-R221

Both are most likely a bug in the lexer, so hopefully, I can address those fairly easily.