function getText() doesn't work properly

[ksudol]

Hello, I have the following grammar file for ANTLR4.5.1:

grammar Test;

start : NAME+;
NAME :  ('aaa'|'ab') {System.out.println("["+getText()+"]");}'X';
WS : . -> skip;

and java file:

import org.antlr.v4.runtime.*;
import org.antlr.v4.runtime.tree.*;

public class ParseTest {
    public static void main(String[] args) throws Exception {
        ANTLRInputStream stream = new ANTLRInputStream("aaaX abX aaaX");
        TestLexer lexer = new TestLexer(stream);
        CommonTokenStream tokens = new CommonTokenStream(lexer);
        TestParser parser = new TestParser(tokens);

        ParserRuleContext tree = parser.start();
    }
}

Function getText() returns: [aaa] // ok [abX] // wrong, 'X' should be matched after calling the function getText() [aaa] // ok

[parrt]

actions are only executed after entire token is matched. i know this action placement is misleading but...

[sharwell]

@parrt Actually this use case was intended to be enabled by #408 and #469 (ANTLR 4.2.1). You can see warning 132 (LEXER_ACTION_PLACEMENT_ISSUE) was deprecated at the same time. The current test for position-dependent lexer actions is in ActionPlacement.stg.

[ksudol]

In the case of token: "aaaX", getText() returns: "aaa" (according to action placement)

But if I'm trying to parse the following string: "abX aaaX" getText() returns: [ab] // ok [aa] // token is cut off (if the first token found is shorter)

[parrt]

@sharwell Are you saying there is a bug in the updates we made to enable this?

[sharwell]

@parrt I'm just saying that I too would expect the output that @ksudol expected in the original post

[parrt]

Ah. seems to me that we simply added an error message that actions can only appear on the right side. Hmm...ok.