Tool does not generate any constants for channels for the TypeScript target

[kaby76]

I don't know if this is by design: channels declared in a lexer grammar file are not generated into constants in the generated lexer. For an example grammar, see PhpLexer.g4, then run the tool with -Dlanguage=TypeScript and look at the generated .ts files.

It seems that this should be possible because it generates constants for token types, e.g., public static readonly PHPStart = 4;.

[ericvergnaud]

Mmm... the target template should be generating them, see TypeScript.stg, line 875: public static readonly channelNames: string[] = [ "DEFAULT_TOKEN_CHANNEL", "HIDDEN"<if (lexer.channelNames)>, <lexer.channelNames:{c| "<c>"}; separator=", ", wrap, anchor><endif> ];

[kaby76]

Sure, those are the names of the channels declared as an array of strings. But, if you look at the generated lexer for CSharp or probably Java, you also get constant declarations. So, when I want to reference the channel in base class code, like this in CSharp (https://github.com/antlr/grammars-v4/blob/c7883a594bf622200c561db181848d277fcd49a7/php/CSharp/PhpLexerBase.cs#L79), I can't in TypeScript (https://github.com/kaby76/grammars-v4/blob/865ccef278193e46e6cb285ea4df2a22480b7268/php/Antlr4ng/PhpLexerBase.ts#L48)--I have to hardwire the number. This is brittle code, and unnecessary.

[ericvergnaud]

Can you provide an example of the generated code ?

[kaby76]

php_lexer.go.txt PhpLexer.cs.txt PhpLexer.dart.txt PhpLexer.g4.txt PhpLexer.h.txt PhpLexer.java.txt PhpLexer.js.txt PhpLexer.py.txt PhpLexer.ts.txt PhpLexer.php.txt PhpLexer.swift.txt

(Files generated via antlr4 -Dlanguage=whatever PhpLexer.g4.)

The targets that have a constant defined for SkipChannel: Cpp, CSharp, Dart, Go, Java, JavaScript, PHP, Python3, Swift.

The targets missing a constant definition for SkipChannel: TypeScript.

$ grep SkipChannel *.txt | grep 4
php_lexer.go.txt:       PhpLexerSkipChannel = 4
PhpLexer.cs.txt:                PhpComments=2, ErrorLexem=3, SkipChannel=4;
PhpLexer.dart.txt:    PhpComments = 2, ErrorLexem = 3, SkipChannel = 4;
PhpLexer.g4.txt:    SkipChannel
PhpLexer.g4.txt:PHPStart       : PhpStartFragment     -> channel(SkipChannel), pushMode(PHP);
PhpLexer.g4.txt:PHPStartInside     : PhpStartFragment     -> channel(SkipChannel), pushMode(PHP);
PhpLexer.g4.txt:PHPStartInsideQuoteString     : PhpStartFragment     -> channel(SkipChannel), pushMode(PHP);
PhpLexer.g4.txt:PHPStartDoubleQuoteString     : PhpStartFragment     -> channel(SkipChannel), pushMode(PHP);
PhpLexer.g4.txt:PHPStartInsideScript     : PhpStartFragment     -> channel(SkipChannel), pushMode(PHP);
PhpLexer.g4.txt:Whitespace        : [ \t\r\n]+    -> channel(SkipChannel);
PhpLexer.g4.txt:SingleLineComment : '//'          -> channel(SkipChannel), pushMode(SingleLineCommentMode);
PhpLexer.g4.txt:ShellStyleComment : '#'           -> channel(SkipChannel), pushMode(SingleLineCommentMode);
PhpLexer.g4.txt:    '{' { this.IsCurlyDollar(1) }? { this.SetInsideString(); } -> channel(SkipChannel), pushMode(PHP)
PhpLexer.g4.txt:CommentEnd              : [\r\n] -> channel(SkipChannel), popMode; // exit from comment.
PhpLexer.h.txt:    PhpComments = 2, ErrorLexem = 3, SkipChannel = 4
PhpLexer.java.txt:              PhpComments=2, ErrorLexem=3, SkipChannel=4;
PhpLexer.js.txt:PhpLexer.SkipChannel = 4;
PhpLexer.php.txt:               public const PhpComments = 2, ErrorLexem = 3, SkipChannel = 4;
PhpLexer.py.txt:    SkipChannel = 4
PhpLexer.swift.txt:     static let PhpComments=2, ErrorLexem=3, SkipChannel=4

[ericvergnaud]

Noted, thanks!

Le 2 avr. 2024 à 00:46, Ken Domino @.***> a écrit :

php_lexer.go.txt https://github.com/antlr/antlr4/files/14829164/php_lexer.go.txt PhpLexer.cs.txt https://github.com/antlr/antlr4/files/14829165/PhpLexer.cs.txt PhpLexer.dart.txt https://github.com/antlr/antlr4/files/14829166/PhpLexer.dart.txt PhpLexer.g4.txt https://github.com/antlr/antlr4/files/14829167/PhpLexer.g4.txt PhpLexer.h.txt https://github.com/antlr/antlr4/files/14829168/PhpLexer.h.txt PhpLexer.java.txt https://github.com/antlr/antlr4/files/14829169/PhpLexer.java.txt PhpLexer.js.txt https://github.com/antlr/antlr4/files/14829170/PhpLexer.js.txt PhpLexer.py.txt https://github.com/antlr/antlr4/files/14829171/PhpLexer.py.txt PhpLexer.ts.txt https://github.com/antlr/antlr4/files/14829172/PhpLexer.ts.txt PhpLexer.php.txt https://github.com/antlr/antlr4/files/14829202/PhpLexer.php.txt PhpLexer.swift.txt https://github.com/antlr/antlr4/files/14829207/PhpLexer.swift.txt (Files generated via antlr4 -Dlanguage=whatever PhpLexer.g4.)

The targets that have a constant defined for SkipChannel: Cpp, CSharp, Dart, Go, Java, JavaScript, PHP, Python3, Swift.

The targets missing a constant definition for SkipChannel: TypeScript.

$ grep SkipChannel *.txt | grep 4 php_lexer.go.txt: PhpLexerSkipChannel = 4 PhpLexer.cs.txt: PhpComments=2, ErrorLexem=3, SkipChannel=4; PhpLexer.dart.txt: PhpComments = 2, ErrorLexem = 3, SkipChannel = 4; PhpLexer.g4.txt: SkipChannel PhpLexer.g4.txt:PHPStart : PhpStartFragment -> channel(SkipChannel), pushMode(PHP); PhpLexer.g4.txt:PHPStartInside : PhpStartFragment -> channel(SkipChannel), pushMode(PHP); PhpLexer.g4.txt:PHPStartInsideQuoteString : PhpStartFragment -> channel(SkipChannel), pushMode(PHP); PhpLexer.g4.txt:PHPStartDoubleQuoteString : PhpStartFragment -> channel(SkipChannel), pushMode(PHP); PhpLexer.g4.txt:PHPStartInsideScript : PhpStartFragment -> channel(SkipChannel), pushMode(PHP); PhpLexer.g4.txt:Whitespace : [ \t\r\n]+ -> channel(SkipChannel); PhpLexer.g4.txt:SingleLineComment : '//' -> channel(SkipChannel), pushMode(SingleLineCommentMode); PhpLexer.g4.txt:ShellStyleComment : '#' -> channel(SkipChannel), pushMode(SingleLineCommentMode); PhpLexer.g4.txt: '{' { this.IsCurlyDollar(1) }? { this.SetInsideString(); } -> channel(SkipChannel), pushMode(PHP) PhpLexer.g4.txt:CommentEnd : [\r\n] -> channel(SkipChannel), popMode; // exit from comment. PhpLexer.h.txt: PhpComments = 2, ErrorLexem = 3, SkipChannel = 4 PhpLexer.java.txt: PhpComments=2, ErrorLexem=3, SkipChannel=4; PhpLexer.js.txt:PhpLexer.SkipChannel = 4; PhpLexer.php.txt: public const PhpComments = 2, ErrorLexem = 3, SkipChannel = 4; PhpLexer.py.txt: SkipChannel = 4 PhpLexer.swift.txt: static let PhpComments=2, ErrorLexem=3, SkipChannel=4 — Reply to this email directly, view it on GitHub https://github.com/antlr/antlr4/issues/4572#issuecomment-2030705278, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAZNQJAJN7Z5W6GV2M3UOVLY3HPUFAVCNFSM6AAAAABFRND5QGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZQG4YDKMRXHA. You are receiving this because you commented.