antlr / antlr4

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.
http://antlr.org
BSD 3-Clause "New" or "Revised" License
17.3k stars 3.3k forks source link

Facing the exact same issue on the import of either the Lexer or Parser generated from the JSON and Protobuf3 grammars from the antlr/gramars-v4 repo https://github.com/antlr/grammars-v4/tree/master/ #4293

Closed philipmiesbauer closed 1 year ago

philipmiesbauer commented 1 year ago
          Facing the exact same issue on the import of either the Lexer or Parser generated from the JSON and Protobuf3 grammars from the antlr/gramars-v4 repo https://github.com/antlr/grammars-v4/tree/master/
Traceback (most recent call last):
  File "/home/<user>/gitRepos/loki-external-interfaces/parse_json.py", line 3, in <module>
    from grammars import Protobuf3Parser
  File "/home/<user>/gitRepos/loki-external-interfaces/grammars/Protobuf3Parser.py", line 194, in <module>
    class Protobuf3Parser ( Parser ):
  File "/home/<user>/gitRepos/loki-external-interfaces/grammars/Protobuf3Parser.py", line 198, in Protobuf3Parser
    atn = ATNDeserializer().deserialize(serializedATN())
  File "/home/<user>/.local/lib/python3.10/site-packages/antlr4/atn/ATNDeserializer.py", line 61, in deserialize
    self.reset(data)
  File "/home/<user>/.local/lib/python3.10/site-packages/antlr4/atn/ATNDeserializer.py", line 91, in reset
    temp = [ adjust(c) for c in data ]
  File "/home/<user>/.local/lib/python3.10/site-packages/antlr4/atn/ATNDeserializer.py", line 91, in <listcomp>
    temp = [ adjust(c) for c in data ]
  File "/home/<user>/.local/lib/python3.10/site-packages/antlr4/atn/ATNDeserializer.py", line 89, in adjust
    v = ord(c)
TypeError: ord() expected string of length 1, but int found

The parse_json.py file uses only a custom version of the offical antlr documetnio demo for python3

import sys
from antlr4 import *
from grammars import Protobuf3Parser
from grammars import Protobuf3Lexer

def main(argv):
    input_stream = FileStream(argv[1])
    lexer = Protobuf3Lexer(input_stream)
    stream = CommonTokenStream(lexer)
    parser = Protobuf3Parser(stream)
    tree = parser.startRule()

if __name__ == '__main__':
    main(sys.argv)

Originally posted by @philipmiesbauer in https://github.com/antlr/antlr4/issues/4213#issuecomment-1567091592

philipmiesbauer commented 1 year ago

The Lexer and Parser were generated with the command

antlr4 -Dlanguage=Python3 grammars/*.g4

with antlr4 version 4.13.0

philipmiesbauer commented 1 year ago

Fault was on my side.

My imports should have been

import sys
from antlr4 import *
from grammars.Protobuf3Parser import Protobuf3Parser
from grammars.Protobuf3Lexer import Protobuf3Lexer

def main(argv):
    input_stream = FileStream(argv[1])
    lexer = Protobuf3Lexer(input_stream)
    stream = CommonTokenStream(lexer)
    parser = Protobuf3Parser(stream)
    tree = parser.startRule()

if __name__ == '__main__':
    main(sys.argv)