search

antlr / antlr4

ANTLR (ANother Tool for Language Recognition) is a powerful parser generator for reading, processing, executing, or translating structured text or binary files.
http://antlr.org
BSD 3-Clause "New" or "Revised" License
17.26k stars 3.29k forks source link

[Python3] TokenStreamRewriter: Interval missing start and stop fields #2020

Open ivan-gavran opened 7 years ago

ivan-gavran commented 7 years ago

In TokenStreamRewriter, the function getText() tries to access fields start and stop of provided interval argument

start = interval.start
stop = interval.stop

However, it seems that interval is an ordinary tuple. Therefore, what works is

start = interval[0]
stop = interval[1]

I also checked in the rest of the codebase, and the suggested fix was done in e.g. BufferedTokenStream. Does this change make sense?

ericvergnaud commented 7 years ago

mmm... in Python3 Interval is a range (with start and stop fields), so while your proposal might work, it would hide the underlying issue.

ivan-gavran commented 7 years ago

Could it then be that I was passing the wrong type of argument to function getText()? So, what I did was (in short) (in a listener callback, with rewriter being an instance of TokenStreamRewriter)

intervalVar = ctx.getSourceInterval()
//some rewriting
rewriter.getText("default", intervalVar)

However, if not intervalVar, then what should've been given as an argument to getText function?

ericvergnaud commented 7 years ago

Hi,

looks like getSourceInterval returns a tuple, which I believe to be a bug (need further digging to be sure). Please keep this issue open so I can fix it when I have time. In the meantime, I’d suggest you convert the result to a range when passing it to getText

Eric

Le 22 sept. 2017 à 01:02, gergia notifications@github.com a écrit :

Could it then be that I was passing the wrong type of argument to function getText()? So, what I did was (in short) (in a listener callback, with rewriter being an instance of TokenStreamRewriter)

intervalVar = ctx.getSourceInterval() //some rewriting rewriter.getText("default", intervalVar) However, if not intervalVar, then what should've been given as an argument to getText function?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/antlr/antlr4/issues/2020#issuecomment-331218794, or mute the thread https://github.com/notifications/unsubscribe-auth/ADLYJLjs2PgmnKeH83taptVX_K7LYAjwks5skpYvgaJpZM4Penmf.