search

antlr / antlr4-intellij-adaptor

A library to support the use of ANTLR grammars in jetbrains IDE plugins for building custom languages.
BSD 2-Clause "Simplified" License
212 stars 38 forks source link

Give access to the original ANTLR Parse Tree #15

Open adesutherland opened 5 years ago

adesutherland commented 5 years ago

Totally understand why we should use the PSI tree, for example, to get all the inbuilt features like renaming and so on. And the ANTLR like Xpath feature works well. In sum, no complaints.

However, I do have a number of ANTLR walkers (for example that do more fine grained work around identifier type safety and other analysis). I really can't be bothered to re-do for PSI trees (and also not sure how to make a walker in PSI - I guess I could learn!).

So I think I could quite easily pump the whole file through the ANTLR Lexer and Parser in an external annotator, and could get row and column numbers to sync, and hence add some warning annotations etc. Although I can't find an example of how to access the file's contents the virtual file system seems to be the answer.

So finally (!) the question is this - can I access either the actual ANTLR (not PSI) parse tree from an annotator, or perhaps the token stream? I obviously have looked but can't decide if there is a suitable extension point or not. For example somehow when the PSI tree is built.

I am thinking it is not possible at the moment and I need to reparse the file. Is this right?

bjansen commented 5 years ago

The ANTLR plugin does something similar in ANTLRv4ExternalAnnotator. It retrieves the file contents, gives it to ANTLR, builds a list of issues and creates annotations based on a TextRange built from a token's start and stop index.

So you're right, you currently have to parse the file a second time.

If you have existing walkers, I suppose the best option to reuse them is implement an ExternalAnnotator and make the adaptor attach the ANTLR parse tree to the PsiFile. This way, you could use your walkers in doAnnotate. I'd have to add a new feature to the library to allow that.

I you don't have existing walkers, you can implement an Annotator, and IntelliJ will recursively walk the file for you. You will then be working on instances of PsiElement (probably even of ANTLRPsiNode). You won't have access to the original ANTLR parse tree nodes. I guess it would also be possible to make the adaptor attach the original parse tree node to each instance of ANTLRPsiNode.

adesutherland commented 5 years ago

Understand. As a feature suggestion I think it would be good to have the adaptor attach the ANTLR Parse Tree to the PsiFile. If I have time I will see if I can make the change.

Thanks again.

bjansen commented 5 years ago

I'm converting this issue to an enhancement then, just to keep track of the feature :)