Custom colours for declarations and uses of local variables

[negora]

Hello:

I think that it would be very useful if we could configure different colours for the declarations and uses of local variables. This feature is available when editing code in Java and JavaScript, and it helps a lot to identify, at first sight, which variables are declared in each method or block of code.

Will you be so kind to add this feature, please?

Thank you.

[negora]

I'm sorry, but I didn't remember that I had opened a similar issue before (#119). Please, close that one.

Today I've downloaded the source code of nbts in an attempt to check if I could help with the colouring of the local variables, method calls, etc. It has taken some extra-steps to build the project, though. I've had to make these changes:

• Upgrade the version of the JDK and the source code from version 7 to 8. The reason is that some parts of the code use the String#join() method.

• Add some modules that were missing: org.netbeans.api.progress.nb, org.netbeans.modules.csl.types, org.netbeans.modules.parsing.indexing and org.openide.util.ui.

• Modify the build.xml file to point to my installation of TypeScript. This step was expected.

That way, I've been able to compile the project.

Later, I've looked into the source code, but my knowledge about the NetBeans Platform, and about lexers and parsers, is scarce. So I've had to read some articles about the latter. My conclusion is that, to achieve what I want, a lexer is not enough: I need a parser that gives me the AST (abstract syntax tree), to know the context and differentiate what's a local variable, what's a method call, etc. Am I right?

Checking the code, I've seen some classes that seem to form the lexer of nbts, but I've not been able to find out where and how the parser takes part. I believe that the TypeScript platform offers a server to handle the code parsing (among other services), so that IDEs have not to re-invent the wheel. But again, I've no idea how it works. Any clue, please?

Thank you.

[Chris2011]

That feature would be great to have. I really would like to help but my knowledge about lexer and parser is same as your level.

[negora]

I've continued reading about this topic today (very interesting, by the way), and I can confirm that we need a parser. But not any parser: we need one that is able to build the AST even though the source code contains errors. Indeed, that will be the most common situation, because the user will be editing the code while the highlighting is in progress. Check this, specially the last answer: https://stackoverflow.com/questions/44484852/is-it-advisable-to-use-tokens-for-the-purpose-of-syntax-highlighting .

On the other hand, I've been wondering how NetBeans highlights the Java code. I remember that they use a custom version of javac for the compilation. This version is able to compile code that contains errors or is incomplete; otherwise, a simple error in one file would make the file completely uncompilable and would cause lots of errors in cascade in other files. It would also be unable to mark multiple errors in a single file (it would stop in the first one).

So I guess that they use the parser of this modified javac for the highlighting. That may be the reason why NetBeans offers such a super-customizable colouring configuration.

Can we easily get the AST from a TypeScript file? Well, 2 years ago I played with the samples shown here: https://github.com/Microsoft/TypeScript/wiki/Using-the-Compiler-API . One of them is called Traversing the AST with a little linter and shows you how to inspect the whole AST. Is it tolerant to errors in the code? I didn't test it but I think so.

The problem is: How would we connect this TypeScript code with nbts? I guess that we would need to create a server in TS and make nbts communicate with it (using the network, a named pipe or other type of bus). But there's already "something" that provides services like that, called tsserver, which comes bundled with TypeScript. Indeed, one of these services is code colouring. So the first step would be to test that service and check whether it allows to do what we want. I'll take a look.

@Chris2011: I guess that our first step should be learning about the NetBeans Platform (what parts make it up, how they work, etc.) Without that, I guess that any further step would be (almost) useless. In my case the problem is that I've not much spare time :S . But I would love learning and being able to extend my favourite IDE.

[Chris2011]

Short answer, nbts uses the Language Server Protocol (LSP) of TypeScript.

[negora]

@Chris2011 But also for the syntax highlighting? I've checked the source code of nbts and there's a class that contains references to each type of token, which leads me to think that use its own lexer. But I've no idea.

[Chris2011]

For this, I think @jamie-everlaw or @jeffrey-easyesi can help here out to answer that question.

[Chris2011]

From the last days, I learned that LSP can't handle syntax highlighting yet. But there is a proposal for smth like that: https://github.com/Microsoft/vscode-languageserver-node/pull/367. So you need to do smth else. What I know, back the days, when I had a look into the plugin, they copied the JS tokens from the JS implementation of NetBeans, because the API for getting the tokens is not public. I don't know whether they used JavaCC or ANTLR.

[Chris2011]

Kind of own lexer, so used the JSTokens from NetBeans and extends them.