Feature Request: Language Server Protocol for AGS Script

[persn]

The Language Server Protocol (LSP) is a way to develop a tool for code completion, and various other language analysis mechanisms used for productive coding. The problem that I would like to be solved is that in today's situation each existing text editor would have to develop their own custom tooling for language analysis, which probably isn't going to happen. This handicap is tragic for someone that would rather code in Visual Studio, Visual Studio Code, Atom, Vim, eMacs or whatever. However by using LSP we could develop a language server that people could use with any text editor (assuming it is compatible with LSP)

• [ ] Develop a language server

• [ ] Change the editor to support getting language analysis from the server

• https://microsoft.github.io/language-server-protocol/

• https://langserver.org/

[morganwillcock]

Whether sticking with the existing editor or not, it would be easier if the editing environment can be desynced from the actual engine versions, so this seems a good idea to me. I guess what is currently missing is a decision on whether the actual scripting language itself will stick around and be improved, or whether a generic scripting language is added instead (which might remove the need for a custom language server). But regardless, to me a modular approach seems a good option...

[ericoporto]

I found this a helpful write up of building a language server in C# -> https://martinbjorkstrom.com/posts/2018-11-29-creating-a-language-server | blogpost_bkp.zip

I also briefly experimented with C#, Tree Sitter, AGS Script and swig : github.com/ericoporto/ags-cs-tree-sitter.

Two language servers written in C++

• https://github.com/MaskRay/ccls
• https://github.com/svenstaro/glsl-language-server

---

Did a sketch test here: github.com/ericoporto/ags-script-language-server

I think my first note is that LSP does NOT provide Syntax Highlighting, so a textmate grammar will still need to be written to AGS Script, even if a LSP is built.

My language server is very stupid for now, since I picked one for bash and simply changed the tree sitter behind it and did some minor cleaning along. Still need to switch the reserved keywords and builtins definitions.

• VSCode Publishing Extensions (requires server published on npm - in this particular case it's not)
• Tutorial for VSCode and Writing the LS
• VS Code Tree Sitter syntax highlighting, in case needed

[persn]

Did a sketch test here: github.com/ericoporto/ags-script-language-server

Bleh. Sorry to be negative but I want to mention once again that I don't like when front-end tech gets used to develop back-end products. This is a great place to ask ourselves if we want to use modern back-end coding languages like Rust or Haskell, alternatively the tree shaker parser appears to be written in C, so just use C or C++. Let's leave JavaScript, TypeScript, PureScript and anything else from that stack to web clients.

I think my first note is that LSP does NOT provide Syntax Highlighting, so a textmate grammar will still need to be written to AGS Script, even if a LSP is built.

Perhaps in the future https://github.com/microsoft/language-server-protocol/issues/18 But for now a textmate grammar yeah

And as mentioned, we should probably decide if we want to keep AGS script before any significant effort is put into this.

[ericoporto]

c.tmLanguage.json.zip : This is NOT WORKING, but the TextMate can be something like this, a C grammar with assembly and some other features removed, and the import and export keywords added. It needs more things, but just a sketch. I will try to work on having something working on github.com/ericoporto/agsscript-textmate, but will take a little while. I will work from scratch using the above as reference. Figuring the differences between C and AGSScript may be useful to get a TextMate grammar working fast. Some useful resources:

• https://macromates.com/manual/en/language_grammars
• https://www.apeth.com/nonblog/stories/textmatebundle.html
• https://pragprog.com/book/textmate/textmate (book is out of print, had to get it in an alternate way)

Bleh. Sorry to be negative but I want to mention once again that I don't like when front-end tech gets used to develop back-end products. This is a great place to ask ourselves if we want to use modern back-end coding languages like Rust or Haskell, alternatively the tree shaker parser appears to be written in C, so just use C or C++. Let's leave JavaScript, TypeScript, PureScript and anything else from that stack to web clients.

About the TypeScript experiment, it's just an experiment. Can it be done on C or Cpp? Yes. Let's consider once something is working it can be ported to C and ported back to TypeScript with Emscripten - it's how the parser was ported there.

Change the editor to support getting language analysis from the server

This is something that could be experimented, if this is the same refactoring features in VSCode, it implies getting working there (at least for a single .asc script) and then porting to AGS Editor - where the full context is available, the characters script name, the defines generated for views and plugin headers.

Ah, about LSP, most Editors I checked actually required some sort of client to interface with LSP instead of delivering on the promise of connecting directly to the LSP. It would be useful if there was an Editor that did this in a generic way and with some debug tooling to visualize the messages to see what's working and what's broken.

[persn]

c.tmLanguage.json.zip : This is NOT WORKING, but the TextMate can be something like this, a C grammar with assembly and some other features removed, and the import and export keywords added. It needs more things, but just a sketch. I will try to work on having something working on github.com/ericoporto/agsscript-textmate, but will take a little while. I will work from scratch using the above as reference. Figuring the differences between C and AGSScript may be useful to get a TextMate grammar working fast. Some useful resources:

edmundito already wrote something here in coffescript which can probably be translated to json, but I'm not sure if it's TextMate or something else https://github.com/edmundito/language-ags-script/blob/master/grammars/tree-sitter-ags-script.cson

About the TypeScript experiment, it's just an experiment. Can it be done on C or Cpp? Yes. Let's consider once something is working it can be ported to C and ported back to TypeScript with Emscripten - it's how the parser was ported there.

Cool, thanks for the verification. It would be neat if the experiment and research discovered how many logical components that can be individually developed all of this can be separated into. We already know that the parser, syntax highlighter and the server are 3 so far.

[ericoporto]

The Edmundito project you linked is the Atom extension, thanks, I forgot about that, there's definitely stuff to pick from there! The parser I am using is also written by him, it's the tree sitter parser here! It mostly works, with some gotchas that are already reported there - these gotchas don't show up if you are using it in Atom - Edit: he fixed some of the gotchas!

I found also reserved symbols in another place: Editor/AGS.CScript.Compiler/Entities/DefaultTokenStore.cs

I looked into ags itself and scintilla and it appears it uses a modified cpp lexer ? But I could not follow the code yet.

[persn]

I looked into ags itself and scintilla and it appears it uses a modified cpp lexer ? But I could not follow the code yet.

Not sure how the whole Scintilla thing works. Should probably refactor the client to use the TextMate grammar instead, will probably need some bridge that converts the json to the lexer or whatever Scintilla needs. Alternatively it could be dropped if we're going to make a new client soonish(?).

[tzachshabtay]

Bleh. Sorry to be negative but I want to mention once again that I don't like when front-end tech gets used to develop back-end products. This is a great place to ask ourselves if we want to use modern back-end coding languages like Rust or Haskell, alternatively the tree shaker parser appears to be written in C, so just use C or C++. Let's leave JavaScript, TypeScript, PureScript and anything else from that stack to web clients.

Typescript (and javascript) is both a front-end a back-end tech, it's quite popular for backend (nodejs + express/koa/etc + typescript is a common modern stack these days) and a very viable choice for this type of project imo. If you want to offer criticism on choosing it I think it should be based on its merits and/or specific issues you have it in regards to a project of this type.