Shift to Language Server Protocol

[tajmone]

@thoni56:

I'm on a long-time quest to resurrect this code to understandable, maintainable and developable standard. This is hard, difficult and time consuming, as it is almost impossible to understand what anything does. At this point I've added a couple of smoke/functional tests, that at least should crash if something basic becomes broken.

I think that a good move would be to reimplement the server part using Language Server Protocol for this would jail-break Xref from being an Emacs only tool and allow using it with any modern editor — which would, in turn, most probably raise awareness on the tool and attract collaborators to it.

There are already numerous LSP Implementations in the wild, and the protocol is supported by an increasing number of editors (including Emacs).

I gather you've looked deeply into Xref source code, so I was wondering if you think that this is doable. Is the server part of the tool coded in a way which would make adapting it to LSP possible?

I'm quite confident that moving Xref to LSP would quickly attract users and contributors from editors like Sublime Text, VSCode, Atom, Vim, and many others, which could really make a big difference in the interest shown for this tool.

As I'm almost dependent on this and refactoring tools for C is sadly lacking, I decided to pick this up and work a bit on it, maybe even make it a bit more visible, by moving it to GitHub.

You're right, there seems to be a serious lack of free refactoring tools for C, which strenghten my convinction that making Xref an LSP-tool would hugely boost its visibility. Once Xref would support LSP, integrating it into other editors via a custom extension/plugins is going to be quite easy, for there are plenty of LSP extension templates available for all the major editors.

It really looks like MicroSoft has come up with a good and successfull idea in regard to LSP, and its VSCode editor is gaining a growing userbase.

See also:

• https://langserver.org/

[thoni56]

This is an extremely interesting suggestion! I've recently been using VS Code more and more and started to think in this direction but have not pursued this further.

I'll keep the link to langserver for future initiatives. As always, any help is welcome, though be warned about the complexity of the source of this beast ;-)

A strategy for this would of course be to create tests for the functions end-to-end over the protocol somehow, and then starting to either implement an alternative protocol or an optional adapter.

[tajmone]

As always, any help is welcome, though be warned about the complexity of the source of this beast ;-)

I'll try my best, even if it's only trying to jot down some notes on how to approach an tame the beast. Fortunately, local branches allow testing without compromising the base code, so it's definitely worth trying to hack through this.

Also, I'm sure that once it's become clear that the shift to LSP is doable, and there is a groundplan on how to approach it, it should be fairly easy to envolve collaborators for the task from the forum communities of the various editors (Sublime Text, Atom, VSCode all have very active communities and a large numbers).

[thoni56]

Another thing I'm thinking of here is to partition the program into its logical parts, the cross-referencer, the langauge server, the refactoring server and the internal code generation tool.

It looks like that when discovering that they could parse and cross-reference C- (and Java) code they bolted a lot of different functionalities on top of that.

A split, with common components of course, would at least make sense of all the various command line options...

[tajmone]

Lately I've planning to get actively involved in this, and even started gathering more material on code refactoring and digging into the topic — it's just that I haven't find the time to setup Emacs on my machine, and to start working with Emacs and this tool (never used Emacs before). Unless I see the tool in action I can't really get to work on this.

But definitely the code look a bit tangled, and the whole idea of separating it into clear-cut units is a prerogative for a good LSP.

I'm quite amazed that there are still so few code refactoring tools available today, yet many great books on the topic. And I'm sure that turning this into an LSP would really draw attention on it.

Tomorrow I could cut out a morning to start setting up Emacs and c-xrefactory on my machine — if I encounter any problems I might email you for some help.

[tajmone]

A couple of years have passed since this Issue was created, and the LSP protocol has been evolving fast in the meantime, introducing new features, many of which are targeting refactoring.

Furthermore, the LSIF standard (Language Server Index Format) has been actively developed alongside LSP — I think LSIF didn't exit when this Issue was created.

The purpose of the Language Server Index Format (LSIF) is it to define a standard format for language servers or other programming tools to dump their knowledge about a workspace. This dump can later be used to answer language server LSP requests for the same workspace without running the language server itself. Since much of the information would be invalidated by a change to the workspace, the dumped information typically excludes requests used when mutating a document. So, for example, the result of a code complete request is typically not part of such a dump.

The LSIF standard could also be applicable here, and might provide a better alternative to whatever way c-xrefactory uses to store its data — even if used in parallel (initially or for good) it would definitely help an LSP implementation, and communicating with other tools, since both LSP and LSIF are gaining ground as language- and editor-agnostic tools.

My understanding is that refactoring is now becoming a major goal of LSP, although due to the technical challenges of implementing Language Servers (speed and fault tolerance) language servers are not proliferating at the same rate as simpler syntax support package do (i.e. RegEx based syntax support, as in TextMate, etc.).

There still isn't a pure C implementation of LSP, but we now have C Lang Server which is also a C-usable SDK at least:

ccls (Apache-2.0 License)

• https://github.com/MaskRay/ccls

C/C++/ObjC language server supporting cross references, hierarchies, completion and semantic highlighting.

ccls uses the Clang+LLVM C++ libraries to parse your source code, so it will recognize all language features supported by Clang.

Being able to hook c-xrefactory into the LSP protocol and interact with C Lang Server should result in new refactoring possibilities, since the context awareness of the Lang Server can then exploited in that respect. I wonder (and have no idea whatsoever) how the addition of LSP could help the task of self-rafactoring c-xrefactory itself — but once you link them together, it would be a good assumption that they should be able to feed off each other, resulting in complementary capabilities and functionality, eventually leading to a fusion into a single Language Server with refactoring capabilities.

[tajmone]

Pure-C LSP Implementation

I've just across the minic-lsp Language Server for miniC. It's entirely written in C, so it contains a full LSP implementation within the source code ( GPL-3.0 License):

https://github.com/BojanStipic/minic-lsp

It's a stable product, and it looks well done. Unlike the other LSP SDK for C that I linked (which was in C++), this one is purely in C, which should make it better for integration with this project.

[thoni56]

Thanks for this link. It surely will be a great resource when I get started on the LSP integration.