CMake language server

[tambry]

Hi!

I'm developing my own CMake language support plugin for Visual Studio Code from scratch, as the TextMate grammar files used by twxs's plugin are ancient and not in line with modern CMake (no multi-line comments, no highlighting for newer properties, no deprecation classification, improper classification of control-flow statements, lackluster grammar definitions for certain constructs, etc).

While I have yet to release my plugin, I have noticed that your project seems to depend on twxs's plugin. I'd like my plugin to be able to also run alongside your plugin. What do you depend on from twxs's plugin? What would I need to do to make my plugin able to support your plugin?

BTW, love your plugin, keep up the good work! :)

[vector-of-bool]

The primary reason for the dependency is the cmake.cmakePath setting, which both plugins use to determine what cmake executable to use. I chose to add the dependency after a few issues were opened relating to both plugins providing that setting (See an early closed issue regarding the duplicate setting).

I'd ask that you consider contributing to twxs's language support. He's still accepting pull requests, and the only reason he doesn't have those newer properties highlighted is because it hasn't really come up.

[tambry]

I see why that was somewhat necessary. I think maybe a more sensible solution would be to have an option with a different name, but also support the option provided by him. This would allow an alternative language support plugin to work too.

I'm not sure if contributing to twxs's language supported is the right choice here. I find his code hard to read and from what I understand in a couple milestones it will be broken, if he doesn't fix it to use a new API. I also did open up 3 issues, 2 of them being for the grammar syntax. None of them received any feedback. I think it might be more sensible to have an alternative extension, that'll be properly maintained (for as long as I'll use CMake and VSC).
The issue also isn't about simply fixing a couple things about the grammar, it needs to be completely re-done. Currently his grammar simply highlights keywords no matter where they are, while mine aims to also understand scopes, how arguments work, what arguments are fit for what, provide much more detailed grammar, show deprecation notices, etc. I think re-doing the whole thing might be a better option at this point.

Thanks for the info and the opinion. If you have any other ideas, then I'd love to hear them.

[vector-of-bool]

Hmm... Would you be interested in collaborating on such new language support? I've felt for a while that it is a bit troublesome to have two extensions for CMake support. Perhaps the language support could be merged into CMake Tools for a more complete CMake experience.

At present, I'm working on a fairly large refactor to support the new cmake-server mode in CMake 3.7, which will help facilitate things like language and build system introspection. I think having newer, better, and more uesful language support will be an excellent addition. Simple things like completion of custom variables and properties is someting I've long desired in VSCode.

[tambry]

That is something that I would be interested in. I think it would be a great idea to have a newer language extensions that support everything.

In such a case it might make sense to open up an organization. What should be the organization's name? I currently grabbed CMake organization name. Maybe it might even make sense to move to GitLab. CMake Tools doesn't necessarily refer to CMake language support itself, but rather tools for CMake, so it might make sense to rename to simply CMake.
Also, CMake server mode supports sounds very useful.

[vector-of-bool]

An organization might be useful.

I've always wanted enhanced native support in VSCode (I find the existing C and C++ extensions severely lacking), and I think that being able to introspect a CMake project will open up a lot of new avenues for more tooling. I have another (very slowly advancing) project for a new Clang-based language service that would integrate tightly with this extension to obtain exact compile options and include directories, which is something severely lacking in other C and C++ extensions.

Perhaps an organization specifically for such native tooling would be useful? Or maybe something a little more laser-focused on CMake support?

[vector-of-bool]

There was a discussion a few months ago regarding a possible merge between this extension and twxs's extension. He said that he was anticipating implementing a new feature, but I haven't heard from him since. It is very well possible that he is no longer actively maintaining or working on the extension. It may be prudent to have some extension pick up on language support. The completion suggestions from twxs's extension are pretty good, but lacking in some regards. I'd like to see improvements, and I'm excited to see what kind of things can be done in this area.

[tambry]

Sorry for the late response. I think tight integration for even languages like C/C++ could be cool. Maybe you disabled exceptions using a compiler switch? Then it should probably noted on the interface. This would be of course made much easier, if the project was using CMake, so we could obtain the options through CMake server.
Also supporting other languages could be interesting and something that I'm interested in. :)

I might also note for CMake I'm kinda blocked by an upstream VSCode issue, that I'd prefer to not workaround.

I guess the next step would be to meet up somewhere sometime and discuss this the details, approach, etc. Would doing this over IRC (probably on freenode) be OK with you?

[vector-of-bool]

Communicating in real-time may prove difficult based on time zone differences, but may be possible. Email may be sufficient, or maybe I should open a gitter community? I've not used gitter, but I've seen it be pretty popular in the open source world.

[tambry]

@vector-of-bool I've never used Gitter before either. Not sure if I'd be comfortable using it. I'd strongly prefer to use Discord (or IRC), if that is fine with you. Since you mentioned timezones, I'm in the GMT+2 timezone, what's your timezone?

If Discord seems appealing, then here's a server invite: https://discord.gg/xtUX72r

[vector-of-bool]

I'm currently in UTC-7. I think IRC will be much more convenient.

[tambry]

Then let's use freenode and the channel #vscext temporarily until we can decide on a name. I'll be on there whenever I'm online.

[vector-of-bool]

Rather than make a new channel, you can just message me with /msg vector-of-bool. I'll also be hanging around #cmake and ##c++.

[vector-of-bool]

Hi @tambry. It's been nearly a month, but I'm still here. I haven't popped onto IRC for a while, but I think the timezone difference is a bit of a killer for real-time communication. I'm still eager to work on a refactored version of CMake support with greater language support and semantic analysis. Are you still interested in collaborating on such a project?

The CMake Server stuff is still up in the air because of some latent CMake issues that are preventing me from doing a supportable release.

[tambry]

Hi!

I'm still interested, but I'm afraid that at this time creating a better grammar is very hard due to the technical limitations of the TextMate grammar format, that I've discovered.
One of the main limitations of TextMate format is that it is unable to match things over multiple lines. Having things on multiple is quite common in CMake and this would prevent accurate highlighting (or highlighting at all), when things are split on multiple lines. I learned about this limitation after submitting an issue here. (Technically it is possible to work around this limitation)
I should note that the original TextMate bug I mentioned a while ago should be getting fixed hopefully for the next release.

Since VSCode itself uses Moncao, which has its own grammar format Monarch, it would probably be the best alternative, since it doesn't have any of the limitations that I have encountered so far in TextMate. Unfortunately VSCode doesn't support Monarch, but it would be possible to add support for it. There's a feature request for support for a better grammar format (or support for the ability to parse incrementally).

At this point I don't have the motivation to workaround the limitations of the TextMate format, but I would be very happy to write a much better grammar in Monarch or an incremental parser, if VSCode were to get support for one of those.

[vector-of-bool]

Sounds good. Besides an improved grammar, would you be interested in collaborating on other editing improvements? I feel like work can still be done to improve the editing experience, even while waiting on improved grammar support.

[tambry]

I won't probably be of much use on that front, as I'm not a JavaScript nor a TypeScript developer. I'll probably stick to developing the grammar, once that becomes possible (which might unfortunately take a while).

[vector-of-bool]

That's all good. I think I may delve into some more language support for a 1.0 release, which is within sight. I'm not quite sure what it will take to get there, I haven't started experimenting yet, but I'm going to be working on some bugs and features for 0.8.8 and then see where I can go from there.

[MathiasMagnus]

I just want to add that such project introspection would be most welcome by C++ developers using CMake and VSCode. Unfortunately, big brother VS has this feature and makes like SO MUCH simpler, therefore I tend to pick that editor for my daily tasks. However, its speed can be aggravating sometimes.

[tambry]

@MathiasMagnus If you mean language support, then it would be nice if you could go and upvote this feature request. 😃

[MathiasMagnus]

That part is the smaller issue. It is vexing that there is no "IntelliSense" for editing CMake scripts, but what is far more annoying, is that one manually has to set include directories and compiler defs, because the editor does not know how the build actually goes. The VS integration invokes CMake in server mode and they extract the necessary info and generate an MSBuild file behind the scenes, and at this point you're back at native Visual Studio developer experience. This is the bigger problem that is missing from the VS Code CMake experience.

[vector-of-bool]

I'm hoping that for a 1.0 release of this ext I'll have enhanced intellisense functionality (And maybe even a CMake debugger? That would require CMake changes, though, which I've been looking into contributing).

I recently talked with some of the VSCode maintainers, and they expressed some interesting in contributing to this extension to further CMake support in VSCode. It really boils down to the fact that I am busy IRL and can't get all the things I want into this extension just on my own (Not enough time in the day).

[navidR]

Any news on this one?

[MathiasMagnus]

I too would welcome the idea of being able to debug CMake scripts. It would make it much easier to teach CMake to C/C++ freshmen. It'd also come in handy when discovering the build process of humongous projects like the LLVM project.

[otreblan]

Any news on this one?

There is a cmake language server made in python. https://github.com/regen100/cmake-language-server

[zweimach]

Any news on basic syntax highlighting?

[bobbrow]

@zweimach this work has not been scheduled yet. In the meantime, we recommend you use the extension called just CMake by twxs. It does a fine job with the language server.

[KaungZawHtet]

@bobbrow Not so fine here. I had to download CLion trial version to format a cmake script.

[vakokako]

@bobbrow that extension is not maintained and the issues we have with it are not being fixed :(

[bobbrow]

I'm sorry to hear that. This work still has not been scheduled, but you can help raise the priority by adding your 👍 to the first comment in this issue. That is how we track community interest.

[ruilvo]

I'm adding my comment in addition to the 👍 to say how this feature is so much needed! twxs's extension isn't just up to the job anymore. I've fixed some of the dictionary on a fork I did, but I really needs a rewrite. This feature is a must have! Also, #534. You should count the sum of both!

[ruilvo]

Also, @otreblan's suggetion of an already existing cmake lang. server could be a great starting point. The LS extension guide makes it seem easy... I'm sure any developer of this extension , which good knowledge in TypeScript can get a proof-of-concept done in couple hours...

[josetr]

Hi everyone. I just released my CMake Language Support extension for VSCode which tries to address most of the issues found in twxs.vs.language.cmake.

Download link: https://marketplace.visualstudio.com/items?itemName=josetr.cmake-language-support-vscode Repository: https://github.com/josetr/cmake-language-support Interactive Preview: https://josetr.github.io/cmake-language-support

I'll be paying close attention to try to address any issues found quickly so please report them in the repository.

Any feedback and feature request is also welcome, and if you found the extension considerably better than what you used before, and you got some time, please leave a review in the Marketplace or a star in the repo so that this extension can reach more people.

Thanks a lot and I hope you guys find it useful.

Edit: tagging users that may need this as they've reported issues in the vs.language.cmake repo that have not been resolved in years.

@rhaschke @Yuwahyu @fuchanglin @stalek71 @mcandre @TempleLin @josephgarnier @liberix @Sunshine334419520 @Xkinler @pratikpc @ruilvo @krasimirkrustev @capra314cabra @ouuan @mfoco @ToweR10 @SoftwareApe @me21 @gpxricky @M-Pixel @dbird137 @am11 @aey76 @maixuanhan @fdk17 @Zingam @rajedan @darthwalsh @schultetwin1 @kymikoloco @wikinee @tambry @TankTheFrank @omicronns @zethon

[ruilvo]

@josetr thanks for the project! will the project be open source?

[Zingam]

@josetr Thanks! I'll definitely try it. Yet I will ask still: why don't you contribute this work as part of vscode-cmake-tools so that we have a single source of truth and more people could help deveop and maintain it.

[josephgarnier]

@josetr Thanks! I'll definitely try it. Yet I will ask still: why don't you contribute this work as part of vscode-cmake-tools so that we have a single source of truth and more people could help deveop and maintain it.

The two projects are not intended for the same purposes and do not have the same level of abstraction. twxs.vs.language.cmake is a plugin intended to offer syntax highlighting and the basic features needed to develop a CMake project, not a C++ project, but modules, CMakelist, etc. vscode-cmake-tools is rather intended for developers who don't have a very complex workflow and especially who don't need to customize it. The two plugins are complementary, not in competition

[josetr]

thanks for the project! will the project be open source?

@ruilvo I haven't decided that yet but there are a few things that I'd need to do to make that happen. Making sure that the source code is in a state where it's extremely easy to contribute would be a priority, there are some design ideas that I haven't made yet, and the documentation explaining the workflow to properly build, test, and contribute is also missing.

@josephgarnier has given a good explanation. The two extensions do not compete with each other and most importantly, wouldn't share any code if they were merged as my project is mostly written in a different language, C#.

My project is mainly a set of C# libraries to handle everything related to the CMake Language, includes a class that implements the Language Server Protocol, and just happens to support VSCode but I could have chosen another editor.

For example, I could easily add support for VS2019 / VS2022 tomorrow if I wanted to but I haven't due to this statement in their LSP documentation:

https://docs.microsoft.com/en-us/visualstudio/extensibility/adding-an-lsp-extension?view=vs-2022 The intent of the LSP and support for it in Visual Studio is to onboard language services that are not part of Visual Studio product. It's not intended to extend existing language services (like C#) in Visual Studio.

Visual Studio already provides a CMake language service.

Yet I will ask still: why don't you contribute this work as part of vscode-cmake-tools so that we have a single source of truth and more people could help develop and maintain it.

@Zingam Do note that just giving control of the source code to Microsoft has its pros and its cons. The main con is that I would lose complete control of the repository, as they do not give important permissions such as the merge permission to non-Microsoft employers which again is understandable but makes my life much harder.

The second con is that they are (rightfully) more strict and it just takes much longer to get things done and let me give you an example:

2016 Visual Studio 2017 introduces built-in support for handling CMake projects

2022 Visual Studio 2022 CMake language support is still nearly the same, flawed, and missing a lot of useful features that mine provides. I'm not talking about the tools that they provide similar to CMake Tools. They've made a lot of progress there.

At the end of the day, CMake Tools can always replace their twxs dependency with mine if they want to. That's up to them. Do note that I'm not using the cmake.cmakePath setting but I can add it.

[bobbrow]

@josetr cool stuff! Do note that problems can arise with multiple extensions contributing settings with the same name so it is preferred to create your own so as not to conflict in case the acceptable values schema ever changes. (I don't think it ever would for something like cmake.cmakePath, but you get the idea). And our "dependency" on the twxs extension is a soft dependency (e.g. it can be uninstalled).

[josetr]

@bobbrow My plan is to detect CMake Tools and use two of its settings:

cmake.cmakePath: to optionally get the latest CMake documentation because by default my extension already comes with the documentation embedded in.

cmake.buildDirectory to optionally write a query file and support features like Go To Definition across multiple files and avoid having to run cmake.exe on save my self and just let CMake Tools do it.

So one of the goals is to have my extension just work without the user having to configure multiple settings.

[bobbrow]

For cmake.buildDirectory I would recommend calling our command directly instead of trying to read the setting because we support variable and command syntax which you probably don't have the code to expand right now (and it would be better if you didn't copy it).

Something like this:

vscode.commands.executeCommand('cmake.buildDirectory', workspaceFolder /* for multi-root workspaces you pass in the vscode.WorkspaceFolder - this can also be undefined if you don't want to support that*/);

[josetr]

@bobbrow

I was currently expanding only the workspace variables. Altho it's viable to support the rest, I agree that it's much better to use the command you just provided so thanks a lot for that info.

[Zingam]

Interesting! Us something actually happening in that area? What would be the benefits?

I hope we could get customizable formatting rules. I don't want enforced predefined auto-formatting which break my preferred style.

[ruilvo]

@Zingam, Microsoft now recommends this extension. It's not open source, but it's better than the one they recommended before as a language server. For formatting, I would recommend this extension. It needs that you install a python package, but it's very good and very costumizeable. It has extensible configuration, in yaml or json. Sometimes, just some little touches is all you want.

[H-G-Hristov]

Will the Language server handle syntax highlighting and replace the twx CMake extension? It will be also nice if CMake variables in strings are highlighted such as:

message(STATUS "Version: ${CMAKE_VERSION}!!!!!")

[gcampbell-msft]

Hi everyone, we would like to update you all on our plans to improve the Language Services for CMake (scripts).

We do not have feature deliverables for language services in the upcoming CMakeTools 1.17 release, which is just around the corner (shipping on 2/13).

However, we do have language service capabilities in Visual Studio, and we would like to further improve them and bring them over to CMake Tools, planning for CMake Tools 1.18. We would like to do this in an iterative fashion, so we deliver what we have first, first. We will create dedicated work items to track each of the features and link them in this Github issue.

We hope to continuously improve on the CMake Tools Language Services story, eventually providing full support with for: hover (quick info), completions, find all references, etc.

FYI, @tambry @H-G-Hristov @ruilvo @Zingam @josetr @navidR @MathiasMagnus and others!

[gcampbell-msft]

Closing this in favor of tracking this was the above referenced issues.