Support for multi-root workspaces

[danieltomasku]

As multi-root workspaces become more popular and widely adopted, it would be great if vscode-clangd could support this as well.

There are many use-cases for C/C++ projects where this would be useful (working with different hardware targets, working across third party libraries, etc.). I imagine there would need to be a fair amount of discussion on how to approach this. Some items for debate are whether the plugin should spawn an instance of clangd per root or whether it should be allowed for the client to specify multiple compilation databases.

Some relevant links to study on the LSP side are the vscode LSP multi-server example which illustrates how to start a server per wokspace folder, and the LSP specification for handling workspace folders. Feel free to discuss below any ideas for how to implement this!

[sam-mccall]

Clangd doesn't rely heavily on the workspace folder, but finds projects by looking for compilation databases relative to source files. It already supports multiple compilation databases, e.g. if you open files in multiple directories, each with compile_commands.json.

There are certainly some things that could be done better (e.g. indexes should probably be scoped to the current project), but what specifically would you like to work differently?

Is this specific to the mode where the CDB is overridden on the command line?

[benjaminhdavis]

The support for multiple roots does appear to work for me in my test cases (I've defined a macro in different roots and I can see it's resolving things correctly, which is really cool).

Is there anyway to change the relative search pattern? My project files tend to be in something like build_output/sha/compile_commands.json, and I'd love to not symlink them.

I'm also curious if there's anyway to deactivate clangd for specific roots?

[studgeek]

Where are the clangd indexes placed now? I don't see the " .clangd/index" that https://releases.llvm.org/10.0.0/tools/clang/tools/extra/docs/clangd/Installation.html describes either in my workspace or $HOME.

[HighCommander4]

I believe in recent trunk they've moved to .cache/clangd/index, to allow .clangd to be a config file.

[studgeek]

Ah, yes. I just saw that. I left them a suggestion to add that to the docs.

[studgeek]

I was working from the older https://releases.llvm.org/10.0.0/tools/clang/tools/extra/docs/clangd/Installation.html since that was the only place I found the index file location documented.

[tristan957]

I am having trouble with this. I have a multi-root workspace with a C project and a JNI/Java project. The C project resolves symbols perfectly and I have no complaints. The JNI/Java project doesn't find jni.h properly even though my build system is able to properly identify it.

I looked in the compile_commands.json file, and the include path is definitely there for the C file I am looking at.

If I open the JNI/Java project alone, clangd is able to properly read things.

[HighCommander4]

Can you share a clangd log?

[tristan957]

Actually it started working. I think I had a bad --compile-commands-dir value from my user settings that was affecting me.

[gsingh93]

Multi-root workspaces do seem to work in vscode, but the fact that indexes aren't scoped to a project can occasionally make it frustrating to use. Here's a simple example.

1. Create the following files and directories:

   dir1
   └── a.c
   dir2
   └── b.c

dir1/a.c:

int bar() { return 0; }
int foo() { return bar(); }

dir2/b.c

int baz() { return 0; }
int foo() { return baz(); }

2. Generate compilation databases for each of those files. I used cd dir1 && bear -- make a.o and cd dir2 && bear -- make b.o to create these files:

dir1/compile_commands.json

[
  {
    "arguments": [
      "/usr/bin/cc",
      "-c",
      "-o",
      "a.o",
      "a.c"
    ],
    "directory": "/path/to/dir1",
    "file": "/path/to/dir1/a.c",
    "output": "/path/to/dir1/a.o"
  }
]

dir2/compile_commands.json:

[
  {
    "arguments": [
      "/usr/bin/cc",
      "-c",
      "-o",
      "b.o",
      "b.c"
    ],
    "directory": "/path/to/dir2",
    "file": "/path/to/dir2/b.c",
    "output": "/path/to/dir2/b.o"
  }
]

3. Open VSCode and create a multi-root workspace by adding dir1 and dir2 as workspace folders
4. Using the "Go to definition" functionality works for bar and baz as expected, as well as for the foo in dir1/a.c. However, when using "Go to definition" on the foo in dir2/b.c, it takes me to the foo in dir1/a.c, which is clearly wrong.

Are there any plans to fix this?

[boldbyteboss]

Many (most?) of clangd's features do seem to be working for me in multi-root workspaces (using VSCode 1.69.2 and clangd v0.1.21 as of today). It's correctly finding the compile-commands.json files that exist in each workspace root, and is thus providing completions, go-to-definition and other source navigation operations as expected.

But one of the glaring exceptions is the support for clang-format. clangd appears to run with a current working directory inside the first root in the workspace and unconditionally uses the _clang-format file found there (or in a parent of that directory) even when there are (differing) _clang-format files in each of the workspace roots.

As much as one might hope otherwise, different groups at my company have very different ideas about how to format code. This goes terribly wrong when I build a workspace with components from different groups and code starts getting reformatted in unexpected ways.

[HighCommander4]

@tmwnewbold probably makes sense to file that as a separate issue. I would expect .clang_format to work similarly to compile_commands.json (looked up based on the directory of the file being edited), so if that's not working you may be running into a bug that's unrelated to clangd's lack of support for the workspace/workspaceFolders request.

[ToBoMi]

I have an opened folder (not really a workspace) and I use external C++ libraries that are

• precompiled in debug mode and
• have their source code attached to them in some folder near the lib location.

There are classes in the main projects source code that are derived from classes in one of the libs. The implemenation (cpp source files) of the superclasses (from the lib) are present in the source folders near the libs. When I try to Go to Definition on the superclass word

class myClass: public Another::Lib::SuperClass // <- cursor in the word SuperClass

I always end up in the header of the superclass, not the source file.

That is just logical, since the include locations of the libs are easily obtained by clangd, since they are mentioned in the compie_commands.json and clangd is not (yet?) aware of the additional debug sources of the libs.

Is this a form of multi-root workspaces?

If this is a multi-root workspace problem, is there already a way to tell clangd where additional source files are.

If this is not a multi-root workspaces problem, is there already an issue for this?

[i-ky]

is there already a way to tell clangd where additional source files are.

You can create compile_commands.json for external libraries and (if necessary) point clangd to them using config file. You can even create an index for these libs, especially if you don't intend to edit them.

[HighCommander4]

@ToBoMi I don't think use of clangd's "external index" feature is needed to solve the problem you're describing; letting the dependent libraries be indexed by clangd's background indexer should be sufficient.

The only caveat is that if the library has its own compile_commands.json, then it will have its own index (the index is stored next to the compile_commands.json, at .cache/clangd/index). In order for navigation to target a source file of the library, clangd needs to have loaded the library's index, which is currently only triggered when opening a source file of the library.

So the behaviour you will experience is:

1. No source file from the library is open yet --> go-to-def will target the header.
2. You open a source file from the library --> clangd loads the library's index --> go-to-def will target the source file.

If this is a problem, then a workaround would be to merge the library's compile_commands.json into your project's compile_commands.json (meaning, add the entries from the former into the latter; if you'd like to script this, jq is a helpful tool). Then, the library source files will be indexed together with the project source files, and go-to-def will target the source file right away.

[ToBoMi]

You can even create an index for these libs, especially if you don't intend to edit them.

I managed to create an external index in the default format: binary. (not YAML). (There are compilation errors during indexing which is normal I think and currently not that important. Includes of the indexed files cannot be found, but the index file gets created)

Index creation log (sensible information altered)

```PowerShell Executing task: powershell 'C:/Users/myusername/Downloads/clangd_15.0.3/bin/clangd-indexer.exe ((Get-ChildItem -Attributes !Directory .\path\to\debug\portable\sources1\*.cpp, .\path\to\debug\portable\sources2\*.cpp) | Select-Object -ExpandProperty FullName) > index.dex' In file included from c:\Users\myusername\path\to\project\path\to\debug\portable\sources1\someSource.cpp:21: c:\Users\myusername\path\to\project\path\to\debug\portable\sources1\someSource/someSource.hpp(27,10): fatal error: 'folder/IsomeSource.hpp' file not found #include "folder/IsomeSource.hpp" ^~~~~~~~~~~~~~~~ 1 error generated. Error while processing c:\Users\myusername\path\to\project\path\to\debug\portable\sources1\someSource.cpp. Failed to run action. ```

I get a YAML parsing error in clangd's log. I think the reason is the index.dex file I created, because the parsing error mistyriously mysteriously contains the first characters of the index file. This looks like clangd expects the index to be in YAML format. Does the index have to be in YAML format, or should it be binary?

index.dex in binary format (partially, first line only)

``` RIFFrÀ| [...] ```

clangd config file parsing log (sensible information altered)

``` [...] I[10:38:47.049] --> textDocument/clangd.fileStatus I[10:38:49.405] <-- textDocument/definition(41) I[10:38:49.408] Associating c:/Users/myusername/path/to/project/ with monolithic index at c:\Users\myusername\path\to\project\index.dex. I[10:38:49.408] --> reply:textDocument/definition(41) 3 ms I[10:38:49.408] --> textDocument/clangd.fileStatus YAML:1:4: error: Got empty plain scalar ��RI[10:38:49.560] --> reply:textDocument/codeAction(42) 1 ms I[10:38:49.560] --> textDocument/clangd.fileStatus I[10:38:49.580] <-- textDocument/documentLink(43) [...] ```

.clangd config file

```YAML Index: External: File: index.dex ``` _Yes, the index.dex is in the root of the project._

Maybe due to this parsing error, providing the index does not help to find the methods source in the library. When I create the index in YAML format and open it I can see, that the method is correctly indexed. The index contains it with the correct location. So it seems the index is not correctly opened by clangd. The index in YAML format also triggers parsing errors:

index.dex in YAML format (partially, first line only)

``` --- !Symbol [...] ```

clangd config file parsing log with index in YAML format (sensible information altered)

``` I[13:43:51.456] <-- textDocument/definition(17) I[13:43:51.457] Associating c:/Users/myusername/path/to/project/ with monolithic index at c:\Users\myusername\path\to\project\index.dex. I[13:43:51.457] --> reply:textDocument/definition(17) 0 ms I[13:43:51.457] --> textDocument/clangd.fileStatus YAML:1:4: error: Got empty plain scalar ��-I[13:43:51.749] <-- textDocument/documentSymbol(18) I[13:43:51.749] <-- textDocument/codeAction(19) ```

---

@HighCommander4's approach seems more straight forward. Therefore I wanted to test it, too.

@ToBoMi I don't think use of clangd's "external index" feature is needed to solve the problem you're describing; letting the dependent libraries be indexed by clangd's background indexer should be sufficient.

if the library has its own compile_commands.json

They do not have its own compile_commands.json.

1. When I freshly open vscode and then try to go to the definition of the library source, I end up at the header.
2. When I then open and close the source file where the definition of a method is in and then try to navigate from a method in a project source file to the definition in the library's source files that works. It works as long as vscode is not closed.

There is no new index file for the source file added after opening the library source once in clangd's cache folder. After closing vscode and opening it once again I have to reopen the library's source file again, to be able to navigate to the source. How can I achieve this permanently?

Opening all source files before being able to work on the code seems a lot of work. How can I trigger clangd to index these sources without opening them? I cannot find anything about all this in the clang docs, except I

create an index.

[ToBoMi]

When I use the YAML format, the file encoding was set to UTF16 LE. I changed it to UTF-8. This solves the issue with parsing the index file in YAML at the beginning. There are still errors at the end of the file:

clangd config file parsing log (sensible information altered)

``` [...] I[14:18:55.621] --> textDocument/clangd.fileStatus I[14:18:56.145] <-- textDocument/definition(32) I[14:18:56.146] Associating c:/Users/myusername/path/to/project/ with monolithic index at c:\Users\myusername\path\to\project\index.dex. I[14:18:56.146] --> reply:textDocument/definition(32) 0 ms I[14:18:56.146] --> textDocument/clangd.fileStatus I[14:18:56.305] <-- textDocument/codeAction(33) I[14:18:56.305] --> reply:textDocument/codeAction(33) 0 ms I[14:18:56.306] --> textDocument/clangd.fileStatus I[14:18:56.325] <-- textDocument/documentLink(34) I[14:18:56.326] --> reply:textDocument/documentLink(34) 0 ms I[14:18:56.326] --> textDocument/clangd.fileStatus I[14:18:56.429] <-- textDocument/codeAction(35) I[14:18:56.429] --> reply:textDocument/codeAction(35) 0 ms I[14:18:56.429] --> textDocument/clangd.fileStatus I[14:18:56.797] <-- textDocument/inlayHint(36) I[14:18:56.799] --> reply:textDocument/inlayHint(36) 2 ms I[14:18:56.799] --> textDocument/clangd.fileStatus YAML:3062396:1: error: missing required key 'References' ID: E46B2F04ED713E3C ^ E[14:18:58.844] Bad index file: Not a RIFF file and failed to parse as YAML: invalid argument I[14:19:12.024] <-- textDocument/hover(37) I[14:19:12.029] --> reply:textDocument/hover(37) 4 ms [...] ```

Somehow the index is incomplete:

index.dex in YAML format (partially, last lines only)

``` [...] References: - Kind: 10 Location: FileURI: 'file:///C:/Program%20Files%20%28x86%29/Windows%20Kits/10/Include/10.0.19041.0/um/winscard.h' Start: Line: 1217 Column: 22 End: Line: 1217 Column: 34 ... --- !Refs ID: E46A76E10EA72645 References: - Kind: 10 Location: FileURI: 'file:///C:/Program%20Files%20%28x86%29/Windows%20Kits/10/Include/10.0.19041.0/um/combaseapi.h' Start: Line: 277 Column: 8 End: Line: 277 Column: 15 ... --- !Refs ID: E46B2F04ED713E3C ```

This file has about 3 million lines and is about 100 Mb in size. I'll try to find out what makes the index generation process stop. That is definetly off-topic and should be addressed at clangd.

[HighCommander4]

if the library has its own compile_commands.json

They do not have its own compile_commands.json.

The library source files need to be listed in some compile_commands.json for them to be indexed up front (without having to open them in the editor).

Most commonly, you'd create it as an artifact of the library's build. (For example, it the library is built with CMake, you'd run CMake with -DCMAKE_EXPORT_COMPILE_COMMANDS=1. See https://clangd.llvm.org/installation#project-setup for more ways to produce a compile_commands.json.)

[ToBoMi]

The libraries are delivered as a package from a package manager. They contain the lib and the source code. Not much around it. Definetly no compile_commands.json. To get that file I need to check out every lib and build it to create the compile_commands.json. That is too much effort I think. And even if I do it, the debug sources of the lib would have a path that is different from the path listed in the generated compile_commands.json. It seems it is better to directly use the libs build, than taking the compile_commands.json and copy it to the lib and debug aource location.

[ToBoMi]

I made it work using the external index by

• using yaml as format for the index
• changing the encoding from UTF-16 LE to UTF-8 and
• have some luck, that the index is created completely (sometimes the file was incomplete - I am unable to find out what triggered this)

[HighCommander4]

Don't you need a compile_commands.json to create an external index, too?

[ToBoMi]

No (and a bit yes).

I had a compile_commands.json in my project root. This is used for the background indexing the sources of the main project (not the libs). I created the external index using clangd-indexer and it worked. I thought clangd-indexer didn't know about the compile_commands.json. But it did.

I deleted my compile_commands.json I have in the project root, just for testing (There is no compile_commands.json left anywhere else - searched for it). The external index file gets created now, but clangd-indexer complains about the missing compilation database now and there are many more errors during creation of the index.

So no you don't need a compilation database, but it is better having it I think. And it gets used by the indexer!

[HighCommander4]

How does clangd-indexer know which library source files to index if they are not listed in the compile_commands.json?

[braindevices]

the simplest solution is just do what others are doing: run clangd instance per root dir.