[question] How to generate a `compile_commands.json` with conan dependencies?

[nonlinearthink]

What is your question?

Environments

OS: Windows Compiler: msvc Build System: CMake IDE: VSCode with Clangd extension

Question

I can run my program correctly from the command line, but VSCode doesn't recognize the includePath, and I need to generate the compile_commands.json file. clangd use it to find the includePath.

Because MSBuild can't use CMAKE_EXPORT_COMPILE_COMMANDS to generate a compile_commands.json file, so I'm using Ninja now.

There is my profile:

[settings]
arch=x86_64
build_type=Release
compiler=msvc
compiler.cppstd=17
compiler.runtime=dynamic
compiler.version=193
os=Windows

[conf]
tools.cmake.cmaketoolchain:generator=Ninja

I finally generated the compile_commands.json file, but it doesn't seem to include the conan installed dependencies. It only tracks files in my workspace. So I still can't find the includePath of conan dependencies in VSCode.

Also I need to run conanbuild.bat before building cmake, otherwise my Ninja generator won't find c1.exe in my enviroment. I wish it also has another solution.

Have you read the CONTRIBUTING guide?

• [X] I've read the CONTRIBUTING guide

[memsharded]

Hi @nonlinearthink

Thanks for your question.

I don't know the details yet, usually users with the VSCode CMake integrations don't have much problems, if CMake is able to build and find the dependencies, it should usually also use the includePaths, etc for the editor without issues. Maybe it is because you are using only the clangd extension, but not any CMake one?

How did you generate the compile_commands.json? There would be missing some details, like what is your conanfile, what Conan command you used to install dependencies, what are the Conan generators (I guess CMakeDeps and CMakeToolchain?), what is the final cmake command used to build your program passing the Conan generated files (the conan_toolchain.cmake or the CMake presets)

[nonlinearthink]

conanfile.py:

class ScratchEngine(ConanFile):
    name = "ScratchEngine"
    version = "0.1.0"
    settings = "os", "compiler", "build_type", "arch"
    generators = "CMakeDeps", "CMakeToolchain"

    def requirements(self):
        # Runtime requirements
        self.requires("imgui/1.90")

    def build_requirements(self):
        # Build requirements
        self.tool_requires("cmake/[>=3.23]")
        self.tool_requires("ninja/1.11.1")

    def generate(self):
        # Copy the ImGUI bindings
        copy(
            self,
            "*impl_dx12*",
            os.path.join(self.dependencies["imgui"].package_folder, "res", "bindings"),
            os.path.join(self.source_folder, "bindings"),
        )
        copy(
            self,
            "*impl_win32*",
            os.path.join(self.dependencies["imgui"].package_folder, "res", "bindings"),
            os.path.join(self.source_folder, "bindings"),
        )

Command Line:

conan install . --output-folder=conan-build-release --build=missing
cmake --preset conan-release
cmake --build --preset conan-release

[memsharded]

Thanks for the feedback. Did you change the default profile to add tools.cmake.cmaketoolchain:generator=Ninja, or your command line is missing the -pr=myprofile?

Just trying to explore the issue, I am just experimenting with vscode. Some suggestions:

• I have added layout() method to your recipe, it is very convenient to not worry about passing -output-folder
• The tool_requires like cmake and ninja are only valid locally if you activate the conanbuild.bat locally. They will be used when creating packages, but for conan install and your flow above, they are ignored and used from the system, not from Conan packages

Without adding the Ninja generator, with the default Visual Studio one, VSCode works out of the box, I only have to tell its Intellisense to use CMake tools, and the includePaths for Conan dependencies are succesfully found and error squiggles dissapear. At least this is good news, things work out of the box for the standard VSCode setup.

Don't you have installed the CMake tools in VSCode?

[nonlinearthink]

The default VSCode settings with "configurationProvider": "ms-vscode.cmake-tools" works for me too, and aslo thanks for the suggestions.

So it's only the clangd that's in trouble. I use it because clangd has better code hints in most cases. And It allows me to use some useful features, like clang-format and clang-tidy. I only know it need compile_commands.json to track the dependencies. But not all generators can generate it, Ninja with a cmake flag CMAKE_EXPORT_COMPILE_COMMANDS used to work for me in a pure cmake project.

[nonlinearthink]

Thank you, that was my mistake.

clangd will try to find build/compile_commands.json, so I changed my settings.json:

"clangd.arguments": [
    "-compile-commands-dir=conan-build-release"
]

[Ext3h]

Also I need to run conanbuild.bat before building cmake, otherwise my Ninja generator won't find c1.exe in my enviroment. I wish it also has another solution.

That's kind of a bug in Conan actually. You would expect that Conan fills in configurePreset.environment with a copy of the active environment block at the point where Conan would do the CMake configure invocation.

At that point Conan had called the setup scripts to populate the environment block with everything relevant to the compiler. It is also missing all the environment variables exported by dependencies it potentially had active at that time.

And you suffer from a limitation in configurePreset that there is no possibility to specify any form of "wrapper" around CMake which could take care of that setup - short off straight out substituting the CMake exectuable itself with a user defined script which mocks conan. But that breaks CMake debugging in VS Code, so not great either.

That makes it currently really difficult to use Conan with Ninja and MSVC without having to let Conan do the initial CMake cache configuration. But that actually brings you to #15427 - there are more pitfalls on that route.

If conan were to run the VirtualBuildEnv generated script, and then dump the environment block, it should actually already resolve most of these issues.

[jcar87]

Hi @nonlinearthink - thank you for raising this question. We've been able to reproduce this:

A few things.

• Indeed if you are on Windows, you need to set tools.cmake.cmaketoolchain:generator=Ninja - as CMake won't generate compile_commands.json for the Visual Studio generators

• You can customize which flags are passed to CMake in the generate() method of the recipe, for instance:

from conan.tools.cmake import cmake_layout, CMakeToolchain
#clipped for brevity

    def generate(self):
        tc = CMakeToolchain(self)
        tc.cache_variables["CMAKE_EXPORT_COMPILE_COMMANDS"] = "ON"
        tc.generate()

This would cause Cmake to generate compile_commands.json

• By default, clangd looks in a build subdirectory relative to the root of the project. This is in their documentation: https://clangd.llvm.org/installation#project-setup . When using the defaults from cmake_toolchain in Conan, the actual c make build directory is likely to be build/Release or similar. You can either customize the build directory further by customizing the layout in the layout() method, or you can create a .vscode/settings.json file in your project, to specify as you have said:

  {
  "clangd.arguments": [
      "-compile-commands-dir=build/Release"
  ]
  }

I would advise against modifying the global setting for all workspace, and rather keep it local.

With these set up we are able to get clangd to index dependencies from the Conan cache as expected, without issues. It is useful to inspect the clangd output window in VSCode as that usually gives enough hints if something goes wrong.

In recent installations of VSCode - if you call conan install first, this will generate CMakeUserPresets.json and that should be enough to let VSCode "drive" the rest of the build (cmake configure, build, run, debug, etc) . I have not needed to change any other setting. However, as you may already be aware, if you are using clangd it is advised to disable the IntelliSense extension.

That's kind of a bug in Conan actually. You would expect that Conan fills in configurePreset.environment with a copy of the active environment block at the point where Conan would do the CMake configure invocation.

I think this is unrelated to the issue experienced by the issue reporter. However, please be advised that https://github.com/conan-io/conan/issues/15427 has already been resolved, and https://github.com/conan-io/conan/pull/15470 should achieve what you described.