Generate compile_commands.json for a component (IDFGH-8709)

[ljden]

Is your feature request related to a problem?

I would like to be able to generate the compile_commands.json file for a component under development for the purposes of using auto completion and code navigation (with clangd lsp).

Describe the solution you'd like.

I would like to be able to run idf.py <opt> to generate the compile_commands.json file. This could be stand alone or in conjunction with another command

Describe alternatives you've considered.

No response

Additional context.

No response

[igrr]

I don't think CMake had capability to generate a compile_commands.json file for a specific library, only for the whole project. This feature is enabled by default and you can find the file in the build directory, after running idf.py build or idf.py reconfigure.

If you really need a compile_commands.json file for just one component, i suggest adding a custom build target (e.g. generate_component_compile_commands) which will run something like jq to produce a filtered compile_commands.json file from the one generated by CMake. You can then invoke this custom target via idf.py.

[ljden]

You can definitely generate compile commands for a static library. Here is a trivial example: CMakeLists.txt

cmake_minimum_required(VERSION 3.8)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
project(foobarlib)
add_library( FooBar STATIC foo.cpp )
target_include_directories(FooBar PUBLIC include)

foo.cpp

#include "foo.h"

int foobar(bool bar) {
    return bar ? 42 : 0;
}

include/foo.h

#pragma once

int foobar(bool bar);

vbox@vbox:~/tmp/demo$ cmake .
-- The C compiler identification is GNU 11.3.0
-- The CXX compiler identification is GNU 11.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vbox/tmp/demo

vbox@vbox:~/tmp/demo$ cat compile_commands.json 
[
{
  "directory": "/home/vbox/tmp/demo",
  "command": "/usr/bin/c++  -I/home/vbox/tmp/demo/include -std=gnu++17 -o CMakeFiles/FooBar.dir/foo.cpp.o -c /home/vbox/tmp/demo/foo.cpp",
  "file": "/home/vbox/tmp/demo/foo.cpp"
}

This doesn't seem to be as easy when building with the esp-idf framework as there are a bunch of extra checks/requirements. For example when building a component with idf.py build the build system complains that I am registering a component from a non-component directory.

$ ls -R1
CMakeLists.txt
include/
KVStore.cpp

./include:
KVStore.h

$ cat CMakeLists.txt 
cmake_minimum_required(VERSION 3.8)
set(CMAKE_CXX_STANDARD 17)
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
# project(test)
idf_component_register(SRCS "KVStore.cpp" INCLUDE_DIRS "include")

$ idf.py build
Executing action: all (aliases: build)
Running cmake in directory /home/vbox/kv_store/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DIDF_TARGET=esp32 -DCCACHE_ENABLE=0 /home/vbox/kv_store"...
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Git: /usr/bin/git (found version "2.17.1") 
CMake Error at /home/vbox/esp/esp-idf/tools/cmake/component.cmake:427 (message):
  Called idf_component_register from a non-component directory.
Call Stack (most recent call first):
  CMakeLists.txt:6 (idf_component_register)

[ljden]

If I put KVStore.cpp in a main/ subdirectory I can run idf.py reconfigure which will generate compile_commands,json as expected. Would it be possible for a similar command to be added for a component such that a component can be registered in the root dir?

Current:

vbox@vbox:~/kv_store$ ls -R1
.:
CMakeLists.txt
include
main

./include:
KVStore.h

./main:
CMakeLists.txt
KVStore.cpp

vbox@vbox:~/kv_store$ cat CMakeLists.txt 
cmake_minimum_required(VERSION 3.8)
set(CMAKE_CXX_STANDARD 17)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(test)

vbox@vbox:~/kv_store$ cat main/CMakeLists.txt 
idf_component_register(SRCS "KVStore.cpp" INCLUDE_DIRS "${PROJECT_DIR}/include")

vbox@vbox:~/kv_store$ idf.py reconfigure
Executing action: reconfigure
...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vbox/kv_store/build

Desired:

vbox@vbox:~/kv_store$ ls -R1
.:
CMakeLists.txt
include
KVStore.cpp

./include:
KVStore.h

vbox@vbox:~/kv_store$ cat CMakeLists.txt 
idf_component_register(SRCS "KVStore.cpp" INCLUDE_DIRS "include")

vbox@vbox:~/kv_store$ idf.py reconfigure_component
Executing action: reconfigure_component
...
-- Configuring done
-- Generating done
-- Build files have been written to: /home/vbox/kv_store/build

[igrr]

I don't think this is easy to achieve without actually having a project... "Configuring" involves preparing the project configuration (sdkconfig), which is a project-level entity. The component can't be configured or built on its own, without having the project configuration. I think you'd have to at least make a "dummy" project, add the component in question to the project, then configure the dummy project.

For such a requirement, I would recommend creating a test application in the component directory. The test application can be used to build the component, and it can also include unit or other type of tests for the component. There's a number of examples of this approach in IDF itself:

• component directory
• test_apps subdirectory in the component

You can then configure/build the test_app inside the component, and compile_commands.json will contain also the component paths.

[ljden]

It's not ideal but I've symlinked the compile_commands.json from host_test/build into the component directory (building for linux target so platform specific stuff will be different)