component manager pulls in and build component that is excluded by Kconfig (IDFGH-10513)

kylezhou commented 1 year ago

Answers checklist.

[X] I have read the documentation ESP-IDF Programming Guide and the issue is not addressed there.
[X] I have updated my IDF branch (master or release) to the latest version and checked that the issue is present there.
[X] I have searched the issue tracker for a similar issue and not found a similar issue.

IDF version.

v5.0

Operating System used.

Linux

How did you build your project?

Command line with idf.py

If you are using Windows, please specify command line type.

None

What is the expected behavior?

I use component manager to manage a few custom components of a project. One of the custom components is ble_gatt_server, which depends on esp-idf/components/bt. I want to make the build time decision to enable or disable the component ble_gatt_server. So I added a Kconfg:

config BLE_GATT_SERVER
  bool "Enable BLE GATT server"
  depends on BT_ENABLED

In the main component CMakeLists.txt, I have:

set(priv_requires "")
if(CONFIG_BLE_GATT_SERVER)
  list(APPEND priv_requires "ble_gatt_server")
endif()
idf_component_register(SRCS "main.cpp"
    INCLUDE_DIRS "."
    PRIV_REQUIRES "${priv_requires}"
    )

In main component idf_component.yml, I have:

  ble_gatt_server:
    path: path/to/ble_gatt_server/

I would expect when I disable CONFIG_BLE_GATT_SERVER, ble_gatt_server will not be included and compiled since it is not in PRIV_REQUIRES list.

What is the actual behavior?

No matter whether I enable or disable CONFIG_BLE_GATT_SERVER, ble_gatt_server is always pulled in by component manager and get compiled. My understanding is CMakeLists.txt specifies what components are needed. idf_component.yml specifies where to retrieve the source code of the required components. But it seems that is not the case.

Steps to reproduce.

Step
Step
Step ...

Build or installation Logs.

No response

More Information.

No response

igrr commented 1 year ago

Hi @kylezhou, the issue you are running into happens because in IDF build system, component requirements (as passed to idf_component_register) can't depend on Kconfig options. Kconfig option values are determined after the list of components to be built has been decided, so at the time of component registration all the options are still undefined.

The sequence of events is:

component manager inspects idf_component.yml files
build system goes over the components, evaluating component requirements to decide which components should be included into the build
once the list of components had been determined, Kconfig is used to build the sdkconfig file.

It's not possible to affect the outcome of the preceding step by the result of the next one.

kylezhou commented 1 year ago

Thanks @igrr for the prompt reply.

component requirements (as passed to idf_component_register) can't depend on Kconfig options.

This sounds like a big constraint. But since Kconfig comes from various components, this makes sense. We need to get all required components first before we know all configs. So in my use case, there isn't a good way to enable/disable optional components. Any suggestions? One option I can think of is to change the optional component's CMakeLists.txt, so that when the Kconfig is not enabled, it does not include any C source files (thus it becomes an empty component).

igrr commented 1 year ago

@kylezhou Sorry for the late response.

One option I can think of is to change the optional component's CMakeLists.txt, so that when the Kconfig is not enabled, it does not include any C source files (thus it becomes an empty component).

Yes, I'm afraid that's the only option for the time being. We are looking at improving the way the build system performs dependency tracking (following the discussion in #10579), we will try to address this problem in the future.

espressif / esp-idf