cmake: Improve library dependencies by removing whole-archive, start/end-group

[tejlmand]

Zephyr subsystems creates multiple (sub-)sublibraries, especially bluetooth and net. Those subsystems also has internal dependencies to each other.

Multiple subsystems in Zephyr has a lot of sub libraries, which I believe should be minimized, especially a KConfig is good for configuring the libraries.

Subsystem Bluetooth contains: subsys_bluetooth, subsysbluetoothcommon, subsysbluetoothcontroller, subsysbluetoothhostmesh, subsysbluetooth__host

Subsystem net contains: subsysnet, subsysnet_ip, subsysnet_ip_l2, subsysnet_ip_l2_ethernet, subsysnet_ip_l2_ieee802154, subsys__net_ip_l2_openthread, subsysnetlibapp, subsysnetlibdns, subsysnetlibhttp, subsysnetliblwm2m, subsysnetlibmqtt, subsysnetlib__websocket

Zephyr links everything together using -Wl,--start-group

-Wl,--end-group, as well as Zephyr build system uses -Wl,--whole-archive. This principle has several drawbacks; The -Wl,--whole-archive drawback is also described in: #6961 Creating executables in this manner requires extra build scripts and is non-standard for a CMake project. Instead of using the CMake API to link together libraries into an executable with target_link_libraries() one must have build scripts that ensures the linker is passed this list of linker flags. It introduces two kinds of libraries, instead of having just one kind. Libraries which must be whole-archived, and libraries that should not. There is also a theoretical code size performance impact. If whole-archive is used and there exists an object file in a library that happens to not be in use (dead code) the object file will still be included in the executable, instead of being pruned as one would want it to be. But if one links without '--whole- archive' only the object files that have symbols needed by the link will be used. In Zephyr, the handling of library content is well handled using KConfig, and doesn't seem to suffer from this risk. However, that doesn't mean the problem may not be present in certain parts of Zephyr. Finally, using a flat linking model where all libraries are included means that we do not have readily available information about how libraries depend on each other. CMake list ${ZEPHYR_LIBS_PROPERTY} is furthermore a global variable to which libraries are added and later used in the linking stage. Use of global variables should be minimized whenever possible. EDIT: This is part of umbrella issue: #8827 EDIT: Removed text covered by #8826

[SebastianBoe]

Would you mind splitting the "many libraries causes noise in IDE's" out into it's own issue?

EDIT: If we split that out then this issue will solely be about the problems that arise from the mechanism we use to link (--whole-archiving a global list of libraries).

[tejlmand]

Would you mind splitting the "many libraries causes noise in IDE's" out into it's own issue?

Done: see #8826

[SebastianBoe]

Done: see #8826

Great that 8826 was created. But could we also remove the parts of 8826 that are redundant with this issue?

e.g. remove the text above

"Furthermore Zephyr links everything together using -Wl,--start-group"

and change the title to

"cmake: Improve library dependencies by removing whole-archive, start/end-group"

[tejlmand]

Great that 8826 was created. But could we also remove the parts of 8826 that are redundant with this issue?

e.g. remove the text above

"Furthermore Zephyr links everything together using -Wl,--start-group"

Removed the text directly related to the IDE, but kept text which describes/lists the sub-libraries, as that is important for the issue. With many sub-libraries, it becomes hard to setup all inter dependencies (and are they circular as well ?) Example, every subsys__bluetooth_X probably depends upon subsysbluetoothcommon but are there inter dependencies between subsysbluetoothcontroller and subsysbluetoothhost ? subsysbluetoothhostmesh and subsysbluetooth__host

Thus, reducing this to single subsys_bluetooth lib removes the need in CMake to described such relations, as well as it makes a cleaner and easier to describe build system, in terms of what libraries are build and what do they provide.

and change the title to

"cmake: Improve library dependencies by removing whole-archive, start/end-group"

Done

[zephyrbot]

Hi @tejlmand,

This issue, marked as an Enhancement, was opened a while ago and did not get any traction. It was just assigned to you based on the labels. If you don't consider yourself the right person to address this issue, please re-assing it to the right person.

Please take a moment to review if the issue is still relevant to the project. If it is, please provide feedback and direction on how to move forward. If it is not, has already been addressed, is a duplicate, or is no longer relevant, please close it with a short comment explaining the reason.

@tejlmand you are also encouraged to help moving this issue forward by providing additional information and confirming this request/issue is still relevant to you.

Thanks!