CMake: Multiple targets (IDFGH-1357)

[PerMalmberg]

v4.0-dev-837-g58df1d93b xtensa-esp32-elf-gcc (crosstool-NG esp32-2019r1) 8.2.0

So, I've finally given up on using IDF as a library, it gets too complicated with linker scripts when the main app is two steps removed from IDF and idf.py is pretty convenient to have.

As such I'm now converting my framework to build as a IDF-component instead, in the hopes of that will make it more easy. While doing so I've stumbled onto a problem I can't see a solution to, namely multiple targets.

I have several test projects, all which are their own free-standing application. In pure CMake, these are easily handled via add_executable(), but what is the equivalent for IDF?

I want to be able to build all these targets at the same time so doing like what has been done with the IDF examples is not an option as they must be compiled separately.

[PerMalmberg]

Respectful nudge @renzbagaporo :)

[renzbagaporo]

Are these test projects ESP-IDF applications as well? Maybe you can take a page how we handle bootloader build as an external project, passing the sdkconfig so that they have the same config, though this entails redundant building of libraries especially which is painful if the executable have the same config essentially.

The build system does not have good support for multiple executables right now. You should be able to use add_executable as you normally would, linking the required libraries. But then to generate binary images you would have to replicate some of the logic in esptool_py/project_include.cmake.

[PerMalmberg]

Yes, they are indeed ESP-IDF projects as well as being compiled natively. I've got ~20 projects at this time and I expect that number to grow.

You should be able to use add_executable as you normally would, linking the required libraries.

So, what you're saying that I should treat them as regular CMake projects then add additional build/linker steps using procedures from project_include.cmake?

[renzbagaporo]

So, what you're saying that I should treat them as regular CMake projects then add additional build/linker steps using procedures from project_include.cmake?

You should be able to. Though like I said, expect some duplication of logic for what ESP-IDF to generate binaries. There are also multiple other problem, such as idf_build_executable being only able to support setting one executable (doing it multiple times would only generate binaries for the last one), idf.py would not know about other images generated, etc.

I've got ~20 projects at this time and I expect that number to grow.

So even with the sheer number of projects, at this point I would actually recommend building them as external projects, with the (recognizably) poor support for multiple output executable we have right now.

[PerMalmberg]

You should be able to

That is great news - I've been under the impression that it couldn't be done; i.e. any IDF project had to use the idf_* functions from start to end in order to compile correctly.

at this point I would actually recommend building them as external projects

Ok, then I'll give that a try.

[PerMalmberg]

@renzbagaporo Do you have a flowchart or something that shows the IDF-build process? I think it'd save me a substantial amount of time.

[renzbagaporo]

@PerMalmberg This is actually on my TODO list. However due to circumstances I will not be able to provide this in the short term.

[PerMalmberg]

@renzbagaporo Ok.

In the end I went with compiling only a single project when targeting ESP, keeping IDF as a library. It took me quite some time to get things working.

For example, if my main project is added via add_subdirectory before the interface library, the IDF-Cmake system gets really confused. Feel free to try it by switching these two lines around; that project uses my library in between itself and IDF. I the problem is the inclusion of tools/cmake/project.cmake that causes it, which brings me to the next point.

In project.cmake, the function __project_info is really handy to generate the project_description.json required by idf.py. Is there any chance it could be moved into another .cmake file that doesn't do anything when it is simply included?

[PerMalmberg]

@renzbagaporo So what do you think? Could __project_info be made into a public API and moved from project.cmake so that it can be included without side effects?

[renzbagaporo]

Hi @PerMalmberg, sorry for the late reply. I think we can actually do one better by actually moving metatool support (like idf.py) into its own component. What do you think of this?

[PerMalmberg]

@renzbagaporo If you see additional benefits doing that, do it.

[PerMalmberg]

@renzbagaporo Did this result in anything?

[jdoubleu]

I'm currently running into the same problem: I'd like to write multiple, stand-alone applications or executables in the same project/repository.

The real issue, in my opinion, is with the flashing tool. While I am able to create multiple executables either through CMake functions directly or by using the idf_ helpers, taking the restrictions into account (see comments above), I cannot specify which executable I'd like to flash to my board. The idf.py tool is always preferring the main component, isn't it?

I could imagine something like:

$ idf.py flash -p COM1 the_second_target

where the flash command would compile the the_second_target and actually flash it to the board. This would save time, because you could just instruct CMake to build the given target (cmake --build build/ --target the_second_target) instead of needing to re-configure CMake, because you changed the sdkconfig or something (e.g. the actual target you want to flash) specified during the the configuration phase.