search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.34k stars 7.21k forks source link

Enable name of partitions (CSV) file to be selectable from within user defined (Kconfig.projbuild) file. (IDFGH-13648) #14529

Open eriksl opened 1 week ago

eriksl commented 1 week ago

Is your feature request related to a problem?

There is no regression here, just not very convenient. And I think contrary to the concept of IDF.

This is what I am stumbling upon. I want to use the same project and set of source files for creating images for the same chip (ESP32-S3) but on different boards. This means different amounts of FLASH, different amounts of SPIRAM, different GPIO's for various functions. So far this is no problem. I created a Kconfig.projbuild with a menu+choice for the board to be used, from there this data is set to invisible items and propagated to CONFIG_ defines.

The problem is that different flash sizes require different partition tables. The various boards have different file names for the partitions.csv file. I tried to set the name CONFIG_PARTITION_TABLE_CUSTOM_FILENAME by using additional "default" statements in the Kconfig.projbuild file. This is accepted, but it yields a warning and it doesn't work. The name always remains at "partitions.csv". I already removed the sdkconfig file completely (no problem there, I have a sdkconfig.defaults file). There is no CONFIG_PARTITION_TABLE_CUSTOM_FILENAME state in the sdkconfig.defaults file, so it is really undefined. Still Kconfig always sets it to the default from IDF: partitions.csv.

Describe the solution you'd like.

Some approach where IDF sets the custom partitions file in a "weaker" way - so it can be overridden. I am not sure if that's possible.

Another approach I can't think of with my limited IDF and Kconfig knowledge.

Describe alternatives you've considered.

I am now using this approach which works but is less conforming to the concept of IDF.

Additional context.

No response

igrr commented 1 week ago

Thanks for the suggestion, it does make sense to allow config options ("board") to be able set other options ("partition table"). It is a known limitation that Kconfig doesn't allow setting anything other than boolean options using select, which is something we are planning to address in the future.

I think the approach I'd recommend now is very close to what you described in the "alternatives" part, with the exception that instead of

symlink the required sdkconfig.defaults. to sdkconfig.defaults as appropriate

i would suggest implementing multiple configurations the way it is done in https://github.com/espressif/esp-idf/tree/master/examples/build_system/cmake/multi_config example.

In one of customer projects I've seen yet another solution: the partition table file for the specific board was copied to build/partitions.csv, and the name of the file was set to build/partitions.csv in sdkconfig. (I think there was also some trick to make sure the file was created early enough to be picked up by the rest of CMake steps...)

eriksl commented 1 week ago

The first alternative I will look into, sounds promising and in line with good practise.

The second one is an approach I already discarded, with the change that I was using symlinks instead of file copy. It doesn't solve my other requirement for having extra configuration features for boards (unknown to ESP-IDF).

eriksl commented 1 week ago

Had a quick look at option one. Looks very promising, I will probably switch to that approach, thanks!

The only drawback is that you need to invoke some magic incantations on the idf.py command line. But I guess there's nothing that can't be solved using a script or an alias. I am already using an "idf" script that sources all environment and then invokes idf.py, so the environment I am working in, isn't contaminated with IDF stuff.

And one advantage is that the build directories are kept isolated from each other.

igrr commented 1 week ago

The only drawback is that you need to invoke some magic incantations on the idf.py command line.

You can somewhat mask this complexity using @argfile syntax, described later in the readme of that example. If you are using a wrapper script it can also do the job, I guess.

eriksl commented 1 week ago

I was just re-reading the link and I saw it too. I think in many cases that will do the job perfectly, but in my case I prefer having a small script, which will do some other stuff do.

I am now in the process of implementing it. At least for the first board it seems to work like a charm, so I guess the others will do just fine too.

BTW I changed the approach just a tiny bit. Instead of having a "custom" version of the partitions.csv and sdkconfig.defaults files, I create a directory for each board. Each board has it's own "build" directory there and also it's own "sdkconfig.defaults" and "partitions.csv" files (with the usual/normal names). The sdkconfig.defaults in the parent directory then serves for common options. Each directory has some extra files, mainly for documentation.

I like it!

Thanks!

PS. I also have an S2 board lying around. At some point, when I am really bored, I will try to get this into the same concept as well. Even though it has a different "target" (S2 vs. S3). We'll see if it's sufficient to move all non-S2-relevant options from the common defaults file to the board specific defaults file (like BT and second CPU core), including the "target" option.

eriksl commented 1 week ago

One thing I noted, maybe this can be improved: if you switch from one board to another, you need to remove the sdkconfig file, otherwise the wrong options are used. If you do so, all generated object files are invalidated and compiling starts from scratch, including IDF components.

It would be nice if there would a fix/workaround for this, so you can switch between boards seamlessly.

igrr commented 1 week ago

One thing I noted, maybe this can be improved: if you switch from one board to another, you need to remove the sdkconfig file

Have you also copied the part of that example CMakeLists.txt where we place the sdkconfig file into the build directory?

https://github.com/espressif/esp-idf/blob/59e18382702b6986be3d3f55e9ac7763c1397cf7/examples/build_system/cmake/multi_config/CMakeLists.txt#L6

This way, each configuration has its own build directory and its own sdkconfig, so the builds are completely independent.

eriksl commented 1 week ago

Yes, that completely did the trick!

Thanks so much!

Sorry for being such a bother...