search

espressif / esp-idf

Espressif IoT Development Framework. Official development framework for Espressif SoCs.
Apache License 2.0
13.27k stars 7.2k forks source link

sdkconfig changes when project is built on different machines (IDFGH-12478) #13490

Open krupis opened 5 months ago

krupis commented 5 months ago

Answers checklist.

IDF version.

5.0.4

Operating System used.

Windows

How did you build your project?

VS Code IDE

If you are using Windows, please specify command line type.

PowerShell

What is the expected behavior?

Build the project without any sdkconfig conflicts

What is the actual behavior?

We have multiple people working on the same esp-idf project. We have noticed that whenever someone makes changes and push to the repository, whenever other people try to pull all the changes and build the project on their own machine, during build process something changes on the sdkconfig.h.

I want to understand why does the sdkconfig.h changes when project is built on different machines even if the same esp-idf version is used?

Steps to reproduce.

  1. Pull someones changes from the repository
  2. Build project

Build or installation Logs.

For example, I have just pulled changes from my colleague and built the project. I can see that changes are automatically detected using the Source Control extension on the VSCode on the sdkconfig.

It seems that the only changes are the order. Please see the example below: image

image

As you can see from 2 images above, on my machine, LittleFS is configured starting on the line 1675 and on my colleagues sdkconfig it is configured starting at line 1737. What is the reason for that and why that happens?

More Information.

No response

dobairoland commented 5 months ago

Because items before writing those files are not sorted and their order is semi-random and could be different between systems just as you observe.

You are not supposed to commit sdkconfig or anything from the build directory, including sdkconfig.h, into source control management systems. Any changes different from the default ones should be stored in sdkconfig.defaults and that could be committed. See idf.py save-defconfig for how to generate that file. You can add items to that file manually as well. sdkconfig should be removed each time there is a change in sdkconfig.defaults or the changes will be ignored.

dobairoland commented 5 months ago

I was notified that we should support committing sdkconfig files into source control systems, and therefore, deterministic order in that file. So I'm reopening this....

krupis commented 5 months ago

So everytime I make a change in sdkconfig asuming I use menuconfig what is being changed is the sdkconfig right? So when I am ready to upload new changes to the git, I have to manually copy and paste all the changes and put them to sdkconfig.defaults? That sounds like unnecessary extra step

Perhaps I do not fully understand how does sdkconfig and sdkconfig.defaults work yet. Im reading about it as we speak.

I simply want to understand what is the optimal way of making changes in the menuconfig and pushing those changes to the repository while avoiding the issue I have described above.

KaeLL commented 5 months ago

So everytime I make a change in sdkconfig asuming I use menuconfig what is being changed is the sdkconfig right?

Yes

So when I am ready to upload new changes to the git, I have to manually copy and paste all the changes and put them to sdkconfig.defaults?

No. You can just run idf.py save-defconfig.
Of if you haven't opened menuconfig yet, you can delete the sdkconfig file, open menuconfig, make the changes you want, press D, and save it to ../sdkconfig.defaults.

krupis commented 5 months ago

So everytime I make a change in sdkconfig asuming I use menuconfig what is being changed is the sdkconfig right?

Yes

So when I am ready to upload new changes to the git, I have to manually copy and paste all the changes and put them to sdkconfig.defaults?

No. You can just run idf.py save-defconfig.
Of if you haven't opened menuconfig yet, you can delete the sdkconfig file, open menuconfig, make the changes you want, press D, and save it to ../sdkconfig.defaults.

Could you just confirm what is the purpose of that and why cant we just simply commit sdkconfig instead?

KaeLL commented 5 months ago

You can commit it, but the file is unstable given that configs are added/removed/moved/renamed at will, resulting in a lot of noise each time IDF is updated and a Kconfig is touched.

dobairoland commented 5 months ago

Hi @krupis.

There is a line in tools/cmake/project.cmake at https://github.com/espressif/esp-idf/blob/v5.0.4/tools/cmake/project.cmake#L292:

file(GLOB component_dirs ${component_dir}/*)

Could you please try adding the following line after it? 

list(SORT component_dirs)

This has to be done on both computers where you experience the difference in the sdkconfig content.

Could you please try this and report back if this fixed the issue? I'm just guessing because I cannot reproduce it.

mickeyl commented 4 months ago

@dobairoland

I have a similar problem, but more annoying in so far that it happens on the same machine.

A call of idf.py build seems to reorder certain parts of the sdkconfig, so that I get a bogus diff afterwards. Adding the suggested list(SORT component_dirs) in project.cmake does not make a difference for me.

Anything else I could try?

dobairoland commented 4 months ago

The greatest help would be to guide us to reproduce it.

Which parts of sdkconfig are changing? There is a one block which is moving around or there are more moving parts? It is always completely different or there is a pattern?

Where the moving parts are coming from? Is it a component in ESP-IDF, added by an environment variable, a local component in the project directory, obtained by component manager, ....?

mickeyl commented 4 months ago

Complete blocks are moving, always the same blocks. Those blocks are coming in from Kconfig files that are part of components referenced via (local) dependencies specified in main/idf_components.yml.

ConnorDuvalS commented 4 months ago

I believe @mickeyl is right about #10419 being related to this. @dobairoland there is a full step-by-step reproduction guide in my comment at https://github.com/espressif/esp-idf/issues/10419#issuecomment-2035833444.

dobairoland commented 4 months ago

Let us wait for https://github.com/espressif/idf-component-manager/issues/58 to see if the ordering can be fixed on the component manager side.

2f811b7975dd85e444ea5885d5bf3f4c5a25de0e already addressed ordering on the ESP-IDF side.