search

elmot / clion-embedded-arm

OpenOCD + STM32CubeMX support for CLion
https://blog.jetbrains.com/clion/2017/12/clion-for-embedded-development-part-ii
Other
170 stars 29 forks source link

Out-of-source CMake builds differ from CLion #124

Closed johnthagen closed 6 years ago

johnthagen commented 6 years ago

Expected Behavior

Building from an out-of-source build folder should perform the same build as in-source CLion builds.

Actual Behavior

The builds are different (e.g. the out-of-source binaries can be larger, or not build properly). It seems like it has to do with linking.

Steps to Reproduce the Problem

Both the command line and CLion are using CMake 3.12.2. To ensure that you are using the exact same CMake, you can replace cmake below with something like /Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake that you see from the CMake pane in CLion.

$ mkdir build
$ cd build
$ cmake -DCMAKE_BUILD_TYPE=Release ..
$ make
$ size <PROGRAM>.elf  # We are in the `build` folder

Compare this to a CLion release build:

$ size cmake-build-release/<PROGRAM>.elf

The .hex and .map files are also different.

I'm still working on tracking down exactly what's going, but it might have to do with the use of CMAKE_BINARY_DIR instead of CMAKE_SOURCE_DIR in some parts of the template.

This is the full path to the top level of the current CMake build tree. For an in-source build, this would be the same as CMAKE_SOURCE_DIR.

It's possible this is causing some kind of difference between what files are seen by an out-of-source build and those seen by CLion.

Specifications

johnthagen commented 6 years ago

I tried building from the command line in the root of the project as well (not in a build subfolder) and still am seeing build changes. Is the plugin doing anything else to the build such that the build would be different from the command line?

elmot commented 6 years ago

Is the plugin doing anything else to the build such that the build would be different from the command line?

It does not.

Some differences my come from different toolchain versions (if you have more than one installed), another version of CMake, or build type you provide from command line.

I do not think the issue is somehow related to the plugin, and I close it. If you still have some concerns, it's better to ask at Clion forum or Clion Bugtracker

elmot commented 6 years ago

BTW, both .elf, .map, and .hex files MAY be different even if the actual binaries the same. Try to compare .bin files, which are naked bytes of the firmware.

johnthagen commented 6 years ago

Thanks for the tip. The .bin files are different sizes, and correlate to what is reported by size <PROGRAM_NAME>.elf.

It's good to know that the plugin should really have no affect on this.

After some further investigation, it appears that CLion's CMake cache was the culprit. After removing the cmake-build-release folder, and running File | Invalidate Caches & Restart, I saw some more similarities in the builds. I'm wondering if CLion is cacheing part of the build or linking step (for example related to libc) and that is causing some problems.

It's a little scary to think that CLion might not be producing clean builds, perhaps because there are some assumptions it makes that can be broken (for example, if the user needs to modify the linker script, does CLion know to clean out it's CMake cache)?

I agree that this is probably not the responsibility of this plugin, but I just wanted to capture my findings here so someone else with a similar problem may find it.

johnthagen commented 6 years ago

For example, removing -flto causes my clean out-of-source build to fail, but CLion will continue to build just fine (because it's cached some part of the build/link step, probably something in the C or C++ stdlib).

If then in CLion you:

$ rm -r cmake-build-release/*

File | Reload CMake Project

CLion will then fail to build as expected. I just wonder if there's something that not being "cleaned" and if possibly there is a way to modify the CMakeLists.txt to properly work around this.

johnthagen commented 6 years ago

Okay, I've confirmed it's definitely CLion's CMake cache not being properly cleared/rebuilt when it should. Removing the cache from cmake-build-* folders allows CLion to build normally again. Will try to confirm on another OS as well.

elmot commented 6 years ago

I do not think you need to confirm the case on any other platform. Probably the issue is known and more or less the same everywhere. There is a button to refresh cached cmake build config: image

johnthagen commented 6 years ago

@elmot Thanks for showing me a screenshot of that command, I'd never found that before and confirmed that it did also properly reset the cache.

Thanks!

johnthagen commented 6 years ago

@elmot Could you consider reopening this issue? I have finally tracked down the cause of this issue, and it is in the CMake template provided by the plugin.

The template uses https://github.com/elmot/clion-embedded-arm/blob/master/resources/xyz/elmot/clion/cubemx/tmpl_CMakeLists.txt#L29-L31

SET(CMAKE_CXX_FLAGS_INIT "$${COMMON_FLAGS} -std=c++11")
SET(CMAKE_C_FLAGS_INIT "$${COMMON_FLAGS} -std=gnu99")
SET(CMAKE_EXE_LINKER_FLAGS_INIT "-Wl,-gc-sections,--print-memory-usage -T $${LINKER_SCRIPT}")

The important part is the use of CMAKE_<LANG>_FLAGS_INIT

Value used to initialize the CMAKE__FLAGS cache entry the first time a build tree is configured for language . This variable is meant to be set by a toolchain file. CMake may prepend or append content to the value based on the environment and target platform.

I bolded the important part. These values are only set once when the CMake project is initialized by CLion, and if the user changes any of these flags, they are not updated automatically as they would be for a normal CLion project.

Changing this line to use CMAKE_<LANG>_FLAGS instead, solves the problem:

SET(CMAKE_CXX_FLAGS "$${COMMON_FLAGS} -std=c++11")
SET(CMAKE_C_FLAGS "$${COMMON_FLAGS} -std=gnu99")
SET(CMAKE_EXE_LINKER_FLAGS "-Wl,-gc-sections,--print-memory-usage -T $${LINKER_SCRIPT}")
elmot commented 6 years ago

Too risky. I had troubles when the flags were set to CMAKE_<LANG> Yes, that time cmake was of another version but anyway sounds too risky. Better to use some combo of CMAKE_<LANG> and CMAKE_<LANG>_INIT

johnthagen commented 6 years ago

I did try some combination of the two on my path towards tracking down this issue, but that actually caused not both sets of flags to be used, which turned out to be worse.

Hmm, I don't doubt you had problems in the past. I wonder why it mattered? I've been using CMake for a long time and always seen the non-_INIT flags used in practice.

johnthagen commented 6 years ago

Also, since the plugin sets a lower bound on the CLion version, moving forward can't we depend on a minimum CMake version since it's bundled with CLion?

We wouldn't have to backport this to older plugin releases that target older CLion versions.