Closed benpicco closed 5 years ago
@marc-hb : FYI
@benpicco can you point at where exactly you got this toolchain to speed up reproduction attempts?
@marc-hb sudo apt install gcc-arm-none-eabi
on Ubuntu 19.04
My .zephyrrc is
export GNUARMEMB_TOOLCHAIN_PATH=/usr
export ZEPHYR_TOOLCHAIN_VARIANT=gnuarmemb
This is strange, I just tested with Ubuntu 18.04 and
usr/bin/arm-none-eabi-gcc --version
arm-none-eabi-gcc (15:6.3.1+svn253039-1build1) 6.3.1 20170620
and everything went fine, the option was correctly disabled as not supported.
Can you please try this:
mv build_samr21/ build_samr21-prefix-failure
cmake -B amd21 -DBOARD=atsamd21_xpro --trace-expand >& cmake-verbose.log
make -C amd21
then open cmake-verbose.log and search for zephyr_cc_option(-fmacro-prefix-map=
About 10 lines later can you check if you find something like this:
set(check 0 PARENT_SCOPE )
return()
(FYI the corresponding code is in cmake/extensions.cmake, line 680)
Next please try this:
cd $HOME
mv .cache/zephyr/ToolchainCapabilityDatabase TC_cache-prefix-failure
Also move the previous cmake-verbose.log, build directory and try building everything from scratch again. Open the new cmake-verbose.log and this time you should find errors like this one in cmake-verbose.log:
arm-none-eabi-gcc: error: unrecognized command line option '-fmacro-prefix-map=
This particular error should be about 200 lines after zephyr_cc_option(-fmacro-prefix-map=
Alright, now this is odd. I've just upgraded my computer at home to Ubuntu 19.04 and here the issue is not present.
So it looks like my ~/.cache/zephyr/ToolchainCapabilityDatabase
got somehow screwed up on my machine at work, deleting that cache then should fix the problem. I can only try this next week, but since I can't reproduce this and it this was probably only caused by some intermediate version, I'm closing this.
I/we would appreciate if you could give my instructions a try at work, carefully not deleting anything, just renaming build directories and caches. The cache should never get screwed up, if it does then there's a bug.
@marc-hb removing ~/.cache/zephyr
fixed it.
I've uploaded cmake-verbose.log and ~/.cache/zephyr if you want to investigate this further.
Thanks @benpicco , appreciated. I scanned this log and extracted the relevant parts below. According to this log, the cache had indeed the wrong, "1" (= True) value.
zephyr/cmake/target_toolchain.cmake(60): file(MD5 /usr/bin/arm-none-eabi-gcc CMAKE_C
_COMPILER_MD5_SUM )
zephyr/cmake/target_toolchain.cmake(61): set(TOOLCHAIN_SIGNATURE 3dbd52291f18fa93cbe3fb92359bcd9e )
...
zephyr/cmake/extensions.cmake(679): string(MD5 key 3_3dbd52291f18fa93cbe3fb92359bcd9e_C_-fmacro-prefix-map=/home/benpicco/dev/zeph-stable/zephyr=._-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files_ )
zephyr/cmake/extensions.cmake(682): set(key_path /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/d103ca4cbb139a183bd58894db563d1c )
...
zephyr/cmake/extensions.cmake(690): set(check 1 PARENT_SCOPE )
This comment in target_toolchain.cmake
drew my attention:
# It is not clear how this signature should be constructed. The
# strategy chosen is to md5sum the CC binary.
In this case the binary is /usr/bin/arm-none-eabi-gcc
. However -fmacro-prefix-map
is not a compiler option, it's a preprocessor option. @benpicco, @SebastianBoe are we sure /usr/bin/arm-none-eabi-cpp
from the same .deb package was always used? Any chance the newer /usr/bin/cpp
or some other cpp was used instead at some point?
AFAIK -gcc does not actually exec -cpp, it links with / has built-in the code for preprocessing. So this should be safe.
In theory, a binary could dynamically link with two different DLLs, which would not be detected by the caching mechanism, but this won't happen for arm-none-eabi-gcc I believe.
It happened again.
@marc-hb here is git reflog
d67d612ec3 (HEAD -> same54) HEAD@{0}: commit: fixup! drivers: entropy: Add SAM0 entropy driver
28bf58fe64 HEAD@{1}: commit: fixup! boards: atsame54_xpro: Add SAM E54 Xplained Pro Evaluation Kit
df1e8413f3 HEAD@{2}: commit: fixup! flash: sam0: Define LOCK_REGIONS in dts
ff7414b991 HEAD@{3}: rebase -i (finish): returning to refs/heads/same54
ff7414b991 HEAD@{4}: rebase -i (continue): flash: sam0: Define LOCK_REGIONS in dts
f6d734ef3f HEAD@{5}: rebase -i (pick): drivers: dma: sam0: fix DMA to peripheral transfer on SAMD5x
3909c296da HEAD@{6}: rebase -i (pick): soc: atmel_sam0: Add SAME53
a857ce74c8 HEAD@{7}: rebase -i (pick): soc: atmel_sam0: Add SAME51
c60f6c17ff HEAD@{8}: rebase -i (pick): ext: Import Atmel SAME51 header files from ASF library
46ee63eab9 HEAD@{9}: rebase -i (pick): ext: Import Atmel SAME53 header files from ASF library
39e60aca4d HEAD@{10}: rebase -i (pick): samd5x: hook up timers
c6a7faea34 HEAD@{11}: rebase -i (pick): soc: atmel_sam0: Add SAMD51
9bee5eaeb3 HEAD@{12}: rebase -i (pick): ext: Import Atmel SAMD51 header files from ASF library
41148942ae HEAD@{13}: rebase -i (pick): boards: atsame54_xpro: Add SAM E54 Xplained Pro Evaluation Kit
158ab43b23 HEAD@{14}: rebase -i (pick): drivers: entropy: Add SAM0 entropy driver
e19ab56710 HEAD@{15}: rebase -i (continue): timer: sam0_rtc_timer: Add support for SAME54
d6062de7e7 HEAD@{16}: rebase -i (continue): i2c: sam0: Add support for SAME54
c9c7234465 HEAD@{17}: rebase -i (continue): interrupt_controller: sam0: Add support for SAME54
ff82d1a624 HEAD@{18}: rebase -i (pick): spi: sam0: Add support for SAME54
379aa1f1da HEAD@{19}: rebase -i (continue): usb: sam0: Add support for SAME54
d3e93f0f5d HEAD@{20}: rebase -i (pick): pinmux: sam0: Add support for PORTD
a8c8c37683 HEAD@{21}: rebase -i (pick): gpio: sam0: Add support for PORTD
1b6a787c39 HEAD@{22}: rebase -i (pick): flash: sam0: Add support for SAME54
c3339fcdef HEAD@{23}: rebase -i (pick): watchdog: sam0: Add support for SAME54
68ecd37be7 HEAD@{24}: rebase -i (continue): serial: sam0: Add support for SAME54
61caded621 HEAD@{25}: rebase -i (pick): soc: sam0: Add SERCOM fixup for samd5x
76cb0c86ce HEAD@{26}: rebase -i (fixup): soc: atmel_sam0: Add SAME54
db55f0af86 HEAD@{27}: rebase -i (fixup): # This is a combination of 2 commits.
9b74843e13 HEAD@{28}: rebase -i (pick): soc: atmel_sam0: Add SAME54
bb68f8de8b HEAD@{29}: rebase -i (continue): ext: Import Atmel SAME54 header files from ASF library
00f88a4e25 HEAD@{30}: rebase -i (pick): scripts: dts: Generate indexed IRQ aliases
c0ae674630 (origin/master, master) HEAD@{31}: rebase -i (start): checkout master
a07b63cc82 (benpicco/same54) HEAD@{32}: checkout: moving from same54-dts-clock to same54
c89f0f306f (benpicco/same54-dts-clock, same54-dts-clock) HEAD@{33}: checkout: moving from same54 to same54-dts-clock
a07b63cc82 (benpicco/same54) HEAD@{34}: checkout: moving from same54 to same54
a07b63cc82 (benpicco/same54) HEAD@{35}: checkout: moving from same54-dts-clock to same54
c89f0f306f (benpicco/same54-dts-clock, same54-dts-clock) HEAD@{36}: checkout: moving from sam0-dts-clock to same54-dts-clock
7a3ca46e1c (sizurka/sam0-dts-clock, sam0-dts-clock) HEAD@{37}: checkout: moving from same54 to sam0-dts-clock
a07b63cc82 (benpicco/same54) HEAD@{38}: reset: moving to benpicco/same54
b4ac39e51f HEAD@{39}: checkout: moving from master to same54
c0ae674630 (origin/master, master) HEAD@{40}: pull: Fast-forward
8e2b9b4ac7 HEAD@{41}: pull: Fast-forward
(branch @same54-rebase)
@benpicco shared his --trace-expand
cmake log and .cache/ToolchainCapabilityDatabase/
with me. Again his .cache
is populated with incorrect information that says the option is supported. Why is this triggered by a rebase? Super weird.
I checked the actual files in .cache/ToolchainCapabilityDatabase/
, so this is not some extensions.cmake
bug reading the cache wrong. It could of course be some bug in the cmake code writing the cache wrong. By the time the problem happens it's unfortunately too late to debug how the cache gets populated.
zephyr/cmake/extensions.cmake(735): string(MD5 key 3_3dbd52291f18fa93cbe3fb92359bcd9e_C_-fmacro-prefix-map=/home/benpicco/dev/zephyr-dev/zephyr/samples/subsys/shell/shell_module=CMAKE_SOURCE_DIR_-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files_ )
zephyr/cmake/extensions.cmake(738): set(key_path /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/5c113ff08daefee0217cf0b0b7bbdd76 )
zephyr/cmake/extensions.cmake(739): if(EXISTS /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/5c113ff08daefee0217cf0b0b7bbdd76 )
zephyr/cmake/extensions.cmake(740): file(READ /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/5c113ff08daefee0217cf0b0b7bbdd76 key_value LIMIT 1 )
zephyr/cmake/extensions.cmake(746): set(check 1 PARENT_SCOPE )
zephyr/cmake/extensions.cmake(735): string(MD5 key 3_3dbd52291f18fa93cbe3fb92359bcd9e_C_-fmacro-prefix-map=/home/benpicco/dev/zephyr-dev/zephyr=ZEPHYR_BASE_-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files_ )
zephyr/cmake/extensions.cmake(738): set(key_path /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/426f0ad5cbac26151fc8ddd471d9c96d )
zephyr/cmake/extensions.cmake(739): if(EXISTS /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/426f0ad5cbac26151fc8ddd471d9c96d )
zephyr/cmake/extensions.cmake(740): file(READ /home/benpicco/.cache/zephyr/ToolchainCapabilityDatabase/426f0ad5cbac26151fc8ddd471d9c96d key_value LIMIT 1 )
zephyr/cmake/extensions.cmake(746): set(check 1 PARENT_SCOPE )
By the time the problem happens it's unfortunately too late to debug how the cache gets populated.
... yet all hope is not lost, because .cache/ToolchainCapabilityDatabase/
has a log.txt
file which I forgot about. The files there also have timestamps. Thanks to these I found a few things.
The reason the rebase triggered this issue is because the way -fmacro-prefix-map
is used has changed in (at least) PR #16536. So it's normal for extensions.cmake
to test this option again. By the way the entry for the old way to invoke -fmacro-prefix-map
is '0' and correct, only the 2 new entries are wrong.
Looking at timestamps, the rebase triggered the addition of only 3 new database entries:
toolchain_is_ok
empty string (why was it missing? The toolchain didn't change. UPDATE: because commit 6212ec9612f / PR #18741 )-fmacro-prefix-map= ... =CMAKE_SOURCE_DIR
-fmacro-prefix-map= ... =ZEPHYR_BASE
The trailing underscore in -Wl,--unresolved-symbols=ignore-in-object-files_
looks suspicious, is it ok? UPDATE: _ is just a separator.
ToolchainCapabilityDatabase/log.txt
# Apr 2019
...
1 c13d4521387e76c543fc03e2a1053657 3_3dbd52291f18fa93cbe3fb92359bcd9e_C__-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files -march=armv6s-m_
1 7f9f2f2dd0343ba80866c9f779cfa950 3_3dbd52291f18fa93cbe3fb92359bcd9e_C__-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files -Wl,--print-memory-usage_
1 73d5419afd0c5f000711510e0523f477 3_3dbd52291f18fa93cbe3fb92359bcd9e_C__-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files -mcpu=cortex-m4_
1 e7a3ca07c2eb23a4c9edc9d6caecf9e2 3_3dbd52291f18fa93cbe3fb92359bcd9e_C__-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files -march=armv7e-m_
# 18 Sep 2019
1 e65c706a3f8d4962b3542fbd0179cc08 3_3dbd52291f18fa93cbe3fb92359bcd9e_C__-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files_
1 5c113ff08daefee0217cf0b0b7bbdd76 3_3dbd52291f18fa93cbe3fb92359bcd9e_C_-fmacro-prefix-map=/home/benpicco/dev/zephyr-dev/zephyr/samples/subsys/shell/shell_module=CMAKE_SOURCE_DIR_-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files_
1 426f0ad5cbac26151fc8ddd471d9c96d 3_3dbd52291f18fa93cbe3fb92359bcd9e_C_-fmacro-prefix-map=/home/benpicco/dev/zephyr-dev/zephyr=ZEPHYR_BASE_-nostartfiles -nostdlib -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include" -isystem "/usr/lib/gcc/arm-none-eabi/7.3.1/include-fixed" -Wl,--unresolved-symbols=ignore-in-object-files_
Describe the bug When trying to build the latest master with GCC 7, the build fails due to the compiler option
-fmacro-prefix-map
not being available.To Reproduce Steps to reproduce the behavior:
Expected behavior The build succeeds.
Impact It's impossible to build Zephyr unless you have GCC 8 installed
Screenshots or console output
Environment (please complete the following information):