Closed nashif closed 7 years ago
by Juro Bystricky:
Yes, multilibs need some cleanup. For example, the following should be in Makefile.toolchain.zephyr:
ifeq ($(ARCH),arm)
CROSS_COMPILE_TARGET_arm = arm-poky-eabi
CROSS_COMPILE_arm=$(TOOLCHAIN_HOME)/usr/bin/$(CROSS_COMPILE_TARGET_arm)/$(CROSS_COMPILE_TARGET_arm)-
SYSROOT_arm := ${ZEPHYR_SDK_INSTALL_DIR}/sysroots/armv5-poky-eabi
# libgcc
LIBGCC_DIR_arm = $(shell dirname `$(CROSS_COMPILE_arm)gcc --sysroot=$(SYSROOT_arm) $(KBUILD_CFLAGS) -print-libgcc-file-name`)
LIB_INCLUDE_DIR_arm += -L ${LIBGCC_DIR_arm}
# newlib lib/include
NEWLIB_DIR_arm = $(shell $(CROSS_COMPILE_arm)gcc --sysroot=$(SYSROOT_arm) $(KBUILD_CFLAGS) -print-multi-directory)
LIB_INCLUDE_DIR_arm += -L $(SYSROOT_arm)/usr/lib/$(NEWLIB_DIR_arm)/
TOOLCHAIN_CFLAGS_arm = -I $(SYSROOT_arm)/usr/include
endif
Similar setup for other multilib toolchains (ARC, Nios2, ...). Basically, we query the compiler itself which library should be used given compiler flags (options). One caveat: KBUILD_CFLAGS must be set, as it contains all GCC command line options. Currently Makefile.toolchain.zephyr is included before this is done, so some re-ordering is needed as well.
by Kumar Gala:
Fixed by combination of SDK 0.8.2 (which supports multilibs for softfp and hard fp ABIs). Plus change to allow user to select which Floating Point ABI they want to build with.
by Sharron LIU:
Please reporter verify and close this. Thanks.
by Mark Linkmeyer:
Hi Kumar Gala , as the Reporter of this bug we need you to verify it's fixed since it's in the Resolved state. Please verify it ASAP so we know whether or not more work is required to fix it before the upcoming 1.5 release. If it's fixed, please move it to the Verified state. Thanks!
by Marcus Shawcroft:
The underlying issue is still present with the SDK 0.8.2, but manifests when building with hard float disabled.
The multilib configuration for arm-poki-eabi-gcc is: .; thumb;@mthumb fpu;@mfloat-abi=hard thumb/thumb2;@mthumb@march=armv7
The KBUILD_CFLAGS options passed for an frdm_k64f build include: -mcpu=cortex-m4 -mthumb -march=armv7e-m
The architecture 'armv7e-m' is correct for a cortex-m4. This set of options will match the first multlib (ie thumb;@thumb). The libgcc.a file in that directory was built for armv4t (where thumb and arm state are legitimate). The armv7e-m architecture is thumb2 only, hence a binary linked against this first multilib is illegal.
For reference, there is comprehensive multilib support for the full range of ARM R and M profile architecture permutations provided in the the ARM/embedded-5 upstream GCC branch. See here for the first commit: https://gcc.gnu.org/viewcvs/gcc/branches/ARM/embedded-5-branch/gcc/config/arm/t-baremetal?revision=231750&view=markup
This patch provides support for the following multilibs: .; thumb;@mthumb fpu;@mfloat-abi=hard armv6-m;@mthumb@march=armv6s-m armv7-m;@mthumb@march=armv7-m armv7e-m;@mthumb@march=armv7e-m armv7-ar/thumb;@mthumb@march=armv7 armv8-m.base;@mthumb@march=armv8-m.base armv8-m.main;@mthumb@march=armv8-m.main armv7e-m/softfp/fpv5-sp-d16;@mthumb@march=armv7e-m@mfloat-abi=softfp@mfpu=fpv5-sp-d16 armv7e-m/softfp/fpv5-d16;@mthumb@march=armv7e-m@mfloat-abi=softfp@mfpu=fpv5-d16 armv7e-m/softfp;@mthumb@march=armv7e-m@mfloat-abi=softfp@mfpu=fpv4-sp-d16 armv7e-m/fpu/fpv5-sp-d16;@mthumb@march=armv7e-m@mfloat-abi=hard@mfpu=fpv5-sp-d16 armv7e-m/fpu/fpv5-d16;@mthumb@march=armv7e-m@mfloat-abi=hard@mfpu=fpv5-d16 armv7e-m/fpu;@mthumb@march=armv7e-m@mfloat-abi=hard@mfpu=fpv4-sp-d16 armv7-ar/thumb/softfp;@mthumb@march=armv7@mfloat-abi=softfp@mfpu=vfpv3-d16 armv7-ar/thumb/fpu;@mthumb@march=armv7@mfloat-abi=hard@mfpu=vfpv3-d16 armv8-m.main/softfp/fpv5-sp-d16;@mthumb@march=armv8-m.main@mfloat-abi=softfp@mfpu=fpv5-sp-d16 armv8-m.main/softfp/fpv5-d16;@mthumb@march=armv8-m.main@mfloat-abi=softfp@mfpu=fpv5-d16 armv8-m.main/fpu/fpv5-sp-d16;@mthumb@march=armv8-m.main@mfloat-abi=hard@mfpu=fpv5-sp-d16 armv8-m.main/fpu/fpv5-d16;@mthumb@march=armv8-m.main@mfloat-abi=hard@mfpu=fpv5-d16
While this is not complete, it does provide a viable lowest common denominator for all current arm aarch32 cortex-m/r architectures.
This patch/support has not yet been presented for inclusion in upstream GCC trunk, but my understanding is that this is likely in the near future.
w.r.t the immediate issue in zephyr, possible short term options include: -) Add multilibs to cover just the current issue of broken cortex-m4 / aarch32e-m -) Backport the relevant multilib patches from upstream gcc/embedded-5 to the zephyr sdk (these patches are ARM centric and hence are not likely to destabalize the compiler behaviour on the other architecture we care about in zephyr).
It's worth noting that folks currently working on cortex-m0 / m0+ support are going to hit the same issue (cortex-m0 == armv6-m), hence a minimum viable set of armv6-m multlibs will be required to enable their work.
by Marcus Shawcroft:
This is still broken:
with 0.8.2. On frdm_k64f configured CONFIG_FP_SOFTABI an inappropriate libgcc.a is linked resulting in ARM state code being executed on arm7e-m
with CONFIG_FP_HARDABI we get a link error (one object uses VFP registers another does not).
See my previous comment above for more detail.
by Kumar Gala:
What error are you seeing. How to reproduce?
by Marcus Shawcroft:
Reproduce like this:
$ cat prj.conf CONFIG_STDOUT_CONSOLE=y CONFIG_FLOAT=y
$ cat src/main.c double f = 2.0;
int main () { printf ("hello\n"); f = f / 3; printf ("f=%f\n", f); printf ("world\n"); }
$ make BOARD=frdm_k64f
gives.... /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-gcc -T linker.cmd @zephyr.lnk -o .tmp_zephyr.prebuilt /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: error: .tmp_zephyr.prebuilt uses VFP register arguments, /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/libgcc.a(_udivmoddi4.o) does not /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: failed to merge target specific data of file /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/libgcc.a(_udivmoddi4.o)
Adding CONFIG_FP_SOFTABI:
Results in an apparently successful build. Howver the binary produced contains ARM state code. If you try to execute the resulting binary it will repeatedly fault.
For direct evidence of the issue: $ arm-none-eabi-objdump -d outdir/frdm_k64f/zephyr.elf | grep -A 10 aeabi_ddiv | head -n 10 00000958 <__aeabi_ddiv>: 958: e92d4070 push {r4, r5, r6, lr} 95c: e3a0c0ff mov ip, #255 ; 0xff 960: e38ccc07 orr ip, ip, #1792 ; 0x700 964: e01c4a21 ands r4, ip, r1, lsr #20 968: 101c5a23 andsne r5, ip, r3, lsr #20 96c: 1134000c teqne r4, ip 970: 1135000c teqne r5, ip 974: 0b00005e bleq af4 <__aeabi_ddiv+0x19c> 978: e0444005 sub r4, r4, r5
ie ARM state code.
by Kumar Gala:
Not able to reproduce. I'm able to run with both CONFIG_FP_SOFTABI and CONFIG_FP_HARDABI without issues. I also don't see the compile issue with HARDABI. Can you build with V=1 and report the link lines you are seeing.
by Marcus Shawcroft:
Its possible to demonstrate correct behaviour here using the pre-built toolchain from https://launchpad.net/gcc-arm-embedded
$ arm-none-eabi-gcc -v ... gcc version 5.4.1 20160609 (release) ...
$ make BOARD=frdm_k64f CROSS_COMPILE=arm-none-eabi-
Generates binaries that execute correctly in both of the examples I gave above.
by Marcus Shawcroft:
Hi, Link lines as request:
rm -f libzephyr.a; /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-ar rcTD libzephyr.a drivers/built-in.o lib/built-in.o kernel/built-in.o misc/built-in.o net/built-in.o boards/built-in.o ext/built-in.o usb/built-in.o fs/built-in.o arch/built-in.o
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-gcc -x assembler-with-cpp -nostdinc -undef -E -P \
-D__GCC_LINKER_CMD__ -I/home/marcus/work/zephyr/zephyr-project/include \
-I./include/generated /home/marcus/work/zephyr/zephyr-project/arch/arm/soc/nxp_kinetis/k6x/linker.ld -o linker.cmd
( echo -nostartfiles -nodefaultlibs -nostdlib -static -Wl,-X -Wl,-N -Wl,--gc-sections -Wl,--build-id=none; echo "-Wl,-Map=zephyr.map"; echo "-L ./include/generated"; echo "-u _OffsetAbsSyms -u _ConfigAbsSyms"; echo "-e __start"; echo "-Wl,--start-group"; echo "-Wl,--whole-archive"; echo ""; echo " src/built-in.o"; echo "libzephyr.a"; echo "-Wl,--no-whole-archive"; echo "./arch/arm/core/offsets/offsets.o"; echo "-Wl,--end-group"; echo "-L /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/thumb -L /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/thumb -lgcc"; ) > zephyr.lnk
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-gcc -T linker.cmd @zephyr.lnk -o .tmp_zephyr.prebuilt
( /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-objdump -S zephyr.elf > zephyr.lst; /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-objcopy -S -O binary -R .note -R .comment -R COMMON -R .eh_frame zephyr.elf zephyr.bin; /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-strip -s -o zephyr.strip zephyr.elf; )
by Marcus Shawcroft:
Kumar, when run, does the example above print 0.0000 or something resembling 0.6 ?
by Kumar Gala:
I saw 0.0000 once, but than starting seeing 0.6.
Can you try removing '-mfpu=fpv4-sp-d16' from arch/arm/Makefile and see what you get. I'm noticing that -mfloat-abi=hard -mfpu=fpv4-sp-d16 isn't getting me the code I'd expect (with either the SDK toolchain or the ARM embedded toolchain). However if I drop the -mfpu option things seem more inline with what I'd expect.
by Marcus Shawcroft:
Configured for CONFIG_FP_HARDABI with '-mfpu=fpv4-sp-d16' from arch/arm/Makefile the link fails with:
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-ld -nostartfiles -nodefaultlibs -nostdlib -static -r -o src/built-in.o src/main.o
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-gcc -x assembler-with-cpp -nostdinc -undef -E -P \
-D__GCC_LINKER_CMD__ -I/home/marcus/work/zephyr/zephyr-project/include \
-I./include/generated /home/marcus/work/zephyr/zephyr-project/arch/arm/soc/nxp_kinetis/k6x/linker.ld -o linker.cmd
rm -f libzephyr.a; /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-ar rcTD libzephyr.a drivers/built-in.o lib/built-in.o kernel/built-in.o misc/built-in.o net/built-in.o boards/built-in.o ext/built-in.o usb/built-in.o fs/built-in.o arch/built-in.o
( echo -nostartfiles -nodefaultlibs -nostdlib -static -Wl,-X -Wl,-N -Wl,--gc-sections -Wl,--build-id=none; echo "-Wl,-Map=zephyr.map"; echo "-L ./include/generated"; echo "-u _OffsetAbsSyms -u _ConfigAbsSyms"; echo "-e __start"; echo "-Wl,--start-group"; echo "-Wl,--whole-archive"; echo ""; echo " src/built-in.o"; echo "libzephyr.a"; echo "-Wl,--no-whole-archive"; echo "./arch/arm/core/offsets/offsets.o"; echo "-Wl,--end-group"; echo "-L /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0 -L /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/. -lgcc"; ) > zephyr.lnk
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/bin/arm-poky-eabi/arm-poky-eabi-gcc -T linker.cmd @zephyr.lnk -o .tmp_zephyr.prebuilt
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: error: .tmp_zephyr.prebuilt uses VFP register arguments, /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/libgcc.a(_udivmoddi4.o) does not
/home/marcus/work/zephyr/install/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: failed to merge target specific data of file /home/marcus/work/zephyr/install/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/libgcc.a(_udivmoddi4.o)
by Juro Bystricky:
Well, I think the problem is there are missing ARM libgcc.a libraries for each library variant. I am looking into it.
by Juro Bystricky:
Confirmed. The binary is linked with incorrect libgcc.a (as it is the only one available - default tune). We have all libc.a (newlib) variants available, but libgcc.a variants are missing. Will be fixed in the next SDK (0.8.3 or 0.9)
by Kumar Gala:
Where are missing, I see:
./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/thumb/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv6-m/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7-m/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/softfp/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/softfp/fpv5-d16/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/softfp/fpv5-sp-d16/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/fpu/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/fpu/fpv5-d16/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/armv7e-m/fpu/fpv5-sp-d16/libgcc.a ./sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/fpu/libgcc.a
by Juro Bystricky:
Correct. All of the above will be present in the next SDK release.
by Kumar Gala:
Sorry, wasn't clear. I was saying that was my list of libgcc.a from the 0.8.2 sdk install, so not sure what we are missing.
by Juro Bystricky:
Argh, was looking at the next SDK under development. Yes, the libraries are present in SDK 0.8.2.
I was not able to reproduce the problem, but I have seen the link error while running sanitycheck (test_fp_shared for ARM). For some reason I don't see it anymore. (...uses VFP register arguments,...).
Marcus, just being paranoid: did you run "make pristine" after changing prj.conf? Can you check the Zephyr distro commit? (I use commit 2aa754497dc9d09b95445719811939334f0cf66b)
by Juro Bystricky:
Additionally, repeating the steps from above (disassembly), I see code that looks like thumb2 code: mix of 32 bit and 16 bit opcodes $OBJDUMP_ARM -d outdir/frdm_k64f/zephyr.elf | grep -A 10 aeabi_ddiv | head -n 10 > harddump.log
0000094c <__aeabi_ddiv>: 94c: b570 push {r4, r5, r6, lr} 94e: f04f 0cff mov.w ip, #255 ; 0xff 952: f44c 6ce0 orr.w ip, ip, #1792 ; 0x700 956: ea1c 5411 ands.w r4, ip, r1, lsr #20 95a: bf1d ittte ne 95c: ea1c 5513 andsne.w r5, ip, r3, lsr #20 960: ea94 0f0c teqne r4, ip 964: ea95 0f0c teqne r5, ip 968: f000 f8a7 bleq aba <__aeabi_ddiv+0x16e>
by Marcus Shawcroft:
Folks, I've looked at this again this morning with fresh eyes and concluded that I botched my original analysis. It looks like I installed 8.2 but continued to execute 8.1.
With 8.2, I do see sane behaviour, my test app runs as expected.
Juro, Kumar, sorry for the distraction.
by Juro Bystricky:
All's well that ends well. A simple FP test like this should be in the test suite though (maybe it is already).
by Mark Linkmeyer:
Correcting the priority field
Reported by Kumar Gala:
When enabling CONFIG_FLOAT=y we aren't linking in the correct libgcc.a on ARM architectures. There are a number of issues here:
/opt/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: error: /opt/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/fpu/libgcc.a(_powisf2.o) uses VFP register arguments, .tmp_zephyr.prebuilt does not /opt/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: failed to merge target specific data of file /opt/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/fpu/libgcc.a(_powisf2.o) /opt/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: error: /opt/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/fpu/libgcc.a(_udivmoddi4.o) uses VFP register arguments, .tmp_zephyr.prebuilt does not /opt/zephyr-sdk/sysroots/i686-pokysdk-linux/usr/libexec/arm-poky-eabi/gcc/arm-poky-eabi/5.2.0/real-ld: failed to merge target specific data of file /opt/zephyr-sdk/sysroots/armv5-poky-eabi/usr/lib/arm-poky-eabi/5.2.0/fpu/libgcc.a(_udivmoddi4.o) collect2: error: ld returned 1 exit status
This is due to building the kernel with -mfloat-abi=softfp, while the Zephyr toolchain only supports -mfloat-abi=hard.
While we can quickly fixing this with some minor tweaks, are more proper fix would utilize -print-libgcc-file-name to get the libgcc path and we'd pickup:
https://gerrit.zephyrproject.org/r/#/c/2899/
To get multilib support for -mfloat-abi=softfp, and -mfloat-abi=hard
(Imported from Jira ZEP-555)