Incompatible ABIs for target emulation in GCC 7.2.0

[chiragbharadwaj]

Hi, I'm having a bit of an issue with ABI target emulation due to what looks like incompatible ABIs, based on the error messages.

So the situation is that I need to generate 32-bit (-compatible) RISC-V assembly code, but I have a 64-bit RISC-V GNU toolchain installed natively on my computer. I built the toolchain from source about a week ago using homebrew-riscv. You can find my computer specs below. To this extent, I've been trying to run riscv64-unknown-elf-gcc in a way that 32-bit RISC-V assembly code is generated. This usually works out fine for me when I use the compiler flags -march=rv32im -mabi=ilp32. Even now, this works most of the time.

However, I was trying to run the following sequence of commands to cross-compile a benchmark last night:

riscv64-unknown-elf-g++ -Wall -g -O3 -D_MIPS_ARCH_LLFU -MMD -MP -I.  -I../ubmark  -c ubmark_.cc
riscv64-unknown-elf-ar rcv libubmark.a ubmark_.o
riscv64-unknown-elf-ranlib libubmark.a
riscv64-unknown-elf-gcc -Wall -g -O3 -march=rv32im -mabi=ilp32 -funroll-loops -MMD -MP -I.  -I../ubmark  -c ../ubmark/ubmark-vvadd.c
riscv64-unknown-elf-gcc -L.  -nostartfiles  -o ubmark-vvadd ubmark-vvadd.o -lubmark

The first four go through without fault. For the fifth one, though, I ended up getting this error message instead:

ld: ubmark-vvadd.o: ABI is incompatible with that of the selected emulation:
  target emulation `elf32-littleriscv' does not match `elf64-littleriscv'
ld: failed to merge target specific data of file ubmark-vvadd.o
ld: warning: cannot find entry symbol _start; defaulting to 00000000000100b0

To me, this looks like some sort of link-time or load-time error. Could somebody please shed some light as to what might be causing this? I suspect that I am not passing the correct flags to get the files to be linked in correctly. When I looked this error up on previous Github issues, all I could find were a few submissions from 2015/2016 describing something called --enable-multilib that I should have included while building from source, apparently. I'm not sure how that figures into my setup, since I built using Homebrew as was suggested on this repo. I'd really appreciate some help figuring out this ABI issue so that I can virtualize the 32-bit code correctly without having to bother using a VM or something.

I think I might be missing a file called libgcc.a? This is what I could gather from reading various sources online. Confirmation in either direction would be greatly appreciated.

Thank you so much for your help in advance.

Computer specs:

• Macbook Pro, mid-2015
• Mac OS X 10.10.5, "Yosemite"
• 2.8 GHz, Intel Core i7
• riscv64-unknown-elf-ar 2.29
• riscv64-unknown-elf-ranlib 2.29
• riscv64-unknown-elf-gcc 7.2.0

Please let me know if you need to know anything else.

[sorear]

In the commands you pasted, the first command (ubmark_.cc) is missing -march ?

[chiragbharadwaj]

The g++ command? When I add the same flags for that command in my Makefile, I seem to get the same errors as above. :o

[sorear]

You should pass -march and -mabi to every gcc and g++ invocation, including those used to link.

[chiragbharadwaj]

Okay, thanks. I added them in and I will let you know what happens once I compile it.

[chiragbharadwaj]

So this is what I am running now:

riscv64-unknown-elf-g++ -Wall -g -O3 -march=rv32im -mabi=ilp32 -D_MIPS_ARCH_LLFU -MMD -MP -I.  -I../ubmark  -c ubmark_.cc
riscv64-unknown-elf-ar rcv libubmark.a ubmark_.o
riscv64-unknown-elf-ranlib libubmark.a
riscv64-unknown-elf-gcc -Wall -g -O3 -march=rv32im -mabi=ilp32 -funroll-loops -MMD -MP -I.  -I../ubmark  -c ../ubmark/ubmark-vvadd.c
riscv64-unknown-elf-gcc -march=rv32im -mabi=ilp32 -L.  -T ../scripts/ubmark.ld -nostartfiles  -o ubmark-vvadd ubmark-vvadd.o -lubmark

And the errors are again on the final invocation above:

ld: lib/libc.a(lib_a-memset.o): ABI is incompatible with that of the selected emulation:
  target emulation `elf64-littleriscv' does not match `elf32-littleriscv'
ld: failed to merge target specific data of file lib/libc.a(lib_a-memset.o)
ld: ubmark-vvadd(.text): relocation ".Laligned+0x0 (type R_RISCV_RVC_JUMP)" goes out of range
ld: lib/libc.a(lib_a-memset.o): file class ELFCLASS64 incompatible with ELFCLASS32
ld: final link failed: File in wrong format

It looks like the error is occurring at a different place now, but I still have this 64-bit/32-bit clash. Do you think that I may be missing the flags for virtualization at other invocation points as well? Thank you for all of your help so far, by the way. This definitely cleared things up.

Also, the ld and lib/libc.a paths above are relativized for ease of reading. If you need the fully-qualified names, they are as shown above, except prepended with the path /usr/local/Cellar/riscv-gcc/gnu/lib/gcc/riscv64-unknown-elf/7.2.0/../../../../riscv64-unknown-elf.

[sorear]

My best guest at this point is that you have a non-multilib toolchain build; a multilib toolchain would have lib32/ilp32/libc.a. Since your toolchain has only one libc.a and it's 64-bit you're out of luck (?) without a new toolchain.

[I only just noticed this is the homebrew repo and you didn't build the toolchain yourself]

[chiragbharadwaj]

I believe that this is the case as well. Is there any way for me to build from source and include that libc.a or somehow "append" a multilib toolchain? If not, is there a good way to enable multilib when building from source via Homebrew's riscv-tools package? I don't mind removing my entire toolchain and then rebuilding as long as I can guarantee a multilib version on OS X somehow.

[sorear]

If you were building from source without homebrew, I would tell you to check out https://github.com/riscv/riscv-gnu-toolchain and do ./configure --prefix=whereever --enable-multilib --enable-linux. riscv-tools is less actively maintained. Can't speak about homebrew.

[chiragbharadwaj]

Gotcha. So I did that originally before discovering the Homebrew repo, but the thing doesn't build (presumably because I have to run make linux and I don't have a Linux device running natively). It takes about 15-20 minutes to build from source and then it just outputs a bunch of errors. =(

Perhaps I will investigate this further and post on that repo's issue board if needed. Thanks for all of your help.

[sorear]

make linux means that it compiles executables that will run under Linux on RISC-V (as opposed to bare metal, which make newlib gets you). Not the host system. I'm not surprised it breaks though; I should duck out now so that someone who has used macOS and Homebrew in the past 12 months can take over.

[chiragbharadwaj]

Ah okay. No worries and thanks again. I will post specific questions to the other board if I still need help later on.

[sbeamer]

I did attempt to allow users to build a multilib version if they wanted. If you use brew info riscv-gcc, you will see a summary of the flags. Hopefully, rebuilding it with --with-multilib should work, and if not, please post on this issue.