[WASM] wasm/function-index.test failing

[Quuxplusone]

Bugzilla Link	PR40278
Status	NEW
Importance	P normal
Reported by	Heejin Ahn (aheejin@gmail.com)
Reported on	2019-01-09 17:21:55 -0800
Last modified on	2019-05-15 17:16:28 -0700
Version	unspecified
Hardware	PC Linux
CC	dschuff@google.com, greened@obbligato.org, llvm-bugs@lists.llvm.org, ruiu@google.com, sbc@chromium.org, smithp352@googlemail.com, srj@google.com
Fixed by commit(s)
Attachments
Blocks
Blocked by
See also

This bug is to keep track of a bug reported by David Greene in llvm-dev mailing
list. I'll paste the his mail contents along the links.

By the way, there's no "Wasm" component in the lld category here. How can I add
that?

---

From http://lists.llvm.org/pipermail/llvm-dev/2019-January/128921.html:

I'm seeing the following fail on Linux x86-64, for the last couple weeks
at least.  This is from 'ninja check-all'.

                                 -David

FAIL: lld :: wasm/function-index.test (1941 of 1955)
******************** TEST 'lld :: wasm/function-index.test' FAILED
********************
Script:
--
: 'RUN: at line 1';   /build/x86_64/bin/llc -filetype=obj
/src/lld/test/wasm/Inputs/ret32.ll -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret32.o
: 'RUN: at line 2';   /build/x86_64/bin/llc -filetype=obj
/src/lld/test/wasm/Inputs/ret64.ll -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret64.o
: 'RUN: at line 3';   /build/x86_64/bin/wasm-ld -r -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.wasm
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret32.o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret64.o
: 'RUN: at line 4';   /build/x86_64/bin/obj2yaml
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.wasm |
/build/x86_64/bin/FileCheck /src/lld/test/wasm/function-index.test
--
Exit Code: 139

Command Output (stderr):
--
+ : 'RUN: at line 1'
+ /build/x86_64/bin/llc -filetype=obj /src/lld/test/wasm/Inputs/ret32.ll -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret32.o
+ : 'RUN: at line 2'
+ /build/x86_64/bin/llc -filetype=obj /src/lld/test/wasm/Inputs/ret64.ll -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret64.o
+ : 'RUN: at line 3'
+ /build/x86_64/bin/wasm-ld -r -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.wasm
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret32.o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret64.o
Stack dump:
0.  Program arguments: /build/x86_64/bin/wasm-ld -r -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.wasm
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret32.o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret64.o
 #0 0x0000000000571d1a llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/build/x86_64/bin/wasm-ld+0x571d1a)
 #1 0x000000000056f538 llvm::sys::RunSignalHandlers() (/build/x86_64/bin/wasm-ld+0x56f538)
 #2 0x000000000056f64c SignalHandler(int) (/build/x86_64/bin/wasm-ld+0x56f64c)
 #3 0x00007f7578656b00 __restore_rt (/lib64/libpthread.so.0+0x10b00)
 #4 0x00000000005b618b llvm::SpecificBumpPtrAllocator<std::unique_ptr<llvm::MemoryBuffer, std::default_delete<llvm::MemoryBuffer> > >::DestroyAll() (/build/x86_64/bin/wasm-ld+0x5b618b)
 #5 0x000000000091af30 lld::freeArena() (/build/x86_64/bin/wasm-ld+0x91af30)
 #6 0x00000000007b9e5c lld::wasm::link(llvm::ArrayRef<char const*>, bool, llvm::raw_ostream&) (/build/x86_64/bin/wasm-ld+0x7b9e5c)
 #7 0x00000000004a67cf main (/build/x86_64/bin/wasm-ld+0x4a67cf)
 #8 0x00007f7576e656e5 __libc_start_main (/lib64/libc.so.6+0x206e5)
 #9 0x0000000000557729 _start /home/abuild/rpmbuild/BUILD/glibc-2.22/csu/../sysdeps/x86_64/start.S:121:0
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.script: line 4:
368695 Segmentation fault      /build/x86_64/bin/wasm-ld -r -o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.wasm
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret32.o
/build/x86_64/tools/lld/test/wasm/Output/function-index.test.tmp.ret64.o

---

From http://lists.llvm.org/pipermail/llvm-dev/2019-January/128951.html:

> Rui Ueyama <ruiu at google.com> writes:
>
>> I cannot reproduce this error, but this could be real.
>>
>> David, is this reproducible every time or is this flaky?
>
> It's every time for me, both on X86-64 and AArch64.  I'll try running in
> a debugger to see what is going on.  What else can I do to help track
> this down?

Ok, I discovered something interesting.  I see the failure when
configuring like this:

  cmake -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_ASSERTIONS=ON \
        -DLLVM_ABI_BREAKING_CHECKS=WITH_ASSERTS ...

I do *not* see the failure when configuring like this:

  cmake -DCMAKE_BUILD_TYPE=Debug -DLLVM_ENABLE_ASSERTIONS=ON \
        -DLLVM_ABI_BREAKING_CHECKS=WITH_ASSERTS ...

I set LLVM_ABI_BREAKING_CHECKS=FORCE_OFF for the Release build in
CMakeCache.txt and it still failed.

I then additionally set LLVM_ENABLE_ASSERTIONS=OFF for the Release build
in CMakeCache.txt and it still failed.

I then attempted to run the command in a debugger.  It passed.  It also
passed on the command-line.  So it appears something about the
environment lit sets up causes problems, but only for Release builds.

I tried adding a batch gdb command to the test but it passed.  The test
doesn't run long enough to allow attaching an external gdb.

So short of hacking on wasm-ld, I'm at a bit of a roadblock.  Ideas
welcome.  :)

---

From http://lists.llvm.org/pipermail/llvm-dev/2019-January/128952.html:

> Are you using static linking, -DBUILD_SHARED_LIBS, or -
> DLLVM_LINK_LLVM_DYLIB?

I use neither -DBUILD_SHARED_LIBS nor -DLLVM_LINK_LLVM_DYLIB.

I do have -DLLVM_ENABLE_CXX1Y=ON, -DLLVM_ENABLE_THREADS=OFF,
-DCLANG_DEFAULT_LINKER=gold.  I also have -DLLVM_ENABLE_EH=ON
-DLLVM_ENABLE_RTTI=ON which are obviously very atypical.

[Quuxplusone]

FYI: I'm seeing a crash failure with essentially identical stacktrace steps in my project (Halide); the difference is that this isn't when running wasm-ld via commandline, but rather, I'm linking in the lld libraries and calling lld::wasm::link() directly from my code (with CanExitEarly=False, of course).

The interesting thing here is that I only get this crash if LLVM is built with GCC instead of Clang, but that may just be a peculiarity of my setup; the crash will replicate for me with GCC 5.3 and GCC 7.3 on Ubuntu-variant systems, but I haven't nailed this down to a GCC bug at all. (That said, the fact that it only occurs with HCC and only in non-debug builds makes me a little suspicious.)

[Quuxplusone]

(BTW: On the two local systems I've seen this on, it's 100% reproducible. Anything I can do to gather useful info?)

[Quuxplusone]

One other note: I am compiling with LLVM_ENABLE_RTTI=ON for both the crashy and non-crashy cases. (I point this out since it is likely a bit unusual and is also something in common with the original reporter.)

[Quuxplusone]

@greened, are you building llvm with gcc when you see this issue?

[Quuxplusone]

I've tried to repro this using gcc to build but do far no luck. Any more repro tips welcome.

[Quuxplusone]

I can repro this 100% of the time with a build that includes Halide as part of the mix. Repro instructions are a little complex; I can post here or send directly to you if you like.

[Quuxplusone]

Actually I have now reproduced it using gcc.

It seems that it only effects GCC and only release builds.

Also it doesn't reproduce if you run the command line directly rather then
running from within lit.

Can you conform that last part?

Also, I am seeing this in 80 or the 94 tests in lld/test/wasm.  Are you seeing
this too?

[Quuxplusone]

Also it doesn't reproduce if you run the command line directly rather then running from within lit. Can you conform that last part?

Yes, because running from lit skips the "CanExitEarly" path, and the bug only occurs when the arenas are destroyed (which is skipped via exit() in that path).

Also, I am seeing this in 80 or the 94 tests in lld/test/wasm. Are you seeing this too?

I haven't evaluated it there; I've seen it when linking lld-wasm directly to my own code.

[Quuxplusone]

Oh! Thanks for the tip regarding LLD_IN_TEST environment variable. Thats nasty.

However... even with that I still don't see the crash the arena is freed just fine when I run directly from the command line.

[Quuxplusone]

Redirecting stdout to file makes the problem manifest. So now I and repro outside of lit. I have not idea why that would effect this ..

[Quuxplusone]

I have not idea why that would effect this

Yeah, this smells like it's gonna be something really weird and/or ugly

[Quuxplusone]

Also doesn't repro with in RelWithDebInfo.. only Release. Great.