search

ldc-developers / ldc

The LLVM-based D Compiler.
http://wiki.dlang.org/LDC
Other
1.18k stars 255 forks source link

[Power] 1.1.0b5: Large quantities of failing unit tests #1909

Open MingcongBai opened 7 years ago

MingcongBai commented 7 years ago

1.1.0b5 could not be built on PPC64 (PowerMac G5) an "Invalid operand types for ICmp instruction" error.

CFLAGS as follows:

-pipe -Wno-error -fstack-protector-strong --param=ssp-buffer-size=4 -fomit-frame-pointer -O2 -m64 -mtune=G5 -maltivec -maltivec=be -mabi=altivec -msecure-plt -mhard-float -fira-loop-pressure -fira-hoist-pressure -ftree-vectorize

LLVM/Clang info:

clang version 3.9.0 (tags/RELEASE_390/final)
Target: powerpc64-aosc-linux-gnu
Thread model: posix
InstalledDir: /bin

Build error:

Invalid operand types for ICmp instruction
  %7 = icmp slt float %6, 0.000000e+00
LLVM ERROR: Broken function found, compilation aborted!
make[2]: *** [runtime/CMakeFiles/druntime-ldc.dir/build.make:154: runtime/src/core/math.o] Error 1
make[2]: *** Waiting for unfinished jobs....
#0 0x00003fffb3d69c48 llvm::sys::PrintStackTrace(llvm::raw_ostream&) (/usr/lib/libLLVM-3.9.so+0x8acc48)
#1 0x00003fffb3d6a168 (/usr/lib/libLLVM-3.9.so+0x8ad168)
#2 0x00003fffb3d678b4 llvm::sys::RunSignalHandlers() (/usr/lib/libLLVM-3.9.so+0x8aa8b4)
#3 0x00003fffb3d67a84 (/usr/lib/libLLVM-3.9.so+0x8aaa84)
#4 0x00003fffb67a04d8 (linux-vdso64.so.1+0x4d8)
#5 0x0000000010063750 RootObject::compare(RootObject*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10063750)
#6 0x00000000102bad78 makeVarDValue(Type*, VarDeclaration*, llvm::Value*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102bad78)
#7 0x00000000102fd588 ToIRVisitor::visit(SwitchStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102fd588)
#8 0x00000000100925fc SwitchStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x100925fc)
#9 0x00000000102f7514 ToIRVisitor::visit(CompoundStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f7514)
#10 0x0000000010074438 CompoundStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10074438)
#11 0x00000000102f44e0 ToIRVisitor::visit(ScopeStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f44e0)
#12 0x0000000010075420 ScopeStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10075420)
#13 0x00000000102f44e0 ToIRVisitor::visit(ScopeStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f44e0)
#14 0x0000000010075420 ScopeStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10075420)
#15 0x00000000102f4e08 ToIRVisitor::visit(CaseStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f4e08)
#16 0x0000000010093018 CaseStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10093018)
#17 0x00000000102f7514 ToIRVisitor::visit(CompoundStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f7514)
#18 0x0000000010074438 CompoundStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10074438)
#19 0x00000000102f44e0 ToIRVisitor::visit(ScopeStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f44e0)
#20 0x0000000010075420 ScopeStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10075420)
#21 0x00000000102fc4a0 ToIRVisitor::visit(SwitchStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102fc4a0)
#22 0x00000000100925fc SwitchStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x100925fc)
#23 0x00000000102f7514 ToIRVisitor::visit(CompoundStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f7514)
#24 0x0000000010074438 CompoundStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10074438)
#25 0x00000000102f7514 ToIRVisitor::visit(CompoundStatement*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f7514)
#26 0x0000000010074438 CompoundStatement::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10074438)
#27 0x00000000102f780c Statement_toIR(Statement*, IRState*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102f780c)
#28 0x00000000102ab2a0 DtoDefineFunction(FuncDeclaration*, bool) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102ab2a0)
#29 0x000000001036d72c CodegenVisitor::visit(FuncDeclaration*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x1036d72c)
#30 0x0000000010159898 FuncDeclaration::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10159898)
#31 0x000000001036dafc CodegenVisitor::visit(StructDeclaration*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x1036dafc)
#32 0x0000000010217b98 StructDeclaration::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10217b98)
#33 0x000000001036ccec CodegenVisitor::visit(AttribDeclaration*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x1036ccec)
#34 0x000000001004b2f4 Visitor::visit(ProtDeclaration*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x1004b2f4)
#35 0x000000001027fe9c ProtDeclaration::accept(Visitor*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x1027fe9c)
#36 0x000000001036e798 Declaration_codegen(Dsymbol*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x1036e798)
#37 0x00000000102c6f08 codegenModule(IRState*, Module*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x102c6f08)
#38 0x0000000010381e68 ldc::CodeGenerator::emit(Module*) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10381e68)
#39 0x00000000103543e4 codegenModules(Array<Module*>&) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x103543e4)
#40 0x0000000010134a74 mars_mainBody(Array<char const*>&, Array<char const*>&) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10134a74)
#41 0x0000000010351c8c cppmain(int, char**) (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10351c8c)
#42 0x0000000010043784 _Dmain (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x10043784)
#43 0x00003fffb2cabd10 _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv (/usr/lib/libdruntime-ldc.so.71+0xe8d10)
#44 0x00003fffb2cabb84 _D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ7tryExecMFMDFZvZv (/usr/lib/libdruntime-ldc.so.71+0xe8b84)
#45 0x00003fffb2cab9f4 _d_run_main (/usr/lib/libdruntime-ldc.so.71+0xe89f4)
#46 0x00000000100438ac main (/var/lib/abbs/build/tmp.BI891kn0VQ/ldc-1.1.0-beta5-src/build/bin/ldc2+0x100438ac)
#47 0x00003fffb26849a8 (/usr/lib/libc.so.6+0x429a8)
#48 0x00003fffb2684bf8 __libc_start_main (/usr/lib/libc.so.6+0x42bf8)
make[2]: *** [runtime/CMakeFiles/druntime-ldc.dir/build.make:130: runtime/src/core/demangle.o] Trace/breakpoint trap (core dumped)
make[1]: *** [CMakeFiles/Makefile2:728: runtime/CMakeFiles/druntime-ldc.dir/all] Error 2
make: *** [Makefile:161: all] Error 2
MingcongBai commented 7 years ago

My bad, did not see #1905.

MingcongBai commented 7 years ago

Actually the issue in the original report is fixed. But there is now a new issue:

Wrong types for attribute: byval inalloca nest noalias nocapture nonnull readnone readonly signext sret zeroext dereferenceable(1) dereferenceable_or_null(1)
double (double)* @_D4core4math3cosFNaNbNiNfeZe
Wrong types for attribute: byval inalloca nest noalias nocapture nonnull readnone readonly signext sret zeroext dereferenceable(1) dereferenceable_or_null(1)
double (double)* @_D4core4math3cosFNaNbNiNfeZe
LLVM ERROR: Broken function found, compilation aborted!
make[2]: *** [runtime/CMakeFiles/druntime-ldc-debug.dir/build.make:154: runtime/src/core/math-debug.o] Error 1
make[2]: *** Waiting for unfinished jobs....
kinke commented 7 years ago

LDC was able to compile itself for the OP of #1904 with #1905 (ppc64 and ppc64le). The signature of double core.math.cos(double) is trivial and none of the LLVM attributes seem to make sense there. Could you please produce the textual IR for core.math via -output-ll -disable-verify and show us the IR for the cosine functions?

MingcongBai commented 7 years ago

Sorry but could you clarify on what I am supposed to do?

kinke commented 7 years ago

Never mind, I'll try it myself at home using your target triple.

kinke commented 7 years ago

So I did a ldc2 -mtriple=powerpc64-aosc-linux-gnu <path>/core/math.d -output-ll -g on my Windows box using LDC master and then looked at the generated math.ll file:

; [#uses = 0] [display name = core.math.cos]
define double @_D4core4math3cosFNaNbNiNfeZe(double %x_arg) #0 comdat !dbg !6 {
  %x = alloca double, align 8                     ; [#uses = 2, size/byte = 8]
  store double %x_arg, double* %x, !dbg !12       ; [debug line = 50:10]
  call void @llvm.dbg.declare(metadata double* %x, metadata !11, metadata !13), !dbg !12 ; [debug line = 50:10] [debug variable = x]
  %1 = load double, double* %x, !dbg !14          ; [#uses = 1] [debug line = 50:43]
  %2 = call double @llvm.cos.f64(double %1) #2, !dbg !14 ; [#uses = 1] [debug line = 50:43]
  ret double %2, !dbg !14                         ; [debug line = 50:43]
}

...

attributes #0 = { "less-precise-fpmad"="false" "no-frame-pointer-elim"="false" "no-infs-fp-math"="false" "no-nans-fp-math"="false" "target-cpu" "target-features"="+64bit" "unsafe-fp-math"="false" }

So no LLVM attributes in the signature itself, and accordingly no error. Can you try to do the same (maybe add -disable-verify if the error persists and no .ll file is generated)?

MingcongBai commented 7 years ago 
; [#uses = 0] [display name = core.math.cos]
define double @_D4core4math3cosFNaNbNiNfeZe(double %x_arg) #0 comdat !dbg !6 {
  %x = alloca double, align 8                     ; [#uses = 2, size/byte = 8]
  store double %x_arg, double* %x, !dbg !12       ; [debug line = 50:10]
  call void @llvm.dbg.declare(metadata double* %x, metadata !11, metadata !13), !dbg !12 ; [debug line = 50:10] [debug variable = x]
  %1 = load double, double* %x, !dbg !14          ; [#uses = 1] [debug line = 50:43]
  %2 = call double @llvm.cos.f64(double %1) #2, !dbg !14 ; [#uses = 1] [debug line = 50:43]
  ret double %2, !dbg !14                         ; [debug line = 50:43]
}
...
attributes #0 = { "unsafe-fp-math"="false" }
attributes #1 = { nounwind readnone }
attributes #2 = { nounwind readnone "unsafe-fp-math"="false" }
attributes #3 = { noinline }

And no error was reported. So how exactly did it fail during the build...

kinke commented 7 years ago

And using -c instead of -output-ll works too, i.e., generating the object file? Then you may want to look inside the Makefile and check which cmdline flags are actually used to compile core/math.d to math-debug.o.

MingcongBai commented 7 years ago 
root@ppc64 [ ldc-1.1.0-beta5-src@staging ] # ldc2 -mtriple=powerpc64-aosc-linux-gnu runtime/druntime/src/core/math.d -c
root@ppc64 [ ldc-1.1.0-beta5-src@staging ] # file math.o
math.o: ELF 64-bit MSB relocatable, 64-bit PowerPC or cisco 7500, version 1 (SYSV), not stripped

Yes the build does pass, generating an object file in correct format. And if I am not mistaken, this is where the flags to compile math-debug.o are located: https://github.com/ldc-developers/ldc/blob/master/runtime/CMakeLists.txt#L15.

kinke commented 7 years ago

Those aren't the final ones, so checking the Makefile makes sense. You should also omit the -mtriple, I just had to use it for cross-compilation from my Win64 box.

kinke commented 7 years ago

This definitely seems to be an ABI issue, so are you sure the LDC used to compile the runtime is the fixed final one? How did you bootstrap it, i.e., which D host compiler did you use?

MingcongBai commented 7 years ago

This definitely seems to be an ABI issue, so are you sure the LDC used to compile the runtime is the fixed final one? How did you bootstrap it, i.e., which D host compiler did you use?

Ah ha, now that I would suspect that I should re-bootstrap then. I am currently using version 1.1.0b3. Do you think I should drop back down to 0.17 and restart again?

kinke commented 7 years ago

Oh yes. :)

MingcongBai commented 7 years ago

Well then, I will be back in about half an hour.

kinke commented 7 years ago

Latest 0.17 has most ABI fixes, but not all, I just cherry-picked some fixes of #1905 into the ltsmaster (0.17) branch, so using a fresh ltsmaster for bootstrapping would be even more ideal.

MingcongBai commented 7 years ago

Got it.

MingcongBai commented 7 years ago

Wasn't able to install with ltsmaster...

-- Installing: /var/lib/abbs/build/tmp.onDmuy1395/ldc/abdist/usr/share/bash-completion/completions/ldc2
CMake Error at runtime/cmake_install.cmake:44 (file):
  file INSTALL cannot find
  "/var/lib/abbs/build/tmp.onDmuy1395/ldc/runtime/druntime/src/object.d".
Call Stack (most recent call first):
  cmake_install.cmake:97 (include)
kinke commented 7 years ago

Did you do a git submodule update and make sure your src tree is clean with git status? Btw installing isn't necessary, the compiler in the working/build directory is fine too (and explicitly setting it for LDC master's CMake cmdline via -DD_COMPILER=<path to ltsmaster ldmd2>).

MingcongBai commented 7 years ago

I am currently building with 0.17.1, and no error so far - 39%.

MingcongBai commented 7 years ago

Let me close this issue then, it's probably due to the fact that I am bootstrapping using a faulty compiler... Sorry for the trouble.

kinke commented 7 years ago

It would be awesome if you could run the library unittests for us, as we're currently completely in the dark wrt. PPC(64). It doesn't take that long if you have a few CPU cores; the debug ones only take an estimated 5 mins on my quadcore. You'd do that by invoking make -j<N> druntime-ldc-unittest-debug phobos2-ldc-unittest-debug && ctest -R "-debug" -E "dmd-testsuite|lit-tests" in the build directory.

MingcongBai commented 7 years ago

Sure thing, I will post the test log once I am done. Also you are welcome to pop me an email so I can provide you with SSH access if need be.

kinke commented 7 years ago

Thank you very much already. The tests in release mode would be make -j<N> druntime-ldc-unittest phobos2-ldc-unittest && ctest -E "-debug|dmd-testsuite|lit-tests" ;), but the compilation takes longer by something like a factor of 3 or so.

MingcongBai commented 7 years ago

I am currently stuck on 249: std.random, the test appears to take up 100/400% processor utilization. It's been about 10 minutes now. The others passed through quite quickly but a large number of them failed.

MingcongBai commented 7 years ago

Also strace attach did not return any output...

kinke commented 7 years ago

Hmm okay, not looking too good then. Just kill the hanging process (phobos-test-runner std.random or so). The log is in Testing/Temporary/LastTest.log IIRC; hopefully it'll spit out a fundamental druntime issue accounting for most failures.

MingcongBai commented 7 years ago

Okay, and it goes on, hang on...

MingcongBai commented 7 years ago

Here you are.

MingcongBai commented 7 years ago

Renamed issue and re-opened.

kinke commented 7 years ago

Very nice, thanks. Unfortunately, it looks as if there's a multitude of issues in Phobos (although std/typecons.d(481): Assertion failure occurs quite frequently). Are you on a big-endian platform?

MingcongBai commented 7 years ago

Yes, a Power Mac G5 with POWER4 (I think) derived PowerPC 970FX.

kinke commented 7 years ago

The std.typecons:481 is a failing string/array comparison, so that should be a fundamental bug [edit: nope, there's a double formatted into the string, and floating point values seem to be garbage]. Can I persuade you to do one more test run, this time the LDC-flavored DMD testsuite, taking a few mins as well? ctest -R dmd-testsuite-debug, LastTest.log is the only needed file.

MingcongBai commented 7 years ago

Right, give me a minute.

MingcongBai commented 7 years ago

Here you go, https://pastebin.aosc.io/view/efb009ad.

kinke commented 7 years ago

And Phobos isn't big-endian aware (at least not everywhere), e.g., there's this test in std.outbuffer:

OutBuffer buf = new OutBuffer();
"hello"w.copy(buf);
assert(buf.toBytes() == "h\x00e\x00l\x00l\x00o\x00");

toBytes() returns the underlying raw ubyte[].

kinke commented 7 years ago

Thanks, but dmd-testsuite-debug horribly failed, it wasn't even able to build (or execute?) the testrunner due to the std.typecons:481 assertion (although I don't know why that Tuple unittest is even run). Anyway, thanks for giving us a picture of where we're currently at on PPC64!

MingcongBai commented 7 years ago

You are very welcome sir.

kinke commented 7 years ago

Multi-byte Unicode encoding also appears to be wrong (UTF-8 at least), this fails (from std.string):

assert(lastIndexOfAny(to!string("ÖABCDEFCDEF"), to!string("ö"), 2, CaseSensitive.no) == 0);

Edit: Nope, the case-sensitive check works, so it must be the case-insensitiveness (toLower/Upper() failing for big-endian?).

dnadlinger commented 7 years ago

I must admit I was largely ignorant to the state PPC64 altogether – did 1.0 work better (i.e. are these regressions)? In any case, it seems unlikely that we can get much further for this release. @redstar?

kinke commented 7 years ago

The ABI issues were there in 1.0 at least, that's for sure, making the more recent LDC 0.17 versions the only halfway-working ones for Power targets. It's quite astonishing that LDC can build itself, the runtimes and their unittests successfully, but that a lot of tests fails for a seemingly pretty small number of rudimentary issues. Stuff like this certainly doesn't help, and that's in there since forever.