Closed shoe42 closed 2 years ago
For this one it would be helpful to get a full backtrace. Make sure to compile Terra in debug mode (-DCMAKE_BUILD_TYPE=Debug
if you're using CMake, and also make sure to use a fresh build directory).
Apologies, that's all the output from Terra itself - from a debugger, there's:
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
frame #0: 0x00000001a314ad98 libsystem_kernel.dylib`__pthread_kill + 8
frame #1: 0x00000001a317fee0 libsystem_pthread.dylib`pthread_kill + 288
frame #2: 0x00000001a30ba340 libsystem_c.dylib`abort + 168
frame #3: 0x00000001a30b9754 libsystem_c.dylib`__assert_rtn + 272
* frame #4: 0x0000000100038cc4 terra`llvm::checkGEPType(Ty=0x0000000000000000) at Instructions.h:921:3
frame #5: 0x0000000100038b10 terra`llvm::GetElementPtrInst::getGEPReturnType(ElTy=0x00000001098ec400, Ptr=0x0000600002908100, IdxList=ArrayRef<llvm::Value *> @ 0x000000016fdfac58) at Instructions.h:1096:26
frame #6: 0x0000000100038970 terra`llvm::GetElementPtrInst::GetElementPtrInst(this=0x0000600003508110, PointeeType=0x00000001098ec400, Ptr=0x0000600002908100, IdxList=ArrayRef<llvm::Value *> @ 0x000000016fdfad38, Values=3, NameStr=0x000000016fdfae40, InsertBefore=0x0000000000000000) at Instructions.h:1169:19
frame #7: 0x00000001000388e8 terra`llvm::GetElementPtrInst::GetElementPtrInst(this=0x0000600003508110, PointeeType=0x00000001098ec400, Ptr=0x0000600002908100, IdxList=ArrayRef<llvm::Value *> @ 0x000000016fdfada0, Values=3, NameStr=0x000000016fdfae40, InsertBefore=0x0000000000000000) at Instructions.h:1173:63
frame #8: 0x00000001000387ac terra`llvm::GetElementPtrInst::Create(PointeeType=0x00000001098ec400, Ptr=0x0000600002908100, IdxList=ArrayRef<llvm::Value *> @ 0x000000016fdfae20, NameStr=0x000000016fdfae40, InsertBefore=0x0000000000000000) at Instructions.h:962:25
frame #9: 0x000000010032a7dc terra`llvm::IRBuilderBase::CreateConstInBoundsGEP2_32(llvm::Type*, llvm::Value*, unsigned int, unsigned int, llvm::Twine const&) + 164
frame #10: 0x00000001023b32e8 terra`clang::CodeGen::CGBuilderTy::CreateStructGEP(clang::CodeGen::Address, unsigned int, llvm::Twine const&) + 112
frame #11: 0x0000000102615e40 terra`(anonymous namespace)::AArch64ABIInfo::EmitVAArg(clang::CodeGen::CodeGenFunction&, clang::CodeGen::Address, clang::QualType) const + 744
frame #12: 0x00000001024bd224 terra`clang::StmtVisitorBase<std::__1::add_pointer, (anonymous namespace)::ScalarExprEmitter, llvm::Value*>::Visit(clang::Stmt*) + 8812
frame #13: 0x00000001024b6034 terra`(anonymous namespace)::ScalarExprEmitter::Visit(clang::Expr*) + 52
frame #14: 0x00000001024bfa3c terra`(anonymous namespace)::ScalarExprEmitter::EmitBinOps(clang::BinaryOperator const*) + 52
frame #15: 0x00000001024bfe78 terra`(anonymous namespace)::ScalarExprEmitter::EmitCompare(clang::BinaryOperator const*, llvm::CmpInst::Predicate, llvm::CmpInst::Predicate, llvm::CmpInst::Predicate, bool) + 896
frame #16: 0x00000001024b6034 terra`(anonymous namespace)::ScalarExprEmitter::Visit(clang::Expr*) + 52
frame #17: 0x00000001024b5ff4 terra`clang::CodeGen::CodeGenFunction::EmitScalarExpr(clang::Expr const*, bool) + 44
frame #18: 0x000000010248a738 terra`clang::CodeGen::CodeGenFunction::EvaluateExprAsBool(clang::Expr const*) + 288
frame #19: 0x0000000102598bb0 terra`clang::CodeGen::CodeGenFunction::EmitBranchOnBoolExpr(clang::Expr const*, llvm::BasicBlock*, llvm::BasicBlock*, unsigned long long, clang::Stmt::Likelihood) + 696
frame #20: 0x000000010255c808 terra`clang::CodeGen::CodeGenFunction::EmitIfStmt(clang::IfStmt const&) + 748
frame #21: 0x0000000102562bd0 terra`clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope(clang::CompoundStmt const&, bool, clang::CodeGen::AggValueSlot) + 316
frame #22: 0x0000000102597920 terra`clang::CodeGen::CodeGenFunction::EmitFunctionBody(clang::Stmt const*) + 100
frame #23: 0x0000000102597f30 terra`clang::CodeGen::CodeGenFunction::GenerateCode(clang::GlobalDecl, llvm::Function*, clang::CodeGen::CGFunctionInfo const&) + 704
frame #24: 0x00000001025ab838 terra`clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition(clang::GlobalDecl, llvm::GlobalValue*) + 308
frame #25: 0x00000001025a723c terra`clang::CodeGen::CodeGenModule::EmitGlobalDefinition(clang::GlobalDecl, llvm::GlobalValue*) + 260
frame #26: 0x00000001025ad8bc terra`clang::CodeGen::CodeGenModule::EmitTopLevelDecl(clang::Decl*) + 240
frame #27: 0x00000001025ff8b0 terra`(anonymous namespace)::CodeGeneratorImpl::HandleTopLevelDecl(clang::DeclGroupRef) + 140
frame #28: 0x00000001000743d8 terra`CodeGenProxy::HandleTopLevelDecl(this=0x0000600003900dd0, D=DeclGroupRef @ 0x000000016fdfe678) at tcwrapper.cpp:592:20
frame #29: 0x000000010262f314 terra`clang::ParseAST(clang::Sema&, bool, bool) + 392
frame #30: 0x0000000100068d88 terra`dofile(T=0x0000000109569d28, TT=0x0000600002c00080, code="#include <stdio.h>\n#include <string.h>\n#include <stdarg.h>\n\nint checkargs(const char* f, va_list vl)\n{\n if(strcmp(f, \"%i%i%f%s\"))\n return 1;\n \n if(va_arg(vl, int) != -10)\n return 2;\n if(va_arg(vl, unsigned long long) != 60000000000001ULL)\n return 3;\n if(va_arg(vl, double) != 60.01)\n return 4;\n if(strcmp(va_arg(vl, const char*), \"TesT\"))\n return 5;\n return 0;\n}\n\n", args=size=13, result=0x000000016fdfeba8) at tcwrapper.cpp:1045:5
frame #31: 0x0000000100068914 terra`include_c(L=0x0000000109560380) at tcwrapper.cpp:1129:9
frame #32: 0x0000000103070328 terra`lj_BC_FUNCC + 44
frame #33: 0x0000000103017d74 terra`lua_pcall(L=<unavailable>, nargs=<unavailable>, nresults=<unavailable>, errfunc=<unavailable>) at lj_api.c:1145:12 [opt]
frame #34: 0x000000010000517c terra`docall(L=0x0000000109560380, narg=0, clear=0) at main.cpp:339:14
frame #35: 0x0000000100004d24 terra`main(argc=2, argv=0x000000016fdfef48) at main.cpp:119:13
frame #36: 0x000000010936108c dyld`start + 520
Based on this backtrace, it sounds like the following Terra file should be sufficient to make this fail:
bug605.t
:
C = terralib.includecstring [[
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
int checkargs(const char* f, va_list vl)
{
if(strcmp(f, "%i%i%f%s"))
return 1;
if(va_arg(vl, int) != -10)
return 2;
if(va_arg(vl, unsigned long long) != 60000000000001ULL)
return 3;
if(va_arg(vl, double) != 60.01)
return 4;
if(strcmp(va_arg(vl, const char*), "TesT"))
return 5;
return 0;
}
]]
Can you confirm?
Yup, looks like that triggers it too:
Assertion failed: (Ty && "Invalid GetElementPtrInst indices for type!"), function checkGEPType, file Instructions.h, line 921.
I suppose the next test is just to extract that C snippet out and see if we hit the same error in Clang:
bug605.c
:
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
int checkargs(const char* f, va_list vl)
{
if(strcmp(f, "%i%i%f%s"))
return 1;
if(va_arg(vl, int) != -10)
return 2;
if(va_arg(vl, unsigned long long) != 60000000000001ULL)
return 3;
if(va_arg(vl, double) != 60.01)
return 4;
if(strcmp(va_arg(vl, const char*), "TesT"))
return 5;
return 0;
}
clang bug605.c -o bug605.o -c -Wall
And ideally run this with the same Clang you used for building Terra.
Clang seems to be able to build that fine. Will see if I can try get any info from inside the debugger at some point.
Since Clang compiles it fine, could you try:
clang bug605.c -o bug605.ll -S -emit-llvm -Wall
And attach that? It should provide us information on what Clang is doing.
After that we'd need to figure out the equivalent way to get information out of Terra, which will be more tricky because it's failing before it can even construct the IR.
But I suspect what's going on (and probably in #603 too) is that we're somehow configuring LLVM differently. Maybe the target machines are different or not entirely compatible, or something like that.
Here's Terra file you can run to at least give us the target and data layout for Terra:
terra f() return 123 end
print(terralib.saveobj(nil, "llvmir", {f=f}, nil, nil, false))
IR for bug605.c
:
; ModuleID = 'bug605.c'
source_filename = "bug605.c"
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
target triple = "arm64-apple-macosx12.0.0"
@.str = private unnamed_addr constant [9 x i8] c"%i%i%f%s\00", align 1
@.str.1 = private unnamed_addr constant [5 x i8] c"TesT\00", align 1
; Function Attrs: noinline nounwind optnone ssp uwtable
define i32 @checkargs(i8* %0, i8* %1) #0 {
%3 = alloca i32, align 4
%4 = alloca i8*, align 8
%5 = alloca i8*, align 8
%6 = alloca i32, align 4
%7 = alloca i64, align 8
%8 = alloca double, align 8
%9 = alloca i8*, align 8
store i8* %0, i8** %4, align 8
store i8* %1, i8** %5, align 8
%10 = load i8*, i8** %4, align 8
%11 = call i32 @strcmp(i8* %10, i8* getelementptr inbounds ([9 x i8], [9 x i8]* @.str, i64 0, i64 0))
%12 = icmp ne i32 %11, 0
br i1 %12, label %13, label %14
13: ; preds = %2
store i32 1, i32* %3, align 4
br label %36
14: ; preds = %2
%15 = va_arg i8** %5, i32
store i32 %15, i32* %6, align 4
%16 = load i32, i32* %6, align 4
%17 = icmp ne i32 %16, -10
br i1 %17, label %18, label %19
18: ; preds = %14
store i32 2, i32* %3, align 4
br label %36
19: ; preds = %14
%20 = va_arg i8** %5, i64
store i64 %20, i64* %7, align 8
%21 = load i64, i64* %7, align 8
%22 = icmp ne i64 %21, 60000000000001
br i1 %22, label %23, label %24
23: ; preds = %19
store i32 3, i32* %3, align 4
br label %36
24: ; preds = %19
%25 = va_arg i8** %5, double
store double %25, double* %8, align 8
%26 = load double, double* %8, align 8
%27 = fcmp une double %26, 0x404E0147AE147AE1
br i1 %27, label %28, label %29
28: ; preds = %24
store i32 4, i32* %3, align 4
br label %36
29: ; preds = %24
%30 = va_arg i8** %5, i8*
store i8* %30, i8** %9, align 8
%31 = load i8*, i8** %9, align 8
%32 = call i32 @strcmp(i8* %31, i8* getelementptr inbounds ([5 x i8], [5 x i8]* @.str.1, i64 0, i64 0))
%33 = icmp ne i32 %32, 0
br i1 %33, label %34, label %35
34: ; preds = %29
store i32 5, i32* %3, align 4
br label %36
35: ; preds = %29
store i32 0, i32* %3, align 4
br label %36
36: ; preds = %35, %34, %28, %23, %18, %13
%37 = load i32, i32* %3, align 4
ret i32 %37
}
declare i32 @strcmp(i8*, i8*) #1
attributes #0 = { noinline nounwind optnone ssp uwtable "frame-pointer"="non-leaf" "min-legal-vector-width"="0" "no-trapping-math"="true" "probe-stack"="__chkstk_darwin" "stack-protector-buffer-size"="8" "target-cpu"="apple-m1" "target-features"="+aes,+crc,+crypto,+dotprod,+fp-armv8,+fp16fml,+fullfp16,+lse,+neon,+ras,+rcpc,+rdm,+sha2,+sha3,+sm4,+v8.5a,+zcm,+zcz" }
attributes #1 = { "frame-pointer"="non-leaf" "no-trapping-math"="true" "probe-stack"="__chkstk_darwin" "stack-protector-buffer-size"="8" "target-cpu"="apple-m1" "target-features"="+aes,+crc,+crypto,+dotprod,+fp-armv8,+fp16fml,+fullfp16,+lse,+neon,+ras,+rcpc,+rdm,+sha2,+sha3,+sm4,+v8.5a,+zcm,+zcz" }
!llvm.module.flags = !{!0, !1, !2, !3, !4, !5, !6, !7, !8}
!llvm.ident = !{!9}
!0 = !{i32 2, !"SDK Version", [2 x i32] [i32 12, i32 3]}
!1 = !{i32 1, !"wchar_size", i32 4}
!2 = !{i32 1, !"branch-target-enforcement", i32 0}
!3 = !{i32 1, !"sign-return-address", i32 0}
!4 = !{i32 1, !"sign-return-address-all", i32 0}
!5 = !{i32 1, !"sign-return-address-with-bkey", i32 0}
!6 = !{i32 7, !"PIC Level", i32 2}
!7 = !{i32 7, !"uwtable", i32 1}
!8 = !{i32 7, !"frame-pointer", i32 1}
!9 = !{!"Apple clang version 13.1.6 (clang-1316.0.21.2.5)"}
And for the latter:
; ModuleID = 'terra'
source_filename = "terra"
target datalayout = "e-m:o-i64:64-i128:128-n32:64-S128"
target triple = "arm64-apple-darwin21.6.0"
define dso_local i32 @f() {
entry:
ret i32 123
dead: ; No predecessors!
ret i32 undef
}
Looks like the data layouts are the same.
Here are the differences I see:
macosx12.0.0
vs darwin21.6.0
).The data layouts do indeed look the same.
I'll make a patch to try to force the OS to be the same. If that doesn't work then unfortunately I think this is going to require digging deeper into Clang and differences in how we initialize it from Terra. (Which I can't do without access to an M1 machine, and probably will be hard unless you know Terra and Clang well.)
Running it with self-compiled Clang/LLVM (built in Debug mode) gives a slight different error:
Assertion failed: (isa<X>(Val) && "cast<Ty>() argument of incompatible type!"), function cast, file Casting.h, line 269.
With the backtrace being:
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program assert
frame #0: 0x00000001a314ad98 libsystem_kernel.dylib`__pthread_kill + 8
frame #1: 0x00000001a317fee0 libsystem_pthread.dylib`pthread_kill + 288
frame #2: 0x00000001a30ba340 libsystem_c.dylib`abort + 168
frame #3: 0x00000001a30b9754 libsystem_c.dylib`__assert_rtn + 272
* frame #4: 0x0000000100015814 terra`llvm::cast_retty<llvm::StructType, llvm::Type*>::ret_type llvm::cast<llvm::StructType, llvm::Type>(Val=0x000000012d0c3400) at Casting.h:269:3
frame #5: 0x0000000107c6ecf4 terra`clang::CodeGen::CGBuilderTy::CreateStructGEP(this=0x000000016fdfb2a8, Addr=Address @ 0x000000016fdf8a30, Index=3, Name=0x000000016fdf9608) at CGBuilder.h:190:30
frame #6: 0x00000001084e7798 terra`(anonymous namespace)::AArch64ABIInfo::EmitAAPCSVAArg(this=0x00006000002073e0, VAListAddr=Address @ 0x000000016fdf97c0, Ty=QualType @ 0x000000016fdf97b8, CGF=0x000000016fdfb1a0) const at TargetInfo.cpp:5933:30
frame #7: 0x00000001084e5764 terra`(anonymous namespace)::AArch64ABIInfo::EmitVAArg(this=0x00006000002073e0, CGF=0x000000016fdfb1a0, VAListAddr=Address @ 0x000000016fdf98a0, Ty=QualType @ 0x000000016fdf9898) const at TargetInfo.cpp:5458:44
frame #8: 0x0000000107e3e384 terra`clang::CodeGen::CodeGenFunction::EmitVAArg(this=0x000000016fdfb1a0, VE=0x000000012c88e460, VAListAddr=0x000000016fdf9a78) at CGCall.cpp:5522:38
frame #9: 0x0000000107fad28c terra`(anonymous namespace)::ScalarExprEmitter::VisitVAArgExpr(this=0x000000016fdfa3f0, VE=0x000000012c88e460) at CGExprScalar.cpp:4666:24
frame #10: 0x0000000107fa4c28 terra`clang::StmtVisitorBase<std::__1::add_pointer, (anonymous namespace)::ScalarExprEmitter, llvm::Value*>::Visit(this=0x000000016fdfa3f0, S=0x000000012c88e460) at StmtNodes.inc:1435:1
frame #11: 0x0000000107f9bf4c terra`(anonymous namespace)::ScalarExprEmitter::Visit(this=0x000000016fdfa3f0, E=0x000000012c88e460) at CGExprScalar.cpp:412:52
frame #12: 0x0000000107fad8f0 terra`(anonymous namespace)::ScalarExprEmitter::EmitBinOps(this=0x000000016fdfa3f0, E=0x000000012c88e4c0) at CGExprScalar.cpp:2993:16
frame #13: 0x0000000107fadc88 terra`(anonymous namespace)::ScalarExprEmitter::EmitCompare(this=0x000000016fdfa3f0, E=0x000000012c88e4c0, UICmpOpc=ICMP_NE, SICmpOpc=ICMP_NE, FCmpOpc=FCMP_UNE, IsSignaling=false) at CGExprScalar.cpp:4026:24
frame #14: 0x0000000107fa543c terra`(anonymous namespace)::ScalarExprEmitter::VisitBinNE(this=0x000000016fdfa3f0, E=0x000000012c88e4c0) at CGExprScalar.cpp:833:3
frame #15: 0x0000000107fa37fc terra`clang::StmtVisitorBase<std::__1::add_pointer, (anonymous namespace)::ScalarExprEmitter, llvm::Value*>::Visit(this=0x000000016fdfa3f0, S=0x000000012c88e4c0) at StmtVisitor.h:64:26
frame #16: 0x0000000107f9bf4c terra`(anonymous namespace)::ScalarExprEmitter::Visit(this=0x000000016fdfa3f0, E=0x000000012c88e4c0) at CGExprScalar.cpp:412:52
frame #17: 0x0000000107f9bebc terra`clang::CodeGen::CodeGenFunction::EmitScalarExpr(this=0x000000016fdfb1a0, E=0x000000012c88e4c0, IgnoreResultAssign=false) at CGExprScalar.cpp:4811:8
frame #18: 0x0000000107f15cd0 terra`clang::CodeGen::CodeGenFunction::EvaluateExprAsBool(this=0x000000016fdfb1a0, E=0x000000012c88e4c0) at CGExpr.cpp:194:33
frame #19: 0x0000000108229b3c terra`clang::CodeGen::CodeGenFunction::EmitBranchOnBoolExpr(this=0x000000016fdfb1a0, Cond=0x000000012c88e4c0, TrueBlock=0x0000600001724180, FalseBlock=0x0000600001724200, TrueCount=0, LH=LH_None) at CodeGenFunction.cpp:1800:13
frame #20: 0x000000010815994c terra`clang::CodeGen::CodeGenFunction::EmitIfStmt(this=0x000000016fdfb1a0, S=0x000000012c88e510) at CGStmt.cpp:761:3
frame #21: 0x00000001081586fc terra`clang::CodeGen::CodeGenFunction::EmitStmt(this=0x000000016fdfb1a0, S=0x000000012c88e510, Attrs=ArrayRef<const clang::Attr *> @ 0x000000016fdfabe0) at CGStmt.cpp:147:32
frame #22: 0x00000001081625c4 terra`clang::CodeGen::CodeGenFunction::EmitCompoundStmtWithoutScope(this=0x000000016fdfb1a0, S=0x000000012c88e8f8, GetLast=false, AggSlot=AggValueSlot @ 0x000000016fdfada0) at CGStmt.cpp:496:7
frame #23: 0x00000001082279c8 terra`clang::CodeGen::CodeGenFunction::EmitFunctionBody(this=0x000000016fdfb1a0, Body=0x000000012c88e8f8) at CodeGenFunction.cpp:1199:5
frame #24: 0x000000010822834c terra`clang::CodeGen::CodeGenFunction::GenerateCode(this=0x000000016fdfb1a0, GD=GlobalDecl @ 0x000000016fdfaff8, Fn=0x0000600002c00208, FnInfo=0x0000600003300140) at CodeGenFunction.cpp:1373:5
frame #25: 0x00000001082520c4 terra`clang::CodeGen::CodeGenModule::EmitGlobalFunctionDefinition(this=0x000000012d0c1a00, GD=GlobalDecl @ 0x000000016fdfb190, GV=0x0000600002c00208) at CodeGenModule.cpp:4861:26
frame #26: 0x00000001082496f4 terra`clang::CodeGen::CodeGenModule::EmitGlobalDefinition(this=0x000000012d0c1a00, GD=GlobalDecl @ 0x000000016fdfc8a0, GV=0x0000000000000000) at CodeGenModule.cpp:3230:12
frame #27: 0x000000010824ddbc terra`clang::CodeGen::CodeGenModule::EmitGlobal(this=0x000000012d0c1a00, GD=GlobalDecl @ 0x000000016fdfcaa0) at CodeGenModule.cpp:2982:5
frame #28: 0x000000010825607c terra`clang::CodeGen::CodeGenModule::EmitTopLevelDecl(this=0x000000012d0c1a00, D=0x000000012c88e1e0) at CodeGenModule.cpp:5698:5
frame #29: 0x000000010843c64c terra`(anonymous namespace)::CodeGeneratorImpl::HandleTopLevelDecl(this=0x000000012d0c1200, DG=DeclGroupRef @ 0x000000016fdfe590) at ModuleBuilder.cpp:170:18
frame #30: 0x00000001000711e0 terra`CodeGenProxy::HandleTopLevelDecl(this=0x0000600003900680, D=DeclGroupRef @ 0x000000016fdfe5e8) at tcwrapper.cpp:592:20
frame #31: 0x00000001085238e8 terra`clang::ParseAST(S=0x000000012d0c4400, PrintStats=false, SkipFunctionBodies=false) at ParseAST.cpp:162:31
frame #32: 0x0000000100065d78 terra`dofile(T=0x000000012c305d28, TT=0x0000600002c08000, code="#include <stdio.h>\n#include <string.h>\n#include <stdarg.h>\n\nint checkargs(const char* f, va_list vl)\n{\n if(strcmp(f, \"%i%i%f%s\"))\n return 1;\n \n if(va_arg(vl, int) != -10)\n return 2;\n if(va_arg(vl, unsigned long long) != 60000000000001ULL)\n return 3;\n if(va_arg(vl, double) != 60.01)\n return 4;\n if(strcmp(va_arg(vl, const char*), \"TesT\"))\n return 5;\n return 0;\n}\n\n", args=size=13, result=0x000000016fdfeba8) at tcwrapper.cpp:1045:5
frame #33: 0x0000000100065904 terra`include_c(L=0x000000012c2fc380) at tcwrapper.cpp:1129:9
frame #34: 0x000000010a896d38 terra`lj_BC_FUNCC + 44
frame #35: 0x000000010a83e78c terra`lua_pcall(L=<unavailable>, nargs=<unavailable>, nresults=<unavailable>, errfunc=<unavailable>) at lj_api.c:1145:12 [opt]
frame #36: 0x000000010000216c terra`docall(L=0x000000012c2fc380, narg=0, clear=0) at main.cpp:339:14
frame #37: 0x0000000100001d14 terra`main(argc=2, argv=0x000000016fdfef48) at main.cpp:119:13
frame #38: 0x000000012c0fd08c dyld`start + 520
Here is a reproducer that can be used on an x86 Mac:
local target = terralib.newtarget {
Triple = 'arm64-apple-darwin21.6.0',
}
C = terralib.includecstring([[
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
int checkargs(const char* f, va_list vl)
{
if(strcmp(f, "%i%i%f%s"))
return 1;
if(va_arg(vl, int) != -10)
return 2;
if(va_arg(vl, unsigned long long) != 60000000000001ULL)
return 3;
if(va_arg(vl, double) != 60.01)
return 4;
if(strcmp(va_arg(vl, const char*), "TesT"))
return 5;
return 0;
}
]], nil, target)
Result with LLVM 14:
Assertion failed: (Ty && "Invalid GetElementPtrInst indices for type!"), function checkGEPType, file Instructions.h, line 923.
I think this is basically the same symptom as on M1, which hopefully means I can debug this the rest of the way on my side.
A backtrace with line numbers:
(Again in LLVM 14.)
I made some progress debugging this in Terra vs Clang. The key difference seems to be this branch:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.6/clang/lib/CodeGen/TargetInfo.cpp#L5513
isDarwinPCS()
returns true
in Clang and false
in Terra.
Kind
gets initialized at:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/CodeGen/TargetInfo.cpp#L11280
Seems to be getting the ABI from the Target
.
In Clang, this seems to come from the TargetOptions
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Basic/Targets.cpp#L702
This call does not appear to occur in Terra; probably the option is not set.
TargetOptions
comes from the Invocation
here:
I don't see CompilerInstance::setInvocation()
used in Clang, so I suppose the value is coming from the constructor.
The CompilerInvocation
seems to come from CompilerInvocation::CreateFromArgsImpl
:
Which calls ParseTargetArgs
:
Which goes into this heavily-macroized code:
clang/Driver/Options.inc
seems to be a generated file that lists every possible Clang option.
This may be a rabbit hole, I'm not sure it's useful to continue.
Running my vanilla Clang with -v
:
$ clang bug605.c -S -emit-llvm -o bug605.ll -target arm64-apple-darwin21.6.0 -v
clang version 14.0.0
Target: arm64-apple-darwin21.6.0
Thread model: posix
InstalledDir: /Users/elliott/Programming/Terra/llvm/llvm-14-src-reldeb/install/bin
(in-process)
"/Users/elliott/Programming/Terra/llvm/llvm-14-src-reldeb/install/bin/clang-14" -cc1 -triple arm64-apple-macosx12.0.0 -Wundef-prefix=TARGET_OS_ -Werror=undef-prefix -Wdeprecated-objc-isa-usage -Werror=deprecated-objc-isa-usage -emit-llvm -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name bug605.c -mrelocation-model pic -pic-level 2 -mframe-pointer=non-leaf -ffp-contract=on -fno-rounding-math -funwind-tables=2 -target-sdk-version=12.3 -fcompatibility-qualified-id-block-type-checking -fvisibility-inlines-hidden-static-local-var -target-cpu apple-m1 -target-feature +v8.5a -target-feature +fp-armv8 -target-feature +neon -target-feature +crc -target-feature +crypto -target-feature +dotprod -target-feature +fp16fml -target-feature +ras -target-feature +lse -target-feature +rdm -target-feature +rcpc -target-feature +zcm -target-feature +zcz -target-feature +fullfp16 -target-feature +sha2 -target-feature +aes -target-abi darwinpcs -fallow-half-arguments-and-returns -mllvm -treat-scalable-fixed-error-as-warning -debugger-tuning=lldb -target-linker-version 764 -v -fcoverage-compilation-dir=/Users/elliott/Programming/Terra/terra -resource-dir /Users/elliott/Programming/Terra/llvm/llvm-14-src-reldeb/install/lib/clang/14.0.0 -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -internal-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include -internal-isystem /Users/elliott/Programming/Terra/llvm/llvm-14-src-reldeb/install/lib/clang/14.0.0/include -internal-externc-isystem /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -fdebug-compilation-dir=/Users/elliott/Programming/Terra/terra -ferror-limit 19 -stack-protector 1 -fblocks -fencode-extended-block-signature -fregister-global-dtors-with-atexit -fgnuc-version=4.2.1 -fmax-type-align=16 -fcolor-diagnostics -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o bug605.ll -x c bug605.c
clang -cc1 version 14.0.0 based upon LLVM 14.0.0 default target x86_64-apple-darwin21.6.0
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/local/include"
ignoring nonexistent directory "/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/Library/Frameworks"
#include "..." search starts here:
#include <...> search starts here:
/Users/elliott/Programming/Terra/llvm/llvm-14-src-reldeb/install/lib/clang/14.0.0/include
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks (framework directory)
End of search list.
I noticed this includes the flag -target-abi darwinpcs
.
Adding that back into the Terra reproducer:
local target = terralib.newtarget {
Triple = 'arm64-apple-darwin21.6.0',
}
C = terralib.includecstring([[
#include <stdio.h>
#include <string.h>
#include <stdarg.h>
int checkargs(const char* f, va_list vl)
{
if(strcmp(f, "%i%i%f%s"))
return 1;
if(va_arg(vl, int) != -10)
return 2;
if(va_arg(vl, unsigned long long) != 60000000000001ULL)
return 3;
if(va_arg(vl, double) != 60.01)
return 4;
if(strcmp(va_arg(vl, const char*), "TesT"))
return 5;
return 0;
}
]], {"-target-abi", "darwinpcs"}, target)
And this passes!
$ ./build/bin/terra bug605.t
I'm not sure what process by which Clang chooses its default arguments, but it seems pretty clear here that Clang is selecting these differently from Terra, and a few of the default values are critical to correctly compiling on macOS M1.
Breaking at: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Frontend/CompilerInvocation.cpp#L4443
I have confirmed that CommandLineArgs
already contains all of the extra flags I saw above:
$ lldb -- .../llvm-14-src-reldeb/install/bin/clang bug605.c -S -emit-llvm -o bug605.ll -target arm64-apple-darwin21.6.0-src-reldeb/install/bin/clang bug605.c -S -emit-llvm -o bug
(lldb) target create ".../llvm-14-src-reldeb/install/bin/clang"
Current executable set to '.../llvm-14-src-reldeb/install/bin/clang' (x86_64).
(lldb) settings set -- target.run-args "bug605.c" "-S" "-emit-llvm" "-o" "bug605.ll" "-target" "arm64-apple-darwin21.6.0"
(lldb) b CompilerInvocation.cpp:4443
Breakpoint 1: where = clang`clang::CompilerInvocation::CreateFromArgsImpl(clang::CompilerInvocation&, llvm::ArrayRef<char const*>, clang::DiagnosticsEngine&, char const*) + 77 at CompilerInvocation.cpp:4443:28, address = 0x000000010288303d
(lldb) r
Process 45217 launched: '.../llvm-14-src-reldeb/install/bin/clang' (x86_64)
clang was compiled with optimization - stepping may behave oddly; variables may not be available.
Process 45217 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1
frame #0: 0x000000010288303d clang`clang::CompilerInvocation::CreateFromArgsImpl(Res=0x000000010b012c00, CommandLineArgs=ArrayRef<const char *> @ 0x0000600001202520, Diags=0x00007ff7bfefd660, Argv0=".../llvm-14-src-reldeb/install/bin/clang-14") at CompilerInvocation.cpp:4443:28 [opt]
4440 const OptTable &Opts = getDriverOptTable();
4441 const unsigned IncludedFlagsBitmask = options::CC1Option;
4442 unsigned MissingArgIndex, MissingArgCount;
-> 4443 InputArgList Args = Opts.ParseArgs(CommandLineArgs, MissingArgIndex,
4444 MissingArgCount, IncludedFlagsBitmask);
4445 LangOptions &LangOpts = *Res.getLangOpts();
4446
Target 0: (clang) stopped.
(lldb) p CommandLineArgs
(llvm::ArrayRef<const char *>) $0 = {
Data = 0x00007ff7bfefdda8
Length = 95
}
(lldb) p CommandLineArgs.Data
(const char *const *) $1 = 0x00007ff7bfefdda8
(lldb) p CommandLineArgs.Data[0]
(const char *const) $2 = 0x0000000106166dd8 "-cc1"
(lldb) p CommandLineArgs.Data[1]
(const char *const) $3 = 0x0000000106173634 "-triple"
(lldb) p CommandLineArgs.Data[2]
(const char *const) $4 = 0x000060000020dbc0 "arm64-apple-macosx12.0.0"
(lldb) p CommandLineArgs.Data[3]
(const char *const) $5 = 0x000000010616a033 "-Wundef-prefix=TARGET_OS_"
(lldb) p CommandLineArgs.Data[4]
(const char *const) $6 = 0x000000010616a04d "-Werror=undef-prefix"
(lldb) p CommandLineArgs.Data[5]
(const char *const) $7 = 0x000000010616a062 "-Wdeprecated-objc-isa-usage"
(lldb) p CommandLineArgs.Data[6]
(const char *const) $8 = 0x000000010616a07e "-Werror=deprecated-objc-isa-usage"
(lldb) p CommandLineArgs.Data[7]
(const char *const) $9 = 0x0000000106166e5c "-emit-llvm"
(lldb) p CommandLineArgs.Data[8]
(const char *const) $10 = 0x000000010617346e "-disable-free"
(lldb) p CommandLineArgs.Data[9]
(const char *const) $11 = 0x000000010617044b "-clear-ast-before-backend"
(lldb) p CommandLineArgs.Data[10]
(const char *const) $12 = 0x0000000106170023 "-disable-llvm-verifier"
(lldb) p CommandLineArgs.Data[11]
(const char *const) $13 = 0x0000000106166f32 "-discard-value-names"
(lldb) p CommandLineArgs.Data[12]
(const char *const) $14 = 0x0000000106170493 "-main-file-name"
(lldb) p CommandLineArgs.Data[13]
(const char *const) $15 = 0x0000600000c20d31 "bug605.c"
(lldb) p CommandLineArgs.Data[14]
(const char *const) $16 = 0x0000000106166fa4 "-mrelocation-model"
(lldb) p CommandLineArgs.Data[15]
(const char *const) $17 = 0x000000010616898c "pic"
(lldb) p CommandLineArgs.Data[16]
(const char *const) $18 = 0x00000001061711c2 "-pic-level"
(lldb) p CommandLineArgs.Data[17]
(const char *const) $19 = 0x0000000106166fc4 "2"
(lldb) p CommandLineArgs.Data[18]
...
(const char *const) $61 = 0x000000010617363c "-target-abi"
(lldb) p CommandLineArgs.Data[60]
(const char *const) $62 = 0x0000000106168132 "darwinpcs"
This in turn seems to come straight from cc1_main
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/tools/driver/cc1_main.cpp#L212
Which goes through driver.cpp
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/tools/driver/driver.cpp#L317
Some sort of job management infrastructure (not changing the args as far as I can tell):
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Job.cpp#L407
Through Compilation::ExecuteCommand
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Compilation.cpp#L196
(C.Arguments
here has size 95, so it seems the command has already been filled at this point.)
Compilation::ExecuteJobs
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Compilation.cpp#L249
Driver::ExecuteCompilation
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L1614
And finally back full circle to main
in driver.cpp
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/tools/driver/driver.cpp#L489
I note that Args
here has size 8 (maybe some arguments have been removed?).
The interesting part of this file seems to be the call to Driver::BuildCompilation
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L1063
ArgList
on entry:
(lldb) p ArgList
(llvm::ArrayRef<const char *>) $10 = {
Data = 0x00007ff7bfefed08
Length = 8
}
(lldb) p ArgList.Data[0]
(const char *const) $11 = 0x00007ff7bfeff998 ".../llvm-14-src-reldeb/install/bin/clang"
(lldb) p ArgList.Data[1]
(const char *const) $12 = 0x00007ff7bfeff9e3 "bug605.c"
(lldb) p ArgList.Data[2]
(const char *const) $13 = 0x00007ff7bfeff9ec "-S"
(lldb) p ArgList.Data[3]
(const char *const) $14 = 0x00007ff7bfeff9ef "-emit-llvm"
(lldb) p ArgList.Data[4]
(const char *const) $15 = 0x00007ff7bfeff9fa "-o"
(lldb) p ArgList.Data[5]
(const char *const) $16 = 0x00007ff7bfeff9fd "bug605.ll"
(lldb) p ArgList.Data[6]
(const char *const) $17 = 0x00007ff7bfeffa07 "-target"
(lldb) p ArgList.Data[7]
(const char *const) $18 = 0x00007ff7bfeffa0f "arm64-apple-darwin21.6.0"
I see that there is a TargetTriple
being set:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L1194
At the call to getToolChain
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L1251
I see that Triple
has this value:
(const llvm::Triple) $29 = (Data = "arm64-apple-darwin21.6.0", Arch = aarch64, SubArch = ARMSubArch_v8, Vendor = Apple, OS = Darwin, Environment = UnknownEnvironment, ObjectFormat = MachO)
We pass through the DarwinClang
toolchain:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L5522
After we come back we initialize the Compilation
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Compilation.cpp#L36
After we come back we call Driver::HandleImmediateArgs
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L1812
This is where -v
seems to get handled. (Running with -v
in a debugger though, I do not see the verbose arguments get printed here. Just some basic version information and installation/resource dirs.)
After we come back we call Driver::BuildInputs
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L2276
This seems just to classify each input argument (e.g., bug605.c
) and its associated language (if not otherwise specified).
After we return we call into Driver::BuildUniversalActions
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L2115
After we return we call into Driver::BuildJobs
:
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/Driver.cpp#L4232
... Skipping ahead, there are many layers of calls and they're mostly not interesting...
Darwin::ComputeEffectiveClangTriple
: This is where the OS seems to get set to something specific.
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Darwin.cpp#L986
Before this call, the Triple
is arm64-apple-darwin21.6.0
; afterwards it is arm64-apple-macosx12.0.0
.
Clang::ConstructJob
: This seems to be setting up a list of command line arguments (CmdArgs
).
https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Darwin.cpp#L986
Note: this is where the -cc1
command line argument comes from: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L4417
Note also, this is the modified triple that gets inserted here (i.e., macos12.0.0
): https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L4417
Warning options: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L4500
The output type: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L4578
The input filename: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L4810
PIC level: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L4874
(... Many, many options ...)
Target options: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L5281
SDKInfo
gets added: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Darwin.cpp#L2657 (turns into -target-sdk-version=12.3
)Target CPU: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L5314
RenderTargetOptions
: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Driver/ToolChains/Clang.cpp#L5320
-target-feature
flags)(And then it comes back and finishes with the rest of the job construction logic.)
So that explains how all that happens, but it's not obvious to me if any piece of this can be reused; it's coupled pretty tightly to Clang's argument parsing and job construction.
Thanks for the thorough bisecting - it's interesting to see the steps that Clang goes through, and a learning experience on my end. So, to clarify, the issue is that Terra is selecting the default (wrong) aapcs
ABI? RenderAArch64ABI
seems to add darwinpcs
for both xxx-xxx-darwin
and xxx-xxx-macosx
targets, which should check for either Darwin or MacOSX in the triple, but I presume Clang::AddAArch64TargetArgs
is part of Clang's frontend, which Terra doesn't use? Please do correct me if I'm wrong. Would it be appropriate to handle adding the -target-api darwinpcs
flag in terra_inittarget
, if macOS + AArch64 is detected at compile-time?
The boundary between Clang and LLVM is messy and often confusing.
In this case, the ABI seems to be a feature of Clang, not LLVM. So it's not stored or tracked in the LLVM Target
at all. It seems to be used by Clang to decide what code to generate; but afterwards everything flows through LLVM in a way that is completely agnostic to this.
But not everything is like this: the target OS (which Clang silently changes from darwin
to macosx
), and the various feature flags (that Clang adds by default) are part of the LLVM Target
and are passed to LLVM. So far none of these seem to be a major issue, but they could be. In theory, Clang and Terra could generate incompatible code if they don't both use the same configuration.
So I think there is a design question here about what level to solve this at. Right now Terra only uses Clang in a very limited capacity to parse C header files. Because Clang doesn't expose much of the internals here, we do go through the CompilerInvocation::CreateFromArgs
code path (see below). But this is not apparently sufficient to get the higher-level Clang default argument code path.
Perhaps there is a better place to start the initialization of Clang. I'll try to do a bit more monkeying around to see what I can find.
If that doesn't work out, then we could try to just add the minimal set of flags to fix this specific case, probably in tcwrapper.cpp
near where I linked above.
It looks like the constructor of clang::targets::AArch64TargetInfo
is failing to set the correct ABI, as it defaults to aapcs
and doesn't take into account darwinpcs
on AArch64 Darwin: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Basic/Targets/AArch64.cpp#L66 - not sure if this is by design, or an edge case specific to Darwin AArch64 when not using clang::driver::tools::Clang::ConstructJob
, and therefore not calling RenderAArch64ABI
. The clang::targets::ARMTargetInfo
constructor, for comparison, seems to handle the ABI manually: https://github.com/llvm/llvm-project/blob/llvmorg-14.0.0/clang/lib/Basic/Targets/ARM.cpp#L282-L326.
I just noticed that the entire infrastructure to call Driver::BuildCompilation
is already here and has been since 2019. Ouch.
I'll dig a bit more to figure out if we can extract what we need.
Re: the TargetInfo
subclasses: I don't think that route is going to work because this all falls under Clang's initialization via CompilerInvocation::CreateFromArgs
. Clang could change to make it work, and arguably it would be more consistent, but it doesn't work that way right now.
I was able to get a Driver::BuildCompilation
to spit out the -cc1
job that I need to get the -target-abi
flag. It's ugly, but it's what I've got right now. Maybe having built it, there is a way to use the constructed ToolChain
to get that argument. I'll need to do more digging.
Do you think it's worth filing an upstream bug report regarding clang::targets::AArch64TargetInfo
not initializing the ABI to darwinpcs
when on AArch64 Darwin?
After a few quick searches on the LLVM bug tracker, the only related issue I can find is https://github.com/llvm/llvm-project/issues/45989#issuecomment-981030476, which makes a passing reference to RenderAArch64ABI
and the toolchain.
I don't know how open the Clang project would be to considering this, but it seems worth trying. You might point out that (as best we can tell) LLDB uses the same CompilerInvocation::CreateFromArgs
code path to instantiate Clang and is probably susceptible to the same bug we are here. So it's not just an issue in one client.
Tracing up the call chain from RenderAArch64ABI
, where clang/lib/Driver/ToolChains/Clang.cpp
adds the -target-abi
flag for AArch64:
RenderAArch64ABI
: anonymous namespace:
Clang::AddAArch64TargetArgs
: private:
Clang::RenderTargetOptions
: private:
Clang::ConstructJob
: public:
It seems like there is no point in attempting to cut down to a level where we can access, because the first public
method already adds basically all of the options and will be more cumbersome to use than Driver::BuildCompilation
. RenderAArch64ABI
does not store the ABI value in any sort of useful way and just sticks it in the output command line arguments.
So I guess we really do have to parse these command line arguments, there is no other way that is exposed by Clang right now.
It's not pretty, but you can try https://github.com/elliottslaughter/terra/tree/fix-macos-m1-target-abi and let me know if it works for you to fix this bug.
Looks like that fixes the test for vararg.t
(tested against LLVM 13 built from source) - nice!
For what it's worth, the following tests still fail on that branch:
The following tests FAILED:
27 - atomicrmw.t (Subprocess aborted)
53 - bug603.t (Subprocess aborted)
55 - bug604.t (Failed)
66 - cconv_array.t (Failed)
I submitted an upstream issue at https://github.com/llvm/llvm-project/issues/57529. We'll see what the Clang developers have to say about this.
Closing as fixed for now.
The
vararg.t
test fails on macOS arm64 (M1). macOS 12.5, using system clang and Homebrew LLVM 13. Trimmed output: