Doesn't compile on OSX Lion (10.7)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Get the sources
2. Run the specified command (using Makefile.old)

What is the expected output? What do you see instead?

A series of tests and a successful build.

instead I get : 

----
/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+A
sserts/bin/clang++  -faddress-sanitizer -mllvm 
-asan-blacklist=/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/tests/asan_
test.ignore -mllvm -asan-stack=1 -mllvm -asan-globals=1 -mllvm 
-asan-mapping-scale=0   -mllvm -asan-mapping-offset-log=-1   -mllvm 
-asan-use-after-return=0 -DASAN_UAR=0 -DASAN_HAS_EXCEPTIONS=1 
-DASAN_NEEDS_SEGV=1 -DASAN_HAS_BLACKLIST=1 -I.. -I../include -Wall 
-fvisibility=hidden -m64 -g -w sample1.o sample1_unittest.o gtest_main.a -o 
sample1_unittest
clang: warning: argument unused during compilation: '-mllvm 
-asan-blacklist=/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/tests/asan_
test.ignore'
clang: warning: argument unused during compilation: '-mllvm -asan-stack=1'
clang: warning: argument unused during compilation: '-mllvm -asan-globals=1'
clang: warning: argument unused during compilation: '-mllvm 
-asan-mapping-scale=0'
clang: warning: argument unused during compilation: '-mllvm 
-asan-mapping-offset-log=-1'
clang: warning: argument unused during compilation: '-mllvm 
-asan-use-after-return=0'
/private/tmp/wwwi/llvm/projects/compiler-rt/lib/asan/../../../../build/Release+A
sserts/bin/clang++  -faddress-sanitizer  -Wall -fvisibility=hidden -m64 -g -O3 
bin_darwin/asan_test64.o bin_darwin/asan_globals_test64.o 
bin_darwin/asan_break_optimization64.o bin_darwin/asan_noinst_test64.o 
bin_darwin/asan_interface_test64.o bin_darwin/asan_mac_test64.o \
        -framework Foundation -o bin_darwin/asan_test64  third_party/googletest/make-darwin64/gtest-all.o 
Assertion failed: (0), function targetContent, file 
/SourceCache/ld64/ld64-127.2/src/ld/parsers/macho_relocatable_file.cpp, line 
4898.
0  0x10920c041  __assert_rtn + 81
1  0x10924565e  
mach_o::relocatable::CFStringSection<x86_64>::targetContent(mach_o::relocatable:
:Atom<x86_64> const*, ld::IndirectBindingTable const&, 
mach_o::relocatable::CFStringSection<x86_64>::ContentType*, unsigned int*) + 398
2  0x10924529e  
mach_o::relocatable::CFStringSection<x86_64>::contentHash(mach_o::relocatable::A
tom<x86_64> const*, ld::IndirectBindingTable const&) const + 30
3  0x10924304a  
mach_o::relocatable::Atom<x86_64>::contentHash(ld::IndirectBindingTable const&) 
const + 42
4  0x109273d97  ld::tool::SymbolTable::findSlotForReferences(ld::Atom const*, 
ld::Atom const**) + 439
5  0x109273b3f  ld::tool::SymbolTable::addByReferences(ld::Atom const&) + 31
6  0x109273f3a  ld::tool::SymbolTable::add(ld::Atom const&, bool) + 58
7  0x10927771a  ld::tool::Resolver::doAtom(ld::Atom const&) + 698
8  0x10924714c  
mach_o::relocatable::File<x86_64>::forEachAtom(ld::File::AtomHandler&) const + 
60
9  0x10927022c  
ld::tool::InputFiles::forEachInitialAtom(ld::File::AtomHandler&) const + 44
10  0x109279eda  ld::tool::Resolver::resolve() + 202
11  0x10920c345  main + 549
12  0x1091fb9a4  start + 52
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[1]: *** [bin_darwin/asan_test64] Error 1
make: *** [b64] Error 2
-----

What version of the product are you using? On what operating system?

$ xcodebuild -version
4.2.1
Build version 4D502

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.7.2
BuildVersion:   11C74

Please provide any additional information below.

Svn revision : 149195

Original issue reported on code.google.com by mkvtooln...@gmail.com on 29 Jan 2012 at 4:22

[GoogleCodeExporter]

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 6:48

• Changed state: Accepted

[GoogleCodeExporter]

This is reproducible for me. I consider this a bug in ld, but it is triggered 
only with -faddress-sanitizer (I'm reducing the testcase now), so we can 
probably make some workaround. 

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 8:10

[GoogleCodeExporter]

$ cat a_t.ii
typedef const struct __CFString * CFStringRef;
void TestBody() {
  CFStringRef str = ((CFStringRef) __builtin___CFStringMakeConstantString ("" "Hello world!\n" ""));
}

$ 
/Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan/../../../../build/Rele
ase+Asserts/bin/clang++  -faddress-sanitizer  a_t.ii -o a_t.o 2>&1 -v
clang version 3.1 (trunk 149199)
Target: x86_64-apple-darwin11.2.0
Thread model: posix
 "/Users/glider/src/asan/llvm/build/Release+Asserts/bin/clang" -cc1 -triple x86_64-apple-macosx10.7.2 -emit-obj -mrelax-all -disable-free -main-file-name a_t.ii -pic-level 1 -mdisable-fp-elim -masm-verbose -munwind-tables -target-cpu core2 -target-linker-version 127.2 -v -resource-dir /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1 -fdeprecated-macro -fdebug-compilation-dir /Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan -ferror-limit 19 -fmessage-length 145 -faddress-sanitizer -stack-protector 1 -mstackrealign -fblocks -fobjc-runtime-has-arc -fobjc-runtime-has-weak -fobjc-dispatch-method=mixed -fcxx-exceptions -fexceptions -fdiagnostics-show-option -fcolor-diagnostics -o /var/folders/00/0k0r8000h01000cxqpysvccm002831/T/a_t-hJP3Ut.o -x c++-cpp-output a_t.ii
clang -cc1 version 3.1 based upon LLVM 3.1svn default target 
x86_64-apple-darwin11.2.0
ignoring nonexistent directory 
"/usr/include/c++/4.2.1/i686-apple-darwin10/x86_64"
ignoring nonexistent directory "/usr/include/c++/4.0.0"
ignoring nonexistent directory "/usr/include/c++/4.0.0/i686-apple-darwin8/"
ignoring nonexistent directory "/usr/include/c++/4.0.0/backward"
ignoring nonexistent directory "/usr/local/include"
#include "..." search starts here:
#include <...> search starts here:
 /usr/include/c++/4.2.1
 /usr/include/c++/4.2.1/backward
 /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1/include
 /usr/include
 /System/Library/Frameworks (framework directory)
 /Library/Frameworks (framework directory)
End of search list.
 "/usr/llvm-gcc-4.2/libexec/gcc/i686-apple-darwin11/4.2.1/ld" -demangle -dynamic -arch x86_64 -macosx_version_min 10.7.2 -o a_t.o -lcrt1.10.6.o /var/folders/00/0k0r8000h01000cxqpysvccm002831/T/a_t-hJP3Ut.o -lstdc++ /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1/lib/darwin/libclang_rt.asan_osx.a -lstdc++ -framework CoreFoundation -lSystem /Users/glider/src/asan/llvm/build/Release+Asserts/bin/../lib/clang/3.1/lib/darwin/libclang_rt.osx.a
Assertion failed: (0), function targetContent, file 
/SourceCache/ld64/ld64-127.2/src/ld/parsers/macho_relocatable_file.cpp, line 
4898.
0  0x10fe20041  __assert_rtn + 81
1  0x10fe5965e  
mach_o::relocatable::CFStringSection<x86_64>::targetContent(mach_o::relocatable:
:Atom<x86_64> const*, ld::IndirectBindingTable const&, 
mach_o::relocatable::CFStringSection<x86_64>::ContentType*, unsigned int*) + 398
2  0x10fe5929e  
mach_o::relocatable::CFStringSection<x86_64>::contentHash(mach_o::relocatable::A
tom<x86_64> const*, ld::IndirectBindingTable const&) const + 30
3  0x10fe5704a  
mach_o::relocatable::Atom<x86_64>::contentHash(ld::IndirectBindingTable const&) 
const + 42
4  0x10fe87d97  ld::tool::SymbolTable::findSlotForReferences(ld::Atom const*, 
ld::Atom const**) + 439
5  0x10fe87b3f  ld::tool::SymbolTable::addByReferences(ld::Atom const&) + 31
6  0x10fe87f3a  ld::tool::SymbolTable::add(ld::Atom const&, bool) + 58
7  0x10fe8b71a  ld::tool::Resolver::doAtom(ld::Atom const&) + 698
8  0x10fe5b14c  
mach_o::relocatable::File<x86_64>::forEachAtom(ld::File::AtomHandler&) const + 
60
9  0x10fe8422c  
ld::tool::InputFiles::forEachInitialAtom(ld::File::AtomHandler&) const + 44
10  0x10fe8deda  ld::tool::Resolver::resolve() + 202
11  0x10fe20345  main + 549
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 9:08

[GoogleCodeExporter]

$ 
/Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan/../../../../build/Rele
ase+Asserts/bin/clang++ a_t.ii -S -emit-llvm -o a_t.ll
$ 
/Users/glider/src/asan/llvm/projects/compiler-rt/lib/asan/../../../../build/Rele
ase+Asserts/bin/clang++ a_t.ii -S -emit-llvm -o a_t_asan.ll -faddress-sanitizer
=============================
$ cat a_t.ll; ModuleID = 'a_t.ii'
target datalayout = 
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v6
4:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.7.2"

%struct.NSConstantString = type { i32*, i32, i8*, i64 }
%struct.__CFString = type opaque

@__CFConstantStringClassReference = external global [0 x i32]
@.str = linker_private unnamed_addr constant [14 x i8] c"Hello world!\0A\00", 
align 1
@_unnamed_cfstring_ = private constant %struct.NSConstantString { i32* 
getelementptr inbounds ([0 x i32]* @__CFConstantStringClassReference, i32 0, 
i32 0), i32 1992, i8* getelementptr inbounds ([14 x i8]* @.str, i32 0, i32 0), 
i64 13 }, section "__DATA,__cfstring"

define void @_Z8TestBodyv() nounwind uwtable ssp {
entry:
  %str = alloca %struct.__CFString*, align 8
  store %struct.__CFString* bitcast (%struct.NSConstantString* @_unnamed_cfstring_ to %struct.__CFString*), %struct.__CFString** %str, align 8
  ret void
}
=============================
$ cat a_t_asan.ll
; ModuleID = 'a_t.ii'
target datalayout = 
"e-p:64:64:64-i1:8:8-i8:8:8-i16:16:16-i32:32:32-i64:64:64-f32:32:32-f64:64:64-v6
4:64:64-v128:128:128-a0:0:64-s0:64:64-f80:128:128-n8:16:32:64-S128"
target triple = "x86_64-apple-macosx10.7.2"

%struct.NSConstantString = type { i32*, i32, i8*, i64 }
%struct.__CFString = type opaque

@__CFConstantStringClassReference = external global [0 x i32]
@.str = linker_private unnamed_addr constant [14 x i8] c"Hello world!\0A\00", 
align 1
@_unnamed_cfstring_ = private constant { %struct.NSConstantString, [64 x i8] } 
{ %struct.NSConstantString { i32* getelementptr inbounds ([0 x i32]* 
@__CFConstantStringClassReference, i32 0, i32 0), i32 1992, i8* getelementptr 
inbounds ([14 x i8]* @.str, i32 0, i32 0), i64 13 }, [64 x i8] zeroinitializer 
}, section "__DATA,__cfstring", align 32
@0 = private constant [28 x i8] c"_unnamed_cfstring_ (a_t.ii)\00"
@1 = private global [1 x { i64, i64, i64, i64 }] [{ i64, i64, i64, i64 } { i64 
ptrtoint ({ %struct.NSConstantString, [64 x i8] }* @_unnamed_cfstring_ to i64), 
i64 32, i64 96, i64 ptrtoint ([28 x i8]* @0 to i64) }]
@llvm.global_dtors = appending global [1 x { i32, void ()* }] [{ i32, void ()* 
} { i32 1, void ()* @asan.module_dtor }]
@__asan_mapping_offset = linkonce_odr constant i64 17592186044416
@__asan_mapping_scale = linkonce_odr constant i64 3
@2 = private constant [27 x i8] c"_Z8TestBodyv 1 32 8 3 str \00"
@llvm.global_ctors = appending global [1 x { i32, void ()* }] [{ i32, void ()* 
} { i32 1, void ()* @asan.module_ctor }]

define void @_Z8TestBodyv() nounwind uwtable ssp address_safety {
entry:
  %MyAlloca = alloca [96 x i8], align 32
  %0 = ptrtoint [96 x i8]* %MyAlloca to i64
  %1 = add i64 %0, 32
  %2 = inttoptr i64 %1 to %struct.__CFString**
  %3 = inttoptr i64 %0 to i64*
  store i64 1102416563, i64* %3
  %4 = add i64 %0, 8
  %5 = inttoptr i64 %4 to i64*
  store i64 ptrtoint ([27 x i8]* @2 to i64), i64* %5
  %6 = lshr i64 %0, 3
  %7 = or i64 %6, 17592186044416
  %8 = inttoptr i64 %7 to i32*
  store i32 -235802127, i32* %8
  %9 = add i64 %7, 4
  %10 = inttoptr i64 %9 to i32*
  store i32 -185273344, i32* %10
  %11 = add i64 %7, 8
  %12 = inttoptr i64 %11 to i32*
  store i32 -202116109, i32* %12
  %str = alloca %struct.__CFString*, align 8
  %13 = ptrtoint %struct.__CFString** %2 to i64
  %14 = lshr i64 %13, 3
  %15 = or i64 %14, 17592186044416
  %16 = inttoptr i64 %15 to i8*
  %17 = load i8* %16
  %18 = icmp ne i8 %17, 0
  br i1 %18, label %25, label %19

; <label>:19                                      ; preds = %entry
  store %struct.__CFString* bitcast ({ %struct.NSConstantString, [64 x i8] }* @_unnamed_cfstring_ to %struct.__CFString*), %struct.__CFString** %2, align 8
  store i64 1172321806, i64* %3
  %20 = inttoptr i64 %7 to i32*
  store i32 0, i32* %20
  %21 = add i64 %7, 4
  %22 = inttoptr i64 %21 to i32*
  store i32 0, i32* %22
  %23 = add i64 %7, 8
  %24 = inttoptr i64 %23 to i32*
  store i32 0, i32* %24
  ret void

; <label>:25                                      ; preds = %entry
  call void @__asan_report_store8(i64 %13) noreturn
  unreachable
}

define internal void @asan.module_ctor() {
  call void @__asan_init()
  call void @__asan_register_globals(i64 ptrtoint ([1 x { i64, i64, i64, i64 }]* @1 to i64), i64 1)
  %1 = load volatile i64* @__asan_mapping_scale
  %2 = load volatile i64* @__asan_mapping_offset
  ret void
}

declare void @__asan_init()

declare void @__asan_register_globals(i64, i64)

define internal void @asan.module_dtor() {
  call void @__asan_unregister_globals(i64 ptrtoint ([1 x { i64, i64, i64, i64 }]* @1 to i64), i64 1)
  ret void
}

declare void @__asan_unregister_globals(i64, i64)

declare void @__asan_report_store8(i64)

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 9:35

[GoogleCodeExporter]

Here's the layout of an instrumented NSString global that causes ld to crash:

========================
$ cat a_t.S
    .section    __TEXT,__text,regular,pure_instructions
    .section    __TEXT,__cstring,cstring_literals
l_.str:                                 ## @.str
    .asciz   "Hello world!\n"

    .section    __DATA,__cfstring
    .align  4                       ## @_unnamed_cfstring_
L__unnamed_cfstring_:
    .quad   ___CFConstantStringClassReference
    .long   1992                    ## 0x7c8
    .space  4
    .quad   l_.str
    .quad   13                      ## 0xd
    .space  64

.subsections_via_symbols
========================

(See http://jens.ayton.se/blag/objc-constant-objects/ for the description of 
NSString fields)

Removing the redzone (the ".space 64" line) fixes the problem.

Assuming that nobody writes the code that accesses the string in 
"__DATA,__cfstring" manually, we actually do not want to add redzones to such 
objects.

We may probably want to add a redzone at the end of l_.str string instead -- 
this will require ASan to instrument linker_private strings.
Kostya, WDYT? I understand that linker_private symbols are removed from the 
executables, but we may still want to instrument them in order to prevent 
incorrect accesses.

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 10:13

[GoogleCodeExporter]

r149243 should fix the problem.
However, the tests crash on Lion due to some memcpy/memmove interception 
problem. I'm going to take a look.

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 10:45

[GoogleCodeExporter]

See issue 34 for the memcpy discussion.

Original comment by ramosian.glider@gmail.com on 30 Jan 2012 at 1:03

[GoogleCodeExporter]

As per discussion with Kostya, we'll not instrument the linker_private symbols 
for now, but we may want to get back to it someday.
Closing the bug.

Original comment by ramosian.glider@gmail.com on 1 Feb 2012 at 9:32

• Changed state: Fixed

[GoogleCodeExporter]

Original comment by ramosian.glider@gmail.com on 13 Sep 2012 at 1:40

• Added labels: OpSys-OSX