search

llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
28.97k stars 11.94k forks source link

Apple clang generates incorrect x86_64 executable when .zerofill is used to create a large segment #54638

Open JayReynoldsFreeman opened 2 years ago

JayReynoldsFreeman commented 2 years ago

This is a duplicate of a bug I have filed with Apple developer feedback: It concerns the version of clang that Apple provides with Xcode 13.2.1. When I input "clang -- version", I get

Apple clang version 13.0.0 (clang-1300.0.29.30) Target: x86_64-apple-darwin20.6.0 Thread model: posix

The essence of the problem is that when targeting the x86_64 architecture, using ".zerofill" in a .s file to specify a segment, clang will not properly create a segment larger than (approximately) 1 Gbye, and provides no warning that it has failed to do so. I am using .zerofill directives of the form:

.zerofill FooSegment, FooSector, FooVariable, 0x40000000

and am using objdump -h to investigate sector layout in the compiled executable. Specifically, clang creates a section of length zero instead of the length specified in the .zerofill directive.

The problem does not occur if I target the arm64 architecture.

I attach two short files which can be compiled to demonstrate the problem. See detailed description of what to do with the files, and what results I have obtained, in file "BugSegment.c++".

Files: Here is "BugSegment.s":

.zerofill FooSegment, FooSector, FooVariable, 0x40000000 //.zerofill FooSegment, FooSector, FooVariable, 0x80000000

And here is "BugSegment.c++":

` /****

                      INSTRUCTIONS

The code is way at the bottom -- literally a one-liner. The purpose of this test is to demonstrate an anomaly in binary file organization when using the ".zerofill" directive to create a large segment. The idea is to use "objdump.h" to inspect the executable file. File "BugSegment.s" contains two .zerofill directives, one specifying a segment size of 0x40000000 and one specifying 0x80000000. The idea is to comment out one or the other directive, to see what happens. The compiler invocation uses the -segaddr linker flag to position the segment at a particular address.

The problem occurs only when I am compiling with a target architecture of Intel Silicon. When I target Apple silicon, the binary looks fine.

Let's be clear: There are four cases of interest:

Target architecture Architecture of development Does it work? machine (where I am compiling) =================== ================================ ============= Intel processor Intel machine (MacPro7,1) NO Intel processor Apple machine (Macmini9,1) NO Apple silicon Intel machine (MacPro7,1) YES Apple silicon Apple machine (Macmini9,1) YES

E.g.:

On an Intel Mac (MacPro7,1), running Big Sur 11.6.5 and with Xcode 13.2.1 (and its compilers) installed, and using .zerofill with 0x40000000, I compiled with

clang BugSegment.c++ BugSegment.s -o BugSegment -segaddr FooSegment 0x10000000000

Then I changed "BugSegment.s" so that the size was 0x80000000 instead of 0x40000000, and tried again.

In the first case (0x40000000), objdump produced a reasonable result. The sector showed up at the expected address with the expected size:

SNIP

objdump -h BugSegment

BugSegment: file format mach-o 64-bit x86-64

Sections: Idx Name Size VMA Type 0 text 00000025 0000000100003f30 TEXT 1 stubs 00000006 0000000100003f56 TEXT 2 stub_helper 0000001a 0000000100003f5c TEXT 3 cstring 00000042 0000000100003f76 DATA 4 unwind_info 00000048 0000000100003fb8 DATA 5 got 00000008 0000000100004000 DATA 6 la_symbol_ptr 00000008 0000000100008000 DATA 7 data 00000008 0000000100008008 DATA 8 FooSector 40000000 0000010000000000 BSS

UNSNIP

BUT, on changing to a size of 0x80000000, things got weird -- a "__huge" sector was created with the expected size, but at an unexpected address, and the sector named in ".zerofill" was at the right address but had size zero:

SNIP

BugSegment: file format mach-o 64-bit x86-64

Sections: Idx Name Size VMA Type 0 text 00000025 0000000100003f30 TEXT 1 stubs 00000006 0000000100003f56 TEXT 2 stub_helper 0000001a 0000000100003f5c TEXT 3 cstring 00000042 0000000100003f76 DATA 4 unwind_info 00000048 0000000100003fb8 DATA 5 got 00000008 0000000100004000 DATA 6 la_symbol_ptr 00000008 0000000100008000 DATA 7 data 00000008 0000000100008008 DATA 8 __huge 80000000 0000000100008010 BSS <==ANOMALY 9 FooSector 00000000 0000010000000000 BSS <==ANOMALY

UNSNIP

Next I went back to the MacPro7,1, and compiled using the "-target" flag to specify Apple Silicon.

clang BugSegment.c++ BugSegment.s -o BugSegment -segaddr FooSegment 0x10000000000 -target arm64-apple-macos11.1

The executable looked fine when I examined it with objdump -h.

Et cetera ...

Thus whatever this problem is, it has to do with file-format generation for Intel silicon. (And my project, from which this simple test case was distilled, cannot work the correctly on both architectures.)

****/

include

int main( int argc, char *argv[] ) { printf("Use objdump -h on the executable to see the segments, et cetera.\n"); } `

DimitryAndric commented 2 years ago

Can you reproduce this with stock clang, and if so, which version? If it is an Apple clang specific bug, it won't be handled here.

llvmbot commented 2 years ago

@llvm/issue-subscribers-clang-codegen

llvmbot commented 2 years ago

@llvm/issue-subscribers-backend-x86

JayReynoldsFreeman commented 2 years ago

I am having trouble reproducing it, but I rarely develop on anything other than a Macintosh, so the problem may be with my own ignorance. Let me tell you what I tried:

On an Intel Linux system (recent Mint Linux, Dell laptop), I installed clang (version 10.0.0-4ubuntu1, target x86_64-pc-linux-gnu, thread model posix), but when I try to compile my test case I get:

SNIP (Linux box results)

clang BugSegment.c++ BugSegment.s -o BugSegment -segaddr FooSegment 0x10000000000 clang: warning: argument unused during compilation: '-segaddr FooSegment 0x10000000000' [-Wunused-command-line-argument] BugSegment.s:1.1: error: unknown directive .zerofill FooSegment, FooSector, FooVariable, 0x40000000

UNSNIP

That seemed very weird. My guess is that I do not have the appropriate linker installed, but I don't know what else to try. "ld -v" gets "GNU ld (GNU Binutils for Ubuntu) 2.34". When I try to find what loader I am using on my Apple development system, I find it is using an Apple linker:

SNIP (Macintosh results)

ld -v @(#)PROGRAM:ld PROJECT:ld64-711 BUILD 21:57:11 Nov 17 2021 configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em LTO support using: LLVM version 13.0.0, (clang-1300.0.29.30) (static support for 27, runtime is 27) TAPI support using: Apple TAPI version 13.0.0 (tapi-1300.0.6.5)

UNSNIP

If you can recommend anything I can do to explore further, let me know. Alternatively, if you are convinced that my problem is an Apple bug and not one of yours, let me know and I will hollar at Apple.

-- Jay Reynolds Freeman

@.*** http://JayReynoldsFreeman.com (personal web site)

On Mar 30, 2022, at 03:14, Dimitry Andric @.***> wrote:

Can you reproduce this with stock clang, and if so, which version? If it is an Apple clang specific bug, it won't be handled here.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

efriedma-quic commented 2 years ago

.zerofill only exists on Darwin (macOS/iOS) targets.

You can download the latest llvm.org release for Mac https://github.com/llvm/llvm-project/releases/download/llvmorg-14.0.0/clang+llvm-14.0.0-x86_64-apple-darwin.tar.xz . (This bug tracker tracks issues on Mac, just not issues that specifically involve the proprietary version of clang Apple ships as part of Xcode.)

mrpippy commented 6 months ago

What you're hitting here is the "huge" pass in Apple's (not the LLVM) linker, ld64. See huge.cpp.

I previously found a mailing list post sort-of explaining its purpose, unfortunately I can't find it again. IIRC: on x86_64 macOS only supports the "small" code model, this allows RIP-relative addressing to always be used but means that all code/data must be accessible using a 32-bit offset from any instruction.

The "huge" pass looks for any sections that are > 2GB from the base address, and if any are, it combines all zero-fill sections to a __huge section. I guess this would help try to fit things into the 2GB limit, but obviously isn't compatible with our use of zerofill sections.

I work on Wine, we have a similar need to reserve large parts of the address space for loading Windows binaries. We previously used a statically-linked "preloader" but this is fragile, relies on deprecated APIs, and is increasingly not effective with modern macOS (where dyld, Rosetta, etc. all allocate memory before the preloader ever executes). A zerofill section covering the low 8GB of address space solves all these problems, but I was also hitting the __huge bug.

Good news! In Xcode 15.3 Apple added a -no_huge linker flag, which disables the huge pass. For me, it's working great. Depending on how you're accessing the section you may get code model-related linker errors, but I think this can be avoided by storing the address first into a pointer variable and then accessing that way (so it's not trying to do RIP-relative accesses).