tinysun212 / swift-windows

Swift compiler for Cygwin, MSVC, MinGW. Full development environment can be downloaded from the Swift for Windows.
http://SwiftForWindows.github.io
Apache License 2.0
331 stars 30 forks source link

Question about the linking process of the swift compiler #2

Closed mominul closed 7 years ago

mominul commented 8 years ago

Hi! I know that it isn't the right place to ask, but because you have recently work on autolink-extract driver. I have some curiosity about how swift compiler links compiled source code. As far as I figured When swiftc compiles files then it initiates swift-autolink-extract, after building *.autollink file then it initiates clang to communicate with linker.

But what does the swift-autolink-extract do and why swiftc initiates clang to pass arguments(?) to linker?

Thanks for helping!

tinysun212 commented 8 years ago

Welcome to my git page.

For convenience, I'll suppose you are in Cygwin environment.

The swift compiler generates the object file (*.o) from the source file (*.swift). To build an executable, the system linker combines the generated object file, swift libraries (cygswiftCore.dll, cygswiftGlibc.dll, cygswiftSwiftOnoneSupport.dll, libFoundation.dll, etc) and other system libraries.

For example, if you compile this one line file with none optimization, the linker combines Hello.o, cygswiftCore.dll, cygswiftSwiftOnoneSupport.dll.

$ cat Hello.swift
print("Hello")

$ swiftc Hello.swift -Onone

, and if you compile this Foundation sample file, the linker combines TestFoundation.o, cygswiftCore.dll, libFoundation.dll, cygswiftGlibc.dll, etc.

$ cat TestFoundation.swift
import Foundation

// Make an URLComponents instance
let swifty = NSURLComponents(string: "https://swift.org")!

// Print something useful about the URL
print("\(swifty.host!)")

$ swiftc TestFoundation.swift -O

The list of libraries to combine varies as the modules you imported in the source and varies as the compiler optimization option you used.

swift-autolink-extract extracts the list of libraries to combine from the generated object files, and pass the list to the linker through clang. For this, the section name .swift1_autolink_entries for object file is promised between the swift compiler and swift-autolink-extract. The compiler saves the list of the libraries to the section, and swift-autolink-extract reads the section.

swift-autolink-extract function can be tested simply.

$ swiftc -c Hello.swift -Onone

$ swift-autolink-extract Hello.o
-lswiftCore
-lswiftSwiftOnoneSupport

$ swiftc -c TestFoundation.swift -O

$ swift-autolink-extract TestFoundation.o
-lswiftCore
-lFoundation
-lswiftGlibc
-lpthread
-lutil
-ldl

You can use the utility objdump to view the internal of the object files generated by the compiler.

$ objdump -s -j .swift1_autolink_entries Hello.o
$ objdump -s -j .swift1_autolink_entries TestFoundation.o

If you have any questions feel free to ask me.

mominul commented 8 years ago

Thank you @tinysun212 ! I am very sorry that I forgot to tell my platform. Actually it is Linux. But thanks to your well description now I have a good knowledge about swift-autolink-extract. Now, Why swiftc initiates clang to pass arguments or communicate with linker?

Please forgive me for asking question over and over!

tinysun212 commented 8 years ago

Cygwin port copied the logic for Linux. But I don't know why Linux port initiates clang.

In my guess, clang was used for linking because it is simpler than ld (or gold) by delegating the detail information.

If swiftc uses the ld directly, swiftc should know the <pre-objects> and <post-objects> in addition to <swift objects & libraries>, so that all objects to link are as follows.

crt1.o crti.o crtbegin.o <swift objects & libraries> -lstdc++ -lm -lgcc_s -lgcc -lc crtend.o crtn.o <pre-objects> = crt1.o crti.o crtbegin.o <post-objects> = -lstdc++ -lm -lgcc_s -lgcc -lc crtend.o crtn.o

Writing to the swift-dev mailing list will be better to get the answer of this question.

mominul commented 8 years ago

For your reference:

$ cat Hello.swift
print("Hello")

$ swiftc -v Hello.swift -o hello
Swift version 3.0-dev (LLVM 440a472499, Clang e10506ae1c, Swift 395e967875)
Target: x86_64-unknown-linux-gnu
/home/mominul/swift/usr/bin/swift -frontend -c -primary-file Hello.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -color-diagnostics -module-name hello -o /tmp/Hello-1cd533.o
/home/mominul/swift/usr/bin/swift-autolink-extract /tmp/Hello-1cd533.o -o /tmp/Hello-3da12a.autolink
/usr/bin/clang++ -fuse-ld=gold -target x86_64-unknown-linux-gnu -Xlinker -rpath -Xlinker /home/mominul/swift/usr/lib/swift/linux /home/mominul/swift/usr/lib/swift/linux/x86_64/swift_begin.o /tmp/Hello-1cd533.o -L /home/mominul/swift/usr/lib/swift/linux --target=x86_64-unknown-linux-gnu -lswiftCore @/tmp/Hello-3da12a.autolink /home/mominul/swift/usr/lib/swift/linux/x86_64/swift_end.o -o hello

But if we want more:

$ swiftc -v Hello.swift -o hello -Xcc -v -Xlinker -v
Swift version 3.0-dev (LLVM 440a472499, Clang e10506ae1c, Swift 395e967875)
Target: x86_64-unknown-linux-gnu
/home/mominul/swift/usr/bin/swift -frontend -c -primary-file Hello.swift -target x86_64-unknown-linux-gnu -disable-objc-interop -Xcc -v -color-diagnostics -module-name hello -o /tmp/Hello-22ebeb.o
clang version 3.9.0 (git@github.com:apple/swift-clang.git e10506ae1c9a29193a18aea7c89e35d3f4ad3511) (git@github.com:apple/swift-llvm.git 440a472499a647dd03776a40ad16c0deb3be8aee)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: 
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/6.0.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0
Found candidate GCC installation: /usr/lib/gcc/x86_64-linux-gnu/6.0.0
Selected GCC installation: /usr/lib/gcc/x86_64-linux-gnu/5.4.0
Candidate multilib: .;@m64
Candidate multilib: 32;@m32
Candidate multilib: x32;@mx32
Selected multilib: .;@m64
ignoring nonexistent directory "/include"
#include "..." search starts here:
#include <...> search starts here:
 /home/mominul/swift/usr/lib/swift
 /usr/local/include
 /home/mominul/swift/usr/lib/swift/clang/include
 /usr/include/x86_64-linux-gnu
 /usr/include
End of search list.
/home/mominul/swift/usr/bin/swift-autolink-extract /tmp/Hello-22ebeb.o -o /tmp/Hello-ad3436.autolink
/usr/bin/clang++ -fuse-ld=gold -target x86_64-unknown-linux-gnu -Xlinker -rpath -Xlinker /home/mominul/swift/usr/lib/swift/linux /home/mominul/swift/usr/lib/swift/linux/x86_64/swift_begin.o /tmp/Hello-22ebeb.o -Xlinker -v -L /home/mominul/swift/usr/lib/swift/linux --target=x86_64-unknown-linux-gnu -lswiftCore @/tmp/Hello-ad3436.autolink /home/mominul/swift/usr/lib/swift/linux/x86_64/swift_end.o -o hello
GNU gold (GNU Binutils for Ubuntu 2.26.1) 1.11

But point to be noted that, if we pass -c flag, swiftc doesn't initiates clang to link the executable.

Thanks!

Nb: I always want to join with the mailing list but currently I don't know how to communicate with mailing lists.

tinysun212 commented 8 years ago

You can join here.

1) subscribe swift-dev (you should follow the confirmation procedure for validating your mail address) 2) browse past articles before sending your question (https://lists.swift.org/pipermail/swift-dev/) 3) send your question to swift-dev@swift.org , you will get answer in a few hours or days

Good luck to you.

mominul commented 8 years ago

Hi, I have posted a thread to discuss about the design of environment check conditional: https://lists.swift.org/pipermail/swift-evolution/Week-of-Mon-20160815/026472.html What is your opinion?

tinysun212 commented 7 years ago

The discussion about the design of environment check conditional is on the mailing list and is not the subject of this issue. I think we knew the linking process of the Swift compiler, thank you.