Closed slyd0g closed 2 years ago
I have made some updates. I tried a simple Swift macho just now, and it worked.
Hey @usiegl00, were the updates pushed to the main branch? I don't see any recent updates. Made sure I had the latest commit but it still didn't appear to work. For reference, I'm using a simple Swift binary that just prints "Hello from Swift!"
Hmm, what version of Mac OS are you using?
I'm on macOS Big Sur Version 11.4
Are you on an M1? (This project does not support ARM right now.)
I'm not on an M1 :/ just a normal Intel Core
What macOS version are you on?
Catalina. Could you send me the swift code you are using to test?
Yep, it's a very basic program.
// // main.swift // SwiftHelloWorldBasic // // Created by Justin Bui on 11/18/21. //
import Foundation
NSLog("Hello, World!")
I was not able to replicate this issue on Big Sur. (11.2) Following the build steps (here)[https://www.swift.org/getting-started/#building-an-executable]. Both print and NSLog worked normally.
Could you send me your "tamatoa.bin" and "macho.s" so that I can take a look at the generated files? If you are using any specific swift build flags, could you post them here as well?
Interesting, I was able to get it to build a basic Swift binary following the steps you mentioned. I wonder what XCode is doing that makes the executable format slightly different.
So I've done a good amount of testing with Swift and tamatoa and wanted to share my results:
Not sure if that helps at all, but just wanted to share incase other people were looking into this 😬
It might be related to some build flags that are being set by Xcode and swiftc. Can you check what build command Xcode is running?
This is from the "Compile Swift source files" section in Xcode: CompileSwiftSources normal x86_64 com.apple.xcode.tools.swift.compiler (in target 'SwiftHelloWorldBasic' from project 'SwiftHelloWorldBasic') cd /Users/slyd0g/Projects/SwiftHelloWorldBasic export DEVELOPER_DIR\=/Applications/Xcode.app/Contents/Developer export SDKROOT\=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -incremental -module-name SwiftHelloWorldBasic -Onone -enable-batch-mode -enforce-exclusivity\=checked @/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic.SwiftFileList -DDEBUG -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -target x86_64-apple-macos11.3 -g -module-cache-path /Users/slyd0g/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -Xfrontend -serialize-debugging-options -enable-testing -index-store-path /Users/slyd0g/Library/Developer/Xcode/DerivedData/SwiftHelloWorldBasic-eseejzupwjyervgiunwndspktfpu/Index/DataStore -swift-version 5 -I /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/Debug -F /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/Debug -c -j16 -output-file-map /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic-OutputFileMap.json -parseable-output -serialize-diagnostics -emit-dependencies -emit-module -emit-module-path /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic.swiftmodule -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-generated-files.hmap -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-own-target-headers.hmap -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-all-target-headers.hmap -Xcc -iquote -Xcc /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-project-headers.hmap -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/Debug/include -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/DerivedSources-normal/x86_64 -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/DerivedSources/x86_64 -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/DerivedSources -Xcc -DDEBUG\=1 -emit-objc-header -emit-objc-header-path /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic-Swift.h -working-directory /Users/slyd0g/Projects/SwiftHelloWorldBasic
Okay, then compare that with running swift build -v
.
slyd0g@Justins-MacBook-Pro~/Projects/SwiftHelloWorldCLIBuild$ swift build -v /usr/bin/xcrun --sdk macosx --show-sdk-platform-path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info /usr/bin/xcrun --sdk macosx --find xctest /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info Planning build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2/libPackageDescription.dylib /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -L /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -target x86_64-apple-macosx10.15 -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -L /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -swift-version 5 -I /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -package-description-version 5.3.0 /Users/slyd0g/Projects/SwiftHelloWorldCLIBuild/Package.swift -o /var/folders/kc/jc64g3ss3n529s1r3d44b0tw0000gn/T/TemporaryDirectory.62lifh/swifthelloworldclibuild-manifest /usr/bin/sandbox-exec -p '(version 1) (deny default) (import "system.sb") (allow file-read) (allow process) ' /var/folders/kc/jc64g3ss3n529s1r3d44b0tw0000gn/T/TemporaryDirectory.62lifh/swifthelloworldclibuild-manifest -fileno 6 /usr/bin/xcrun --sdk macosx --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/XCTest.framework/XCTest /usr/bin/xcrun --sdk iphoneos --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest /usr/bin/xcrun --sdk appletvos --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest /usr/bin/xcrun --sdk watchos --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest
Try adding the flag: -enable-objc-interop
to your Xcode build.
Sorry, I'm not the quickest with Xcode settings but it appears this might be a similar setting?
What lead you to believe that was the difference? I didn't see that within the swift build
output.
It was in my swift build output, but not in your Xcode output. Did it work?
It didn't unfortunately, the setting was already enabled. I'll look into how to add that exact CLI argument in Xcode and let you know.
It appears that CLI argument doesn't exist on my version of swiftc
It's particularly odd because I was able to compile this project in Xcode and get it working with tamatoa https://github.com/slyd0g/SwiftLiverpool
Are there any Xcode build config differences between SwiftLiverpool and SwiftHelloWorldBasic?
Not to my knowledge, I didn't modify any particular settings for either projects.
It might have to do with the executable size... Try removing the strip commands from the Makefile.
Removed the strips under the tamatoa: section. Still no luck unfortunately, I can give you all the source code if that makes it easier. SwiftHelloWorldBasic fails, but SwiftLiverpool still works.
It's honestly not too big of a deal, I've tried a couple other Swift projects also compiled with Xcode and they work fine. I'm just confused as to why a simple Hello World in Xcode does not.
So am I.
I'll let you know projects that work and projects that don't as I continue testing and maybe we can narrow something down. Thank you again for your work on this project, it's really awesome and very useful :)
The other case that particularly stumped me was NSLog compiled with swiftc
. The same exact program with print works just fine:
This is very strange... That exact series of commands works for me on Big Sur.
Then I ran make clean
and tried again.
It didn't work after I ran make clean
.
I will do some more investigating...
It works about half the time in lldb.
Okay, tamatoa is running the swift code at the correct entrypoint. However I believe the garbage collector is causing the program to crash every other time.
Maybe we can change the mmap flags? Right now they are MAP_EXECUTABLE and MAP_PRIVATE.
The garbage collection is removing the memory that argv points to. This memory is being accessed later by NSLog.
There must be a way that we can allocate a new page for the args and copy them over.
Even after correcting argv and argc, swiftc still fails around half the time.
Yeah very odd, I wonder if it's just an issue with "smaller" programs that causes this race condition. I'm consistently able to get more complex Swift programs to work.
I have added a Wiki page recommending the use of swift build
instead of swiftc
or Xcode.
Ah awesome :) you can consider this closed then. Appreciate you looking into that!
I've made some changes and have been unable to trigger the crash. Do you want to take another look at replicating this issue?
I've made a note to test the cmdline argument patching + this fix, will take a look as soon as I can and let you know!
I re-tested with some binaries that didn't work previously and it seems they work consistently now! :)
Awesome work
+------------------------+ | Describe Your Question | +------------------------+
Hello! Awesome work on this tool first off. I was testing it and it works great with binaries written in C and ObjC but I haven't been able to get it working with Swift binaries. I played around with it a bit and thought maybe it was some error calculating offsets with Swift binaries but I manually ran the Ruby commands and saw the offset was correct. Maybe some funkiness with Swift shellcode? Wondering if you had some insight!