search

usiegl00 / tamatoa

If you have any questions, please open an issue.
The Unlicense
23 stars 6 forks source link

Can this work with binaries written in Swift? #7

Closed slyd0g closed 2 years ago

slyd0g commented 3 years ago

+------------------------+ | Describe Your Question | +------------------------+

Hello! Awesome work on this tool first off. I was testing it and it works great with binaries written in C and ObjC but I haven't been able to get it working with Swift binaries. I played around with it a bit and thought maybe it was some error calculating offsets with Swift binaries but I manually ran the Ruby commands and saw the offset was correct. Maybe some funkiness with Swift shellcode? Wondering if you had some insight! image

usiegl00 commented 2 years ago

I have made some updates. I tried a simple Swift macho just now, and it worked.

slyd0g commented 2 years ago

Hey @usiegl00, were the updates pushed to the main branch? I don't see any recent updates. Made sure I had the latest commit but it still didn't appear to work. For reference, I'm using a simple Swift binary that just prints "Hello from Swift!" image

usiegl00 commented 2 years ago

Hmm, what version of Mac OS are you using?

slyd0g commented 2 years ago

I'm on macOS Big Sur Version 11.4

usiegl00 commented 2 years ago

Are you on an M1? (This project does not support ARM right now.)

slyd0g commented 2 years ago

I'm not on an M1 :/ just a normal Intel Core

slyd0g commented 2 years ago

What macOS version are you on?

usiegl00 commented 2 years ago

Catalina. Could you send me the swift code you are using to test?

slyd0g commented 2 years ago

Yep, it's a very basic program.

// // main.swift // SwiftHelloWorldBasic // // Created by Justin Bui on 11/18/21. //

import Foundation

NSLog("Hello, World!")

usiegl00 commented 2 years ago

I was not able to replicate this issue on Big Sur. (11.2) Following the build steps (here)[https://www.swift.org/getting-started/#building-an-executable]. Both print and NSLog worked normally.

Could you send me your "tamatoa.bin" and "macho.s" so that I can take a look at the generated files? If you are using any specific swift build flags, could you post them here as well?

slyd0g commented 2 years ago

Interesting, I was able to get it to build a basic Swift binary following the steps you mentioned. I wonder what XCode is doing that makes the executable format slightly different. image

slyd0g commented 2 years ago

So I've done a good amount of testing with Swift and tamatoa and wanted to share my results:

Not sure if that helps at all, but just wanted to share incase other people were looking into this 😬

usiegl00 commented 2 years ago

It might be related to some build flags that are being set by Xcode and swiftc. Can you check what build command Xcode is running?

slyd0g commented 2 years ago

This is from the "Compile Swift source files" section in Xcode: CompileSwiftSources normal x86_64 com.apple.xcode.tools.swift.compiler (in target 'SwiftHelloWorldBasic' from project 'SwiftHelloWorldBasic') cd /Users/slyd0g/Projects/SwiftHelloWorldBasic export DEVELOPER_DIR\=/Applications/Xcode.app/Contents/Developer export SDKROOT\=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -incremental -module-name SwiftHelloWorldBasic -Onone -enable-batch-mode -enforce-exclusivity\=checked @/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic.SwiftFileList -DDEBUG -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -target x86_64-apple-macos11.3 -g -module-cache-path /Users/slyd0g/Library/Developer/Xcode/DerivedData/ModuleCache.noindex -Xfrontend -serialize-debugging-options -enable-testing -index-store-path /Users/slyd0g/Library/Developer/Xcode/DerivedData/SwiftHelloWorldBasic-eseejzupwjyervgiunwndspktfpu/Index/DataStore -swift-version 5 -I /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/Debug -F /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/Debug -c -j16 -output-file-map /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic-OutputFileMap.json -parseable-output -serialize-diagnostics -emit-dependencies -emit-module -emit-module-path /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic.swiftmodule -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/swift-overrides.hmap -Xcc -iquote -Xcc /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-generated-files.hmap -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-own-target-headers.hmap -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-all-target-headers.hmap -Xcc -iquote -Xcc /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/SwiftHelloWorldBasic-project-headers.hmap -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/Debug/include -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/DerivedSources-normal/x86_64 -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/DerivedSources/x86_64 -Xcc -I/Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/DerivedSources -Xcc -DDEBUG\=1 -emit-objc-header -emit-objc-header-path /Users/slyd0g/Projects/SwiftHelloWorldBasic/build/SwiftHelloWorldBasic.build/Debug/SwiftHelloWorldBasic.build/Objects-normal/x86_64/SwiftHelloWorldBasic-Swift.h -working-directory /Users/slyd0g/Projects/SwiftHelloWorldBasic

usiegl00 commented 2 years ago

Okay, then compare that with running swift build -v.

slyd0g commented 2 years ago

slyd0g@Justins-MacBook-Pro~/Projects/SwiftHelloWorldCLIBuild$ swift build -v /usr/bin/xcrun --sdk macosx --show-sdk-platform-path /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info /usr/bin/xcrun --sdk macosx --find xctest /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info Planning build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -print-target-info /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2/libPackageDescription.dylib /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/swiftc -L /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -lPackageDescription -Xlinker -rpath -Xlinker /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -target x86_64-apple-macosx10.15 -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -F /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks -I /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -L /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/usr/lib -swift-version 5 -I /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/pm/4_2 -sdk /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.3.sdk -package-description-version 5.3.0 /Users/slyd0g/Projects/SwiftHelloWorldCLIBuild/Package.swift -o /var/folders/kc/jc64g3ss3n529s1r3d44b0tw0000gn/T/TemporaryDirectory.62lifh/swifthelloworldclibuild-manifest /usr/bin/sandbox-exec -p '(version 1) (deny default) (import "system.sb") (allow file-read) (allow process) ' /var/folders/kc/jc64g3ss3n529s1r3d44b0tw0000gn/T/TemporaryDirectory.62lifh/swifthelloworldclibuild-manifest -fileno 6 /usr/bin/xcrun --sdk macosx --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/Library/Frameworks/XCTest.framework/XCTest /usr/bin/xcrun --sdk iphoneos --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest /usr/bin/xcrun --sdk appletvos --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/AppleTVOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest /usr/bin/xcrun --sdk watchos --show-sdk-platform-path /usr/bin/xcrun vtool -show-build /Applications/Xcode.app/Contents/Developer/Platforms/WatchOS.platform/Developer/Library/Frameworks/XCTest.framework/XCTest

usiegl00 commented 2 years ago

Try adding the flag: -enable-objc-interop to your Xcode build.

slyd0g commented 2 years ago

Sorry, I'm not the quickest with Xcode settings but it appears this might be a similar setting? image

What lead you to believe that was the difference? I didn't see that within the swift build output.

usiegl00 commented 2 years ago

It was in my swift build output, but not in your Xcode output. Did it work?

slyd0g commented 2 years ago

It didn't unfortunately, the setting was already enabled. I'll look into how to add that exact CLI argument in Xcode and let you know.

slyd0g commented 2 years ago

It appears that CLI argument doesn't exist on my version of swiftc image

slyd0g commented 2 years ago

It's particularly odd because I was able to compile this project in Xcode and get it working with tamatoa https://github.com/slyd0g/SwiftLiverpool image

usiegl00 commented 2 years ago

Are there any Xcode build config differences between SwiftLiverpool and SwiftHelloWorldBasic?

slyd0g commented 2 years ago

Not to my knowledge, I didn't modify any particular settings for either projects.

usiegl00 commented 2 years ago

It might have to do with the executable size... Try removing the strip commands from the Makefile.

slyd0g commented 2 years ago

Removed the strips under the tamatoa: section. Still no luck unfortunately, I can give you all the source code if that makes it easier. SwiftHelloWorldBasic fails, but SwiftLiverpool still works. image image

slyd0g commented 2 years ago

It's honestly not too big of a deal, I've tried a couple other Swift projects also compiled with Xcode and they work fine. I'm just confused as to why a simple Hello World in Xcode does not.

usiegl00 commented 2 years ago

So am I.

slyd0g commented 2 years ago

I'll let you know projects that work and projects that don't as I continue testing and maybe we can narrow something down. Thank you again for your work on this project, it's really awesome and very useful :)

slyd0g commented 2 years ago

The other case that particularly stumped me was NSLog compiled with swiftc. The same exact program with print works just fine: image

usiegl00 commented 2 years ago

This is very strange... That exact series of commands works for me on Big Sur.

Then I ran make clean and tried again. It didn't work after I ran make clean. I will do some more investigating...

usiegl00 commented 2 years ago

It works about half the time in lldb.

usiegl00 commented 2 years ago

Okay, tamatoa is running the swift code at the correct entrypoint. However I believe the garbage collector is causing the program to crash every other time.

Maybe we can change the mmap flags? Right now they are MAP_EXECUTABLE and MAP_PRIVATE.

usiegl00 commented 2 years ago

The garbage collection is removing the memory that argv points to. This memory is being accessed later by NSLog.

There must be a way that we can allocate a new page for the args and copy them over.

usiegl00 commented 2 years ago

Even after correcting argv and argc, swiftc still fails around half the time.

slyd0g commented 2 years ago

Yeah very odd, I wonder if it's just an issue with "smaller" programs that causes this race condition. I'm consistently able to get more complex Swift programs to work.

usiegl00 commented 2 years ago

I have added a Wiki page recommending the use of swift build instead of swiftc or Xcode.

slyd0g commented 2 years ago

Ah awesome :) you can consider this closed then. Appreciate you looking into that!

usiegl00 commented 2 years ago

I've made some changes and have been unable to trigger the crash. Do you want to take another look at replicating this issue?

slyd0g commented 2 years ago

I've made a note to test the cmdline argument patching + this fix, will take a look as soon as I can and let you know!

slyd0g commented 2 years ago

I re-tested with some binaries that didn't work previously and it seems they work consistently now! :)

Awesome work