search

GaloisInc / saw-script

The SAW scripting language.
BSD 3-Clause "New" or "Revised" License
442 stars 63 forks source link

Mac ARM build issue (ld error in libHSlmdb-0.2.5-inplace.a) #2084

Closed Torrencem closed 3 months ago

Torrencem commented 3 months ago

I'm trying to build using build.sh on an ARM mac, when I run build.sh I get

... normal build process ...
Building executable 'saw-remote-api' for saw-remote-api-1.1.0..
[1 of 1] Compiling Main             ( saw-remote-api/Main.hs, /Users/may.torrence/Documents/saw-script/dist-newstyle/build/aarch64-osx/ghc-9.6.5/saw-remote-api-1.1.0/x/saw-remote-api/build/saw-remote-api/saw-remote-api-tmp/Main.o )
[2 of 2] Linking /Users/may.torrence/Documents/saw-script/dist-newstyle/build/aarch64-osx/ghc-9.6.5/saw-remote-api-1.1.0/x/saw-remote-api/build/saw-remote-api/saw-remote-api
final section layout:
    __TEXT/__text addr=0x100009DC0, size=0x08B68CB0, fileOffset=0x00009DC0, type=1
... A LOT of data ...
    __DATA/__bss addr=0x10A3F6CD8, size=0x000011E8, fileOffset=0x00000000, type=26
ld: b(l) ARM64 branch out of range (136736708 max is +/-128MB): from _mdb_rebalance (0x100CA7158) to _mdb_assert_fail (0x108F0EFAC) in '_mdb_rebalance' from /Users/may.torrence/Documents/saw-script/dist-newstyle/build/aarch64-osx/ghc-9.6.5/lmdb-0.2.5/build/libHSlmdb-0.2.5-inplace.a(mdb.o) for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ghc-9.6.5: `gcc' failed in phase `Linker'. (Exit code: 1)
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-aclpsvXx] source_file target_file
       cp [-R [-H | -L | -P]] [-fi | -n] [-aclpsvXx] source_file ... target_directory

COPIED EXECUTABLES TO /Users/may.torrence/Documents/saw-script/bin.

I googled around, and wasn't finding anything helpful related to anything here, but it's probably like a configuration issue. Any suggestions for troubleshooting this?

Torrencem commented 3 months ago

I get this error on the v1.1 tag as well as on master, by the way, although I think it happens at a different place in master, in a different library. I unfortunately don't have the data without completely rerunning it for 15 minutes, but it was a very similar error message

kquick commented 3 months ago

Looks like it might be an Xcode 15 issue: https://forums.developer.apple.com/forums/thread/738275

You can try editing the build.sh to add that flag. I'm don't have a Mac to verify this, but it might be something like cabal build --clang-option=-ld64 ... or alternatively ... --ld-option=-ld64.

RyanGlScott commented 3 months ago

This also bears a strong resemblance to https://gitlab.haskell.org/ghc/ghc/-/issues/24648 . In that issue, this comment suggests using -ld_classic to work around the issue. You could try to put the following in the cabal.project file:

package *
  ghc-options: -optl-Wl,-ld_classic
Torrencem commented 3 months ago

Yeah, with cabal build ... --ld-option=-ld64 I'm getting 

ld: library not found for -ld64

let me try -ld_classic

Torrencem commented 3 months ago

I'm also getting

ld: library not found for -ld_classic

on compiling with those ghc-options in cabal.project. Hmm... Let me see if I can figure out how to get ld aware of these

Torrencem commented 3 months ago

Curiously, ld64 and ld_classic seem to be options only on xcode version 15, and I'm running Xcode 14.3.1, so I don't think this will solve it. I'm going to try updating my system to see if that helps though (I have a feeling it won't)

RyanGlScott commented 3 months ago

Ugh, that is unfortunate. It appears even Apple acknowledges this issue and the need to use a particular Xcode version for the -ld_classic/-ld64 workaround to succeed. I'll look to see if there is a way to work around on the issue on Xcode 14.

In the meantime, this discussion on this GHC issue notes that you should also be able to work around the issue by enabling -fllvm, which instructs GHC to use the LLVM backend instead of the native AArch64 backend. Do note that this will require having LLVM installed in order to work.

RyanGlScott commented 3 months ago

I am somewhat surprised that our nightly AArch64 macOS builds of SAW aren't running into this same issue. If nothing else, the nightly builds could serve as a way to obtain a working SAW binary.

Torrencem commented 3 months ago

adding -fllvm to ghc args gives me:

... a lot of compiling ...
[ 9 of 10] Compiling SAWScript.REPL   ( saw/SAWScript/REPL.hs, /Users/may.torrence/Documents/saw-script/dist-newstyle/build/aarch64-osx/ghc-9.6.5/saw-script-1.1/build/saw/saw-tmp/SAWScript/REPL.o ) [SAWScript.REPL.Haskeline changed]
[10 of 10] Compiling Main             ( saw/Main.hs, /Users/may.torrence/Documents/saw-script/dist-newstyle/build/aarch64-osx/ghc-9.6.5/saw-script-1.1/build/saw/saw-tmp/Main.o ) [Data.AIG.CompactGraph changed]
[11 of 11] Linking /Users/may.torrence/Documents/saw-script/dist-newstyle/build/aarch64-osx/ghc-9.6.5/saw-script-1.1/build/saw/saw [Objects changed]
clang: error: unable to execute command: Segmentation fault: 11
clang: error: linker command failed due to signal (use -v to see invocation)
ghc-9.6.5: `gcc' failed in phase `Linker'. (Exit code: 254)
cabal: Failed to build saw-script-1.1 (which is required by exe:saw-remote-api
from saw-remote-api-1.1.0).

this is certainly a new one for me haha. I will take your advice and pull the nightly binaries. I was initially looking for those but I thought building it myself would be easy enough (I didn't see any AArch64 macOS binaries in the release location).

Torrencem commented 3 months ago

I don't know if I want to troubleshoot this more or just accept that my linker is haunted and close the issue, since you have built macos binaries and it certainly isn't critical I get this building on my system. Feel free to probe more but I'm ending it at "managed to make clang segfault"

RyanGlScott commented 3 months ago

managed to make clang segfault

:upside_down_face:

I didn't see any AArch64 macOS binaries in the release location

Right, this is something which has only been implemented for a relatively short period of time—we haven't had a chance to make a full release cycle with AArch64 macOS binaries as of yet. In the meantime, you can download a nightly build by:

(I wish GitHub offered a better user interface for this, but that's what we have to work with at the moment.)

RyanGlScott commented 3 months ago

Sadly, I uncovered the same bug when attempting to upgrade the CI from GHC 9.2.8 to 9.4.8. After trying several different alternative compilers and linkers (see the discussion thread on the GHC issue tracker here), I was unable to find a workaround. I think this means that we will need to keep the macOS CI build pinned to GHC 9.2.8 for now until we find a better solution.

(Note: there is a GHC merge request here which claims to implement a workaround. This merge request has not yet landed, however, and it is unclear if it will be backported to any of the GHC series that we currently build SAW against in the CI.)

I am unclear if this is an actual GHC regression that was introduced between 9.2.8 and 9.4.8, or it's just a matter of the SAW binary size increasing ever-so-slightly when built with 9.4.8 so as to trigger the linker size limitation seen above. Either way, using GHC 9.2.8 is the only way I know to avoid this problem. When upgrading the CI next, we should make sure to leave comments explaining why we do this, as well as a general disclaimer in the README. I'll re-open this issue as a reminder to do so.

Ptival commented 3 months ago

Note: this problem seems to be strictly with building saw-remote-api. Building the saw tool itself seems to work.

RyanGlScott commented 3 months ago

@Torrencem: As it turns out, the issue involved a C library that SAW links against. I've pushed a fix in #2095, so it should now be much easier to build SAW from source on AArch64 macOS.