search

HarbourMasters / Shipwright

3.15k stars 481 forks source link

Linux/gcc/clang: ZAPD.out refuses to link with libultraship.a #358

Closed jason-oliveira closed 2 years ago

jason-oliveira commented 2 years ago

Description: ZAPD.out refuses to link with the libultraship.a built.

Expected Results: code compiles cleanly and links without issue.

Actual Results: ZAPD.out vomits all over itself by saying libultraship.a is incompatible.

How to reproduce: 1) install gentoo. 2) follow BUILDING.md, ignoring docker appropriately 3) watch as make setup -j$(nproc) OPTFLAGS=-O0 DEBUG=0 fails because of libultraship.a

sohcompilationfail.log

th-2021 commented 2 years ago

you need to compile soh for 32 bit and also ultraship for soh. You can use 64 bit only for OTRExporter.

GreenSwede commented 2 years ago

(unrelated to the issue, but you may want to include the -O2 flag instead of -O0 for better performance)

IvarWithoutBones commented 2 years ago

I'm having the same issue on NixOS when trying to use make setup. The -m32 flag does get appended to the compilation commands for libultraship, yet it fails to link exactly the same as in the log described above with the regular ld and a 32bit compatible gcc.

Forcing a 32bit-only GCC build gives more linking issues, see this log file. It fails to link against BinaryReader functions (and more) a bunch of times: libultraship/libultraship/Factories/CutsceneFactory.cpp:8: undefined reference to BinaryReader::ReadUInt32(). I tried to add -m32 to CXXFLAGS/CFLAGS as well just to double check, but that didn't seem to fix anything.

I noticed the dockerfile uses lld, which fails as follows:

shipwright-unstable> g++ -fpic -std=c++17 -Wall -Wextra -fno-omit-frame-pointer -DDEPRECATION_ON build/ZAPD/Declaration.o build/ZAPD/FileWorker.o build/ZAPD/GameConfig.o build/ZAPD/Globals.o build/ZAPD/ImageBackend.o build/ZAPD/Main.o build/ZAP
D/OutputFormatter.o build/ZAPD/WarningHandler.o build/ZAPD/ZAnimation.o build/ZAPD/ZArray.o build/ZAPD/ZBackground.o build/ZAPD/ZBlob.o build/ZAPD/ZCollision.o build/ZAPD/ZCutscene.o build/ZAPD/ZCutsceneMM.o build/ZAPD/ZDisplayList.o build/ZAPD
/ZFile.o build/ZAPD/ZLimb.o build/ZAPD/ZMtx.o build/ZAPD/ZPath.o build/ZAPD/ZPlayerAnimationData.o build/ZAPD/ZResource.o build/ZAPD/ZRom.o build/ZAPD/ZScalar.o build/ZAPD/ZSkeleton.o build/ZAPD/ZString.o build/ZAPD/ZSymbol.o build/ZAPD/ZText.o
 build/ZAPD/ZTexture.o build/ZAPD/ZTextureAnimation.o build/ZAPD/ZVector.o build/ZAPD/ZVtx.o build/ZAPD/Overlays/ZOverlay.o build/ZAPD/ZRoom/ZRoom.o build/ZAPD/ZRoom/ZRoomCommand.o build/ZAPD/ZRoom/Commands/EndMarker.o build/ZAPD/ZRoom/Commands
/SetActorCutsceneList.o build/ZAPD/ZRoom/Commands/SetActorList.o build/ZAPD/ZRoom/Commands/SetAlternateHeaders.o build/ZAPD/ZRoom/Commands/SetAnimatedMaterialList.o build/ZAPD/ZRoom/Commands/SetCameraSettings.o build/ZAPD/ZRoom/Commands/SetColl
isionHeader.o build/ZAPD/ZRoom/Commands/SetCsCamera.o build/ZAPD/ZRoom/Commands/SetCutscenes.o build/ZAPD/ZRoom/Commands/SetEchoSettings.o build/ZAPD/ZRoom/Commands/SetEntranceList.o build/ZAPD/ZRoom/Commands/SetExitList.o build/ZAPD/ZRoom/Comm
ands/SetLightList.o build/ZAPD/ZRoom/Commands/SetLightingSettings.o build/ZAPD/ZRoom/Commands/SetMesh.o build/ZAPD/ZRoom/Commands/SetMinimapChests.o build/ZAPD/ZRoom/Commands/SetMinimapList.o build/ZAPD/ZRoom/Commands/SetObjectList.o build/ZAPD
/ZRoom/Commands/SetPathways.o build/ZAPD/ZRoom/Commands/SetRoomBehavior.o build/ZAPD/ZRoom/Commands/SetRoomList.o build/ZAPD/ZRoom/Commands/SetSkyboxModifier.o build/ZAPD/ZRoom/Commands/SetSkyboxSettings.o build/ZAPD/ZRoom/Commands/SetSoundSett
ings.o build/ZAPD/ZRoom/Commands/SetSpecialObjects.o build/ZAPD/ZRoom/Commands/SetStartPositionList.o build/ZAPD/ZRoom/Commands/SetTimeSettings.o build/ZAPD/ZRoom/Commands/SetTransitionActorList.o build/ZAPD/ZRoom/Commands/SetWind.o build/ZAPD/
ZRoom/Commands/SetWorldMapVisited.o build/ZAPD/ZRoom/Commands/Unused09.o build/ZAPD/ZRoom/Commands/Unused1D.o build/ZAPD/ZRoom/Commands/ZRoomCommandUnk.o build/ZAPD/OtherStructs/SkinLimbStructs.o build/ZAPD/yaz0/yaz0.o build/lib/tinyxml2/tinyxm
l2.o build/ZAPD/BuildInfo.o lib/libgfxd/libgfxd.a ZAPDUtils/ZAPDUtils.a -Wl,--whole-archive ../OTRExporter/OTRExporter/OTRExporter.a -Wl,--no-whole-archive -lm -ldl -lpng -L../external -L../libultraship -lz -lbz2 -pthread -lpulse -lultraship -l
storm -lSDL2 -lGLEW -lGL -lX11 -fuse-ld=lld -Wl,-export-dynamic -lstdc++fs -o ZAPD.out
shipwright-unstable> ld.lld: error: imgui_impl_opengl3.cpp:(.debug_info+0x6898FD8): has non-ABS relocation R_386_GOTOFF against symbol '.LC35'
shipwright-unstable> ld.lld: error: ArrayExporter.cpp:(.debug_info+0x261C48): has non-ABS relocation R_386_GOTOFF against symbol '.LC16'
shipwright-unstable> ld.lld: error: PlayerAnimation.cpp:(.debug_info+0x63F63B1): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: BackgroundExporter.cpp:(.debug_info+0x3CA457): has non-ABS relocation R_386_GOTOFF against symbol '.LC17'
shipwright-unstable> ld.lld: error: Texture.cpp:(.debug_info+0x669200C): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: AnimationExporter.cpp:(.debug_info+0x1030D9): has non-ABS relocation R_386_GOTOFF against symbol '.LC16'
shipwright-unstable> ld.lld: error: CollisionExporter.cpp:(.debug_info+0x6A5CD7): has non-ABS relocation R_386_GOTOFF against symbol '.LC16'
shipwright-unstable> ld.lld: error: CutsceneExporter.cpp:(.debug_info+0x81466E): has non-ABS relocation R_386_GOTOFF against symbol '.LC17'
shipwright-unstable> ld.lld: error: Matrix.cpp:(.debug_info+0x6545326): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: BlobExporter.cpp:(.debug_info+0x533558): has non-ABS relocation R_386_GOTOFF against symbol '.LC17'
shipwright-unstable> ld.lld: error: DisplayListExporter.cpp:(.debug_info+0x9A4C71): has non-ABS relocation R_386_GOTOFF against symbol '.LC17'
shipwright-unstable> ld.lld: error: GameOverlay.cpp:(.debug_info+0x628DC11): has non-ABS relocation R_386_GOTOFF against symbol '.LC16'
shipwright-unstable> ld.lld: error: Skeleton.cpp:(.debug_info+0x592C6D0): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: Text.cpp:(.debug_info+0x5EA7B05): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: SohImGuiImpl.cpp:(.debug_info+0x605B17F): has non-ABS relocation R_386_GOTOFF against symbol '.LC395'
shipwright-unstable> ld.lld: error: Material.cpp:(.debug_info+0x5A7E7A6): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: ModManager.cpp:(.debug_info+0x5D58CDB): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: GameSettings.cpp:(.debug_info+0x57DC0F0): has non-ABS relocation R_386_GOTOFF against symbol '.LC15'
shipwright-unstable> ld.lld: error: Controller.cpp:(.debug_info+0x5BDA0B5): has non-ABS relocation R_386_GOTOFF against symbol '.LC16'
shipwright-unstable> ld.lld: error: PulseAudioPlayer.cpp:(.debug_info+0x5654E58): has non-ABS relocation R_386_GOTOFF against symbol '.LC17'
shipwright-unstable> ld.lld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)
shipwright-unstable> collect2: error: ld returned 1 exit status
shipwright-unstable> make[2]: *** [Makefile:135: ZAPD.out] Error 1
shipwright-unstable> make[2]: Leaving directory '/build/source/ZAPDTR'
shipwright-unstable> make[1]: *** [Makefile:130: mpq] Error 2
shipwright-unstable> make[1]: Leaving directory '/build/source/soh'
shipwright-unstable> make: *** [Makefile:125: setup] Error 2

Output of gcc -v seems to confirm the 32 bit target is set correctly:

$ gcc -v
Using built-in specs.
COLLECT_GCC=/nix/store/n2ayv1l6gi3gp30wx48yzbxd2anp4jjv-gcc-10.3.0/bin/gcc
COLLECT_LTO_WRAPPER=/nix/store/n2ayv1l6gi3gp30wx48yzbxd2anp4jjv-gcc-10.3.0/libexec/gcc/i686-unknown-linux-gnu/10.3.0/lto-wrapper
Target: i686-unknown-linux-gnu
Configured with:
Thread model: posix
Supported LTO compression algorithms: zlib
gcc version 10.3.0 (GCC)

readelf also seems to think libultraship.a has an ELF32 header.

PaulGrandperrin commented 2 years ago

Hey @IvarWithoutBones ! I'm also playing with packaging soh on nix :) Would you be interested in sharing code/experience? My code is still very WIP so I haven't published it yet but it's based on another similar thing I did with sm64ex: https://github.com/PaulGrandperrin/sm64nix/blob/main/flake.nix

IvarWithoutBones commented 2 years ago

Hi @PaulGrandperrin, I have the WIP derivation pushed to my nixpkgs fork: https://github.com/IvarWithoutBones/nixpkgs/commit/7d931fcc105f9211578ee680ef53482fd5d4835d. Funny you should mention sm64ex, I wrote and maintain the derivation for that :)

PaulGrandperrin commented 2 years ago

@IvarWithoutBones yeah ahah, I so didn't expect sm64ex to be packaged in Nix that I didn't even check before doing my own! It was nice for practicing anyway ;-)

I'm currently able to build and run master, I'll cleanup my code and share it with you.

Is there a good place where we could communicate without polluting this issue?

IvarWithoutBones commented 2 years ago

@IvarWithoutBones yeah ahah, I so didn't expect sm64ex to be packaged in Nix that I didn't even check before doing my own! It was nice for practicing anyway ;-)

Always nice to have stuff in the main repos IMO, I was planning on doing the same for soh :blush:

I'm currently able to build and run master, I'll cleanup my code and share it with you.

That would be great! Very interested in seeing how you worked around these linker issues, a great excuse to learn a bit more about cross compilation.

Is there a good place where we could communicate without polluting this issue?

I shot you a message on what I believe is your Discord account, searched for your name in the NixOS server and found @paulg :)

PaulGrandperrin commented 2 years ago

@IvarWithoutBones yes, please drop a message on discord (@paulg), I would also be happy to push some of my work on sm64 :)

Here's my WIP on zelda: https://github.com/PaulGrandperrin/zelda64nix

nix build -L github:PaulGrandperrin/zelda64nix

Looking at your code, I'm quite surprised that we absolutely didn't need to patch the same things!

Also, I'm still learning Nix and this is the first time I do cross-compilation, so maybe it's not optimal... It pulls quite a few dependencies that needs local building.

IvarWithoutBones commented 2 years ago

@IvarWithoutBones yes, please drop a message on discord (@paulg), I would also be happy to push some of my work on sm64 :)

@PaulGrandperrin I already have. Searching my name on the NixOS discord should also give you my account, in case my DM got lost :)

Here's my WIP on zelda: https://github.com/PaulGrandperrin/zelda64nix

Thanks for sharing! After copying your hardeningDisable and llvmPackages.binutils lines it appears I can build master just fine as well, so I'll go ahead and PR my derivation to nixpkgs :+1: