Restore LLVM compatibility

[Keithcat1]

I downloaded cosmopolitan.zip and tried to create me a portable executable using Clang. I thought that it should work since Clang can generate code for any CPU as long as you have the correct headers and libs ETC. C:\py\cosmopolitan>clang -g -O -o hello.com.dbg hello.c -static -fno-pie -no-pie -mno-red-zone -fno-omit-frame-pointer -nostdlib -nostdinc -Wl,--gc-sections -Wl,-z,max-page-size=0x1000 -Wl,-T,ape.lds -include cosm opolitan.h crt.o ape.o cosmopolitan.a --target=x86_64-pc-linux-gnu ld.lld: error: cosmopolitan.a(zip.o):(.zip.5+0x10): relocation R_X86_64_32 out o f range: 18446744073705357312 is not in [0, 4294967295] clang-12: error: linker command failed with exit code 1 (use -v to see invocation) It would be nice if this was supported, because just about everyone has clang on their computer.

[jart]

Could you try the 0.2 release? A regression slipped into the project at some point since then regarding LLVM support. We'll likely need to git bisect the repo to figure out exactly what caused it.

I would recommend building on Linux if at all possible. That's the recommended and most supported approach. There's also this: https://justine.lol/cosmopolitan/windows-compiling.html

[Keithcat1]

0.2 works, or at least gives the impression of working since it spat out some kind of thing. However, this doesn't work:

C:\py\cosmopolitan>llvm-objcopy.exe -SO binary hello.com.dbg hello.com llvm-objcopy.exe: error: too many positional arguments

I had to do: C:\py\cosmopolitan>llvm-objcopy.exe hello.com.dbg hello.com

But when I ran hello.com, I got: C:\py\COSMOP~1>hello Program too big to fit in memory

On 6/2/21, Justine Tunney @.***> wrote:

Could you try the 0.2 release? A regression slipped into the project at some point since then regarding LLVM support. We'll likely need to git bisect the repo to figure out exactly what caused it.

I would recommend building on Linux if at all possible. That's the recommended and most supported approach. There's also this: https://justine.lol/cosmopolitan/windows-compiling.html

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/jart/cosmopolitan/issues/180#issuecomment-853439501

[jart]

Try doing llvm-objcopy -S -O binary hello.com.dbg hello.com. Here's what I did and it worked with the 0.2 release.

clang -g -Os -static -nostdlib -nostdinc -fno-pie -mno-red-zone -fno-omit-frame-pointer -o hello.com.dbg hello.c -Wl,-T,ape.lds -include cosmopolitan.h --target=x86_64-pc-linux-gnu crt.o ape.o cosmopolitan.a -Wl,-z,max-page-size=0x1000 -Wl,--gc-sections -fuse-ld=lld
llvm-objcopy -S -O binary hello.com.dbg hello.com

Note that old release might have a bug with CMD.EXE integration. If you run the binary in mintty it's guaranteed to work. Whatever might be causing that, it got fixed before the 1.0 release.

Now that we know LLVM on Windows worked before, could you help us git bisect the repo so we can fix whatever introduced the regression? By the way, --target=x86_64-pc-linux-gnu is a brilliant insight I failed to consider. That could help us greatly.

[Keithcat1]

I followed your steps and I think the binary works, but I'm on a 32-bit computer and it seems that Cosmopolitan only supports 64-bit binaries so I am unsure. The target thing just seemed obvious to me. I guess though that I enjoy poking around with such things. It's also easy to forget about it because it's hard to find a good use for it, given that you need platform-specific stuff to compile and link against for your avarage C++ project. Can't bisect now, but I have WSL at home and will see what I can do.

On 6/3/21, Justine Tunney @.***> wrote:

Try doing llvm-objcopy -S -O binary hello.com.dbg hello.com. Here's what I did and it worked with the 0.2 release.

clang -g -Os -static -nostdlib -nostdinc -fno-pie -mno-red-zone
-fno-omit-frame-pointer -o hello.com.dbg hello.c -Wl,-T,ape.lds -include
cosmopolitan.h --target=x86_64-pc-linux-gnu crt.o ape.o cosmopolitan.a
-Wl,-z,max-page-size=0x1000 -Wl,--gc-sections -fuse-ld=lld
llvm-objcopy -S -O binary hello.com.dbg hello.com

Note that old release might have a bug with CMD.EXE integration. If you run the binary in mintty it's guaranteed to work. Whatever might be causing that, it got fixed before the 1.0 release.

Now that we know LLVM on Windows worked before, could you help us git bisect the repo so we can fix whatever introduced the regression? By the way, --target=x86_64-pc-linux-gnu is a brilliant insight I failed to consider. That could help us greatly.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/jart/cosmopolitan/issues/180#issuecomment-854298325

[Keithcat1]

I'm having problems with WSL2 similar to issue #6
C:\py\git>wsl keith@Keith-PC:/mnt/c/py/git$ cd cosmopolitan keith@Keith-PC:/mnt/c/py/git/cosmopolitan$ make

ERROR

Thompson Shell Backwards Compatibility Issue Detected

DETAILS

Actually Portable Executable assumes stock Linux configuration. Normal behavior is non-ELF files with x bit are run by /bin/sh. Linux lets people globally define arbitrary magic interpreters. Your computer couldve been tuned to run MZ scripts inside WINE. So if you use binfmt_misc you need to explicitly register this.

WORKAROUND

sudo sh -c "echo ':APE:M::MZqFpD::/bin/sh:' >/proc/sys/fs/binfmt_misc/register"

SEE ALSO

https://justine.storage.googleapis.com/ape.html

Terminated

That workaround didn't seem to do anything even after restarting WSL.

Unsupported 16-Bit Application The program or feature "\??\C:\py\git\cosmopolitan\build\sanitycheck2" cannot start or run due to incompatibity with 64-bit versions of Windows. Please contact the software vendor to ask if a 64-bit Windows compatible version is available.

[alisonatwork]

Try running wsl -l -v and check what version your WSL install is. For some reason I haven't had time to figure out, Cosmo doesn't work on WSL1 (see #76).

[Keithcat1]

Did a bisection and found it. Unfortunately, my computer died yesterday so the original git bisect message is lost. Here is the commit that I found, though it looks like the right one given that both the commit and the error have the word 'zip' in them. commit 94afa982c35d4322999b5591f6d41d5cfc3f19a5 Author: Justine Tunney jtunney@gmail.com Date: Sat Feb 27 18:02:32 2021 -0800

Fix zip executables on MacOS

Here's why we got those `Killed: 11` failures on MacOS after modifying
the contentns of the redbean.com executable. If you were inserting a
small file, such as a HelloWorld.html file, then InfoZIP might have
decreased the size of the executable to less than what the Mach-O
section had been expecting.

That's because when zipobj.com put things like time zone data in the
executable, it aligned each zip file entry on a 64-byte boundary, simply
for the sake of readability in binary dumps. But when InfoZIP edited the
file it would rewrite every entry using ZIP's usual 2-byte alignment.
Thus causing shrinkage.

The solution was to reconfigure the linker script so that zip file bits
that get put into the executable at link-time, such as timezone data,
aren't officially part of the executable image, i.e. we don't want the
operating system to load that part.

The original decision to put the linked zip files into the .data section
was mostly made so that when the executable was run in its .com.dbg form
it would still have the zip entries be accessible, even though there was
tons of GNU debug data following the central directory. We're not going
to be able to do that. The .com executable should be the canonical
executable. We have really good tools for automatically attaching and
configuring GDB correctly with debug symbols even when the .com is run.
We'll have to rely on those in cases where zip embedding is used.

See #53
See #54
See #68

[Keithcat1]

It turns out I am having trouble actually running the resulting binaries. They build find, but hello.com.dbg runs fine, printing 'Hi!' as expected, but if I run hello.com from Windows I get: The C:\py\cosmo\hello.com application cannot be run in Win32 mode.
If I delete hello.com, run objcopy again and run ./hello.com from linux, I get: ./hello.com: Invalid argument
Just to clarify, I am using version 0.2 and this is a different problem than the linker error. My build commands are as follows: clang -g -Os -static -nostdlib -nostdinc -fno-pie -mno-red-zone -fno-omit-frame-pointer -o hello.com.dbg hello.c -Wl,-T,ape.lds -include cosmopolitan.h --target=x86_64-pc-linux-gnu crt.o ape.o cosmopolitan.a -Wl,-z,max-page-size=0x1000 -Wl,--gc-sections -fuse-ld=lld llvm-objcopy -S -O binary hello.com.dbg hello.com

[Keithcat1]

Apparently the secret is to build Cosmopolitan with LLVM. It works now.

[jart]

Thanks for bisecting that! You've added a lot of clarity. It kind of works for me, but not Windows. It seems to have something to do with the .zip section that was added to the linker script. We do kind of a weird thing, where we put the zip files in-between .data and .bss and we don't define them in the program headers since we don't want it to be loaded by the operating system. No one's probably done that before. It bewilders me why LLD is behaving so differently on Windows.

[Keithcat1]

Why would it be a windows thing? We're targeting Linux, and I think I also got the same relocation linker error when trying to compile Cosmopolitan 1.0 from within WSL.

On 7/5/21, Justine Tunney @.***> wrote:

Thanks for bisecting that! You've added a lot of clarity. It kind of works for me, but not Windows. It seems to have something to do with the .zip section that was added to the linker script. We do kind of a weird thing, where we put the zip files in-between .data and .bss and we don't define them in the program headers since we don't want it to be loaded by the operating system. No one's probably done that before. It bewilders me why LLD is behaving so differently on Windows.

-- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: https://github.com/jart/cosmopolitan/issues/180#issuecomment-874415901

[Keithcat1]

I went and built a hello world example with Clang, using the exact same invocation: clang -g -Ofast -static -nostdlib -nostdinc -fno-pie -mno-red-zone -fno-omit-frame-pointer %* --include cosmopolitan.h -Wl,-T,ape.lds -include cosmopolitan.h --target=x86_64-pc-linux-gnu crt.o ape.o cosmopolitan.a -Wl,-z,max-page-size=0x1000 -Wl,--gc-sections -fuse-ld=lld -v I added the -v flag to print out all the command-line options that are being passed to the clang-driver. I attached the results for Windows and Linux because it's possible that making Clang a windows > linux cross-compiler might require setting more options than --target. windows.txt wsl.txt

[Keithcat1]

@jart You could try raising an issue over at [https://bugs.launchpad.net/bugs/bugtrackers/llvm-bugs](The LLVM bug tracker) with all the r/relevant info, I don't understand what the problem is.

[kassane]

Cosmopolitan: 2.2

Current error using LLD

$>  clang -g -Os -static -nostdlib -nostdinc -fno-pie -mno-red-zone -fno-omit-frame-pointer -o hello.com.dbg hello.c -Wl,-T,libcosmo/ape.lds -include libcosmo/cosmopolitan.h libcosmo/crt.o libcosmo/ape.o libcosmo/cosmopolitan.a -Wl,-z,max-page-size=0x1000 -Wl,--gc-sections -fuse-ld=lld -v
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib/llvm-14/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -static-define -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -disable-red-zone -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/kassane/rust-ape-example -nostdsysteminc -nobuiltininc -resource-dir /usr/lib/llvm-14/lib/clang/14.0.0 -include libcosmo/cosmopolitan.h -Os -fdebug-compilation-dir=/home/kassane/rust-ape-example -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -vectorize-loops -vectorize-slp -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/hello-314d15.o -x c hello.c
clang -cc1 version 14.0.0 based upon LLVM 14.0.0 default target x86_64-pc-linux-gnu
#include "..." search starts here:
End of search list.
 "/usr/bin/ld.lld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -static -o hello.com.dbg -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/llvm-14/bin/../lib -L/lib -L/usr/lib /tmp/hello-314d15.o -T libcosmo/ape.lds libcosmo/crt.o libcosmo/ape.o libcosmo/cosmopolitan.a -z max-page-size=0x1000 --gc-sections
ld.lld: error: libcosmo/cosmopolitan.a: could not get the buffer for the member defining symbol __fmt_dtoa: truncated or malformed archive (long name offset characters after the '/' are not all decimal numbers: '10462\000' for archive member header at offset 6920446)
clang: error: linker command failed with exit code 1 (use -v to see invocation)

[Ev1lT3rm1nal]

Cosmopolitan: 2.2

Current error using LLD

$>  clang -g -Os -static -nostdlib -nostdinc -fno-pie -mno-red-zone -fno-omit-frame-pointer -o hello.com.dbg hello.c -Wl,-T,libcosmo/ape.lds -include libcosmo/cosmopolitan.h libcosmo/crt.o libcosmo/ape.o libcosmo/cosmopolitan.a -Wl,-z,max-page-size=0x1000 -Wl,--gc-sections -fuse-ld=lld -v
Ubuntu clang version 14.0.0-1ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/11
Found candidate GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
Selected GCC installation: /usr/bin/../lib/gcc/x86_64-linux-gnu/12
Candidate multilib: .;@m64
Selected multilib: .;@m64
 "/usr/lib/llvm-14/bin/clang" -cc1 -triple x86_64-pc-linux-gnu -emit-obj --mrelax-relocations -disable-free -clear-ast-before-backend -disable-llvm-verifier -discard-value-names -main-file-name hello.c -static-define -mrelocation-model static -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -disable-red-zone -tune-cpu generic -mllvm -treat-scalable-fixed-error-as-warning -debug-info-kind=constructor -dwarf-version=5 -debugger-tuning=gdb -v -fcoverage-compilation-dir=/home/kassane/rust-ape-example -nostdsysteminc -nobuiltininc -resource-dir /usr/lib/llvm-14/lib/clang/14.0.0 -include libcosmo/cosmopolitan.h -Os -fdebug-compilation-dir=/home/kassane/rust-ape-example -ferror-limit 19 -fgnuc-version=4.2.1 -fcolor-diagnostics -vectorize-loops -vectorize-slp -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -o /tmp/hello-314d15.o -x c hello.c
clang -cc1 version 14.0.0 based upon LLVM 14.0.0 default target x86_64-pc-linux-gnu
#include "..." search starts here:
End of search list.
 "/usr/bin/ld.lld" -z relro --hash-style=gnu --build-id --eh-frame-hdr -m elf_x86_64 -static -o hello.com.dbg -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12 -L/usr/bin/../lib/gcc/x86_64-linux-gnu/12/../../../../lib64 -L/lib/x86_64-linux-gnu -L/lib/../lib64 -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib64 -L/usr/lib/llvm-14/bin/../lib -L/lib -L/usr/lib /tmp/hello-314d15.o -T libcosmo/ape.lds libcosmo/crt.o libcosmo/ape.o libcosmo/cosmopolitan.a -z max-page-size=0x1000 --gc-sections
ld.lld: error: libcosmo/cosmopolitan.a: could not get the buffer for the member defining symbol __fmt_dtoa: truncated or malformed archive (long name offset characters after the '/' are not all decimal numbers: '10462\000' for archive member header at offset 6920446)
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Any update on how to resolve this? @jart

[theoparis]

I'm getting a similar error with llvm 17 on debian 😕

ld.lld: error: /home/theo/dev/wasmbruh/deps/toywasm/../cosmopolitan/cosmopolitan.a: could not get the buffer for a child of the archive: truncated or malformed archive (long name offset characters after the '/' are not all decimal numbers: '0\000' for archive member header at offset 615134)

[theoparis]

The libc almost builds with LLVM 18 built from git and make -j12 m=llvm o/llvm/libc now. However there are a lot of errors related to warnings such as -Walloca-larger-than=4096. I also got this:

libc/calls/sig.c:444:11: error: address argument to atomic operation must be a pointer to _Atomic type ('uint64_t *' (aka 'unsigned long *') invalid)
  444 |         !(atomic_load_explicit(&pt->pt_blkmask, memory_order_relaxed) &

Casting it to a atomic_int * lets the build continue until it gets to a

libc/intrin/ffs.c:24:5: error: definition of builtin function '__builtin_ffs'

Removing those definitions then gives:

libc/intrin/mman.greg.c:321:33: error: invalid operand for inline asm constraint 'i'
  321 |                      .p_vaddr = ABS64(ape_stack_vaddr),

[Ev1lT3rm1nal]

Maybe you should try to un-archive it and archive it again with llvm-ar

[theoparis]

The libc almost builds with LLVM 18 built from git and make -j12 m=llvm o/llvm/libc now. However there are a lot of errors related to warnings such as -Walloca-larger-than=4096. I also got this:

libc/calls/sig.c:444:11: error: address argument to atomic operation must be a pointer to _Atomic type ('uint64_t *' (aka 'unsigned long *') invalid)
  444 |         !(atomic_load_explicit(&pt->pt_blkmask, memory_order_relaxed) &

Casting it to a atomic_int * lets the build continue until it gets to a

libc/intrin/ffs.c:24:5: error: definition of builtin function '__builtin_ffs'

Removing those definitions then gives:

libc/intrin/mman.greg.c:321:33: error: invalid operand for inline asm constraint 'i'
  321 |                      .p_vaddr = ABS64(ape_stack_vaddr),

The first error I got when trying again now was this:

make: *** [build/rules.mk:52: o/llvm/libc/calls/virtualmax.o] Error 1
error: unknown warning option '-Wno-prio-ctor-dtor'; did you mean '-Wno-reorder-ctor'? [-Werror,-Wunknown-warning-option]

`make MODE=llvm -j32 o/llvm/libc/calls/getntsyspath.o` exited with 1: