Help on compiling for iOS

holzschu commented 1 year ago

Hi, I've been trying to cross-compile blink for iOS. Part of the problem is that, from the compiler viewpoint, it is very difficult to separate iOS from a M1 Mac: both have __APPLE__ and __aarch64__ defined.

(There are other issues, such as /tmp not being accessible, the need to redirect standard output, and the fact that many files don't have the X-bit set even though they're executable, but I'm used to these by now).

So far, my efforts have resulted into:

compile with the JIT enabled: it compiles, but it breaks when loading an executable.

without the -m flag when launching blink:

E2023-08-14T18:05:14.583468:blink/memorymalloc.c:913:8640 mmap(0x400000[0x88800400000], 0x4000) -> 0xfffff77909274000[0x109274000] crisis: requested memory overlapped blink image or system memory. try using `blink -m` to disable memory optimizations, or try compiling blink using -Wl,--image-base=0x23000000 or possibly -Wl,-Ttext-segment=0x23000000 in LDFLAGS
unrecoverable mmap() crisis: see log for further details

with the -m flag: it crashes the entire app (that's expected) and blink.log contains the following:

I2023-08-14T18:07:17.450723:blink/random.c:88:8640 generating weak random data
E2023-08-14T18:07:17.454305:blink/blink.c:174:8640 terminating due to SIGSEGV (rip=0x400078 code=2 faultaddr=0x400078)
I2023-08-14T18:07:17.454714:blink/diself.c:115:8640 could not load elf string table
E2023-08-14T18:07:17.454898:blink/blink.c:158:8640 additional information
         PC 400078 push $1 6a 01 5f 48 8d 35 11 00
         AX 0000000000000000  CX 0000000000000000  DX 0000000000000000  BX 0000000000000000
         SP 00004fffffffea50  BP 0000000000000000  SI 0000000000000000  DI 0000000000000000
         R8 0000000000000000  R9 0000000000000000 R10 0000000000000000 R11 0000000000000000
        R12 0000000000000000 R13 0000000000000000 R14 0000000000000000 R15 0000000000000000
         FS 0000000000000

compile without the JIT enabled (which is probably a good idea since the AppStore rules sort of forbid JIT, although there are some apps that still provide it): compilation fails with the following error:
```
Undefined symbols for architecture arm64:
"_FastCall", referenced from:
  _OpCallJvds in blink.a(stack.o)
"_FastLeave", referenced from:
  _OpLeave in blink.a(stack.o)
"_Jitter", referenced from:
  _OpLeave in blink.a(stack.o)
ld: symbol(s) not found for architecture arm64
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
Compiling for OSX with --disable-jit does work, it's just with iOS that I have this weird issue. I don't understand what is happening; I'd appreciate if anyone could point me in the right direction.

GNUfan42 commented 3 weeks ago

That's weird, in theory it's more likely to pass compilation if JIT is disabled.

BTW it's 2024 now, have you got around this issue?

jart commented 3 weeks ago

Try changing this code in machine.h so that it always returns false.

#if defined(NOLINEAR) || defined(__SANITIZE_THREAD__) || \
    defined(__CYGWIN__) || defined(__NetBSD__) || defined(__COSMOPOLITAN__)
#define CanHaveLinearMemory() false
#else
#define CanHaveLinearMemory() CAN_64BIT
#endif

holzschu commented 3 weeks ago

Thanks for the reply! The current version compiles for iOS with --disable-jit. I've added -DNOLINEAR to the C flags, and now blink is running on iOS (at least with third_party/cosmo/goodhello.elf). I'm going to restart work on this project.

holzschu commented 3 weeks ago

I spoke too fast. blink goodhello.elf does work, but blink tinyhello.elf crashes, with this error message.

[~/Documents/blink.git/third_party/cosmo]$ blink tinyhello.elf 
E2024-08-26T15:35:58.261726:blink/blink.c:174:1929 terminating due to SIGSEGV (rip=0x400078 code=2 faultaddr=0x400078)
E2024-08-26T15:35:58.262525:blink/blink.c:158:1929 additional information
         PC 400078 push $1 6a 01 5f 48 8d 35 11 00
         AX [0000000000000000](tel:0000000000000000)  CX [0000000000000000](tel:0000000000000000)  DX 00004ffffffffff2  BX [0000000000000000](tel:0000000000000000)
         SP 00004fffffffe880  BP [0000000000000000](tel:0000000000000000)  SI [0000000000000000](tel:0000000000000000)  DI [0000000000000000](tel:0000000000000000)
         R8 [0000000000000000](tel:0000000000000000)  R9 [0000000000000000](tel:0000000000000000) R10 [0000000000000000](tel:0000000000000000) R11 [0000000000000000](tel:0000000000000000)
        R12 [0000000000000000](tel:0000000000000000) R13 [0000000000000000](tel:0000000000000000) R14 [0000000000000000](tel:0000000000000000) R15 [0000000000000000](tel:0000000000000000)
         FS 0000000000000...
interps                          = 1
tlb_hits                         = 76
tlb_misses                       = 3
[~/Documents/blink.git/third_party/cosmo]$ cat blink.log 
I2024-08-26T15:50:04.500575:blink/random.c:88:1966 generating weak random data
E2024-08-26T15:50:04.504094:blink/blink.c:174:1966 terminating due to SIGSEGV (rip=0x400078 code=2 faultaddr=0x400078)
I2024-08-26T15:50:04.504464:blink/diself.c:115:1966 could not load elf string table
E2024-08-26T15:50:04.504858:blink/blink.c:158:1966 additional information
         PC 400078 push $1 6a 01 5f 48 8d 35 11 00
         AX [0000000000000000](tel:0000000000000000)  CX [0000000000000000](tel:0000000000000000)  DX 00004ffffffffff2  BX [0000000000000000](tel:0000000000000000)
         SP 00004fffffffe880  BP [0000000000000000](tel:0000000000000000)  SI [0000000000000000](tel:0000000000000000)  DI [0000000000000000](tel:0000000000000000)
         R8 [0000000000000000](tel:0000000000000000)  R9 [0000000000000000](tel:0000000000000000) R10 [0000000000000000](tel:0000000000000000) R11 [0000000000000000](tel:0000000000000000)
        R12 [0000000000000000](tel:0000000000000000) R13 [0000000000000000](tel:0000000000000000) R14 [0000000000000000](tel:0000000000000000) R15 [0000000000000000](tel:0000000000000000)
         FS 0000000000000...
[~/Documents/blink.git/third_party/cosmo]$

jart commented 3 weeks ago

That's the first instruction of your program. Something inside Blink is possibly crashing. Try to install libunwind and get blink to depend on libunwind, so that when it crashes, it can print a backtrace into blink. Maybe add printf() statements to loader.c and argv.c. It's probably crashing around there. I'm particularly interested in knowing what number blink detected for FLAG_vabits. See GetBitsInAddressSpace().

holzschu commented 3 weeks ago

Thanks for the help. I'm going to investigate where you said. FLAG_vabits is equal to 39.

jart commented 2 weeks ago

You're making sure that the linear memory optimization is disabled, correct?

holzschu commented 2 weeks ago

I've been compiling with -DNOLINEAR, which deactivates it. Just to be sure, I've re-compiled with #define CanHaveLinearMemory() false, I got the same behaviour.

holzschu commented 2 weeks ago

So far, I've tracked it to LookupAddress2, when called from LoadInstruction2 https://github.com/jart/blink/blob/222064a6c325248ccbe5f61f7359dd91ebc29bac/blink/instruction.c#L89 The first entry read from memory when executing is 0x6000010398cc05 for goodhello.elf, and 0x8060000120024c07 for tinyhello.elf. Since mask (PAGE_XD) is 0x8000000000000000, that makes this test in LookupAddress2 fail (need is 0): https://github.com/jart/blink/blob/222064a6c325248ccbe5f61f7359dd91ebc29bac/blink/memory.c#L279

I'll keep investigating.

holzschu commented 1 week ago

Continuing my investigation: the first difference between goodhello.elf (which works on iOS) and tinyhello.elf (which doesn't work on iOS) is this line: https://github.com/jart/blink/blob/222064a6c325248ccbe5f61f7359dd91ebc29bac/blink/loader.c#L215 which is just before this warning: https://github.com/jart/blink/blob/222064a6c325248ccbe5f61f7359dd91ebc29bac/blink/loader.c#L216 So we use LoaderCopy() to load the binary content instead of what is used for goodhello.elf.

LoaderCopy() writes down: copy 40009f-401000 from 0-f61, while before we had load 400000-401000 from 0-1000.

So (tentative hypothesis) it would be the copy operator that has an issue with iOS. I'll keep investigating.

holzschu commented 1 week ago

The first entry read from memory when executing is 0x6000010398cc05 for goodhello.elf, and 0x8060000120024c07 for tinyhello.elf. Since mask (PAGE_XD) is 0x8000000000000000, that makes this test in LookupAddress2 fail (need is 0):

If I change the value of PAGE_XD, say to 0xa000000000000000, then the value read for tinyhello.elf becomes 0xa060000120024c07. So at some point, the loading code has disabled the "memory executable" bit. I need to find out where that happened.

This is probably connected to a specificity of iOS: there are several ways to check if a file is executable in Unix. On iOS, some of these work (they check the executable bit), while other methods will always return false. Which makes sense from the system point of view, since files are not supposed to be executed.

holzschu commented 1 week ago

If I replace this protection in a call to ProtectVirtual() in LoaderCopy(): https://github.com/jart/blink/blob/222064a6c325248ccbe5f61f7359dd91ebc29bac/blink/loader.c#L72 with:

        PROT_READ | PROT_WRITE | PROT_EXEC , false));

then blink can run tinyhello.elf on iOS. I don't fully understand why it's only required on iOS, or whether it is going to cause bugs further down, but I'll take it.

jart / blink

Help on compiling for iOS #151