fork-test failed after xcode 11 (osx)

[thehajime]

all 11.{0,1,2,3} versions have this issue.

% RUMP_VERBOSE=1 ./rumpobj/tests/fork-test 
[    0.000000] Linux version 4.19.0+ (tazaki@mars.local) (Apple clang version 11.0.0 (clang-1100.0.33.16)) #1 Thu Jan 16 09:11:11 JST 2020
[    0.000000] bootmem address range: 0x100aee000 - 0x106eed000
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 25249
[    0.000000] Kernel command line: mem=100M virtio-pci.force_legacy=1
[    0.000000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes)
[    0.000000] Memory available: 100592k/102396k RAM
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 4096
[    0.000000] lkl: irqs initialized
[    0.000000] clocksource: lkl: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.000001] lkl: time and timers initialized (irq1)
[    0.000004] pid_max: default: 4096 minimum: 301
[    0.000034] Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
[    0.000037] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
[    0.004023] console [lkl_console0] enabled
[    0.004079] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.004176] futex hash table entries: 16 (order: -4, 384 bytes)
[    0.004310] random: get_random_u32 called from 0x100237eae with crng_init=0
[    0.004408] NET: Registered protocol family 16
[    0.004900] vgaarb: loaded
[    0.005061] clocksource: Switched to clocksource lkl
[    0.005213] NET: Registered protocol family 2
[    0.005462] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes)
[    0.005545] TCP established hash table entries: 1024 (order: 1, 8192 bytes)
[    0.005645] TCP bind hash table entries: 1024 (order: 1, 8192 bytes)
[    0.005710] TCP: Hash tables configured (established 1024 bind 1024)
[    0.005821] UDP hash table entries: 128 (order: 0, 4096 bytes)
[    0.005883] UDP-Lite hash table entries: 128 (order: 0, 4096 bytes)
[    0.006141] workingset: timestamp_bits=62 max_order=15 bucket_order=0
[    0.007307] SGI XFS with ACLs, security attributes, no debug enabled
[    0.007805] 9p: Installing v9fs 9p2000 file system support
[    0.008444] jitterentropy: Initialization failed with host not compliant with requirements: 2
[    0.008533] io scheduler noop registered
[    0.008570] io scheduler deadline registered
[    0.008666] io scheduler cfq registered (default)
[    0.008714] io scheduler mq-deadline registered
[    0.008756] io scheduler kyber registered
[    0.009072] NET: Registered protocol family 10
[    0.009340] Segment Routing with IPv6
[    0.009401] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.009572] 9pnet: Installing 9P2000 support
[    0.009652] PCI: root bus 00: using default resources
[    0.009721] PCI host bridge to bus 0000:00
[    0.009763] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.009821] pci_bus 0000:00: root bus resource [mem 0x00000000-0xffffffffffffffff]
[    0.009893] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.010020] This architecture does not have kernel memory protection.
[    0.010088] Run /init as init process
[    0.010157] FDPIC ____ LOAD 1 ____
rumpuser started.
parent pid=1, ppid=0
zsh: segmentation fault (core dumped)  RUMP_VERBOSE=1 ./rumpobj/tests/fork-test

[retrage]

The generated code unexpectedly uses r14 register. This is the root of the cause. The below is lldb log.

[    0.010887] FDPIC ____ LOAD 1 ____
rumpuser started.
parent pid=1, ppid=0
Process 62194 stopped
* thread #1, stop reason = EXC_BAD_ACCESS (code=1, address=0xfd)
    frame #0: 0x000000010000cfc4 fork-test`thread_bootstrap.cold.1 + 148
fork-test`thread_bootstrap.cold.1:
->  0x10000cfc4 <+148>: movq   0x8(%r14), %rax
    0x10000cfc8 <+152>: movq   0x160(%rax), %r13
    0x10000cfcf <+159>: movq   0x8(%r14), %rax
    0x10000cfd3 <+163>: movq   0x168(%rax), %r12

Here is the workaround.

diff --git a/arch/lkl/kernel/threads.c b/arch/lkl/kernel/threads.c
index 7688e62e7b24..2aff31d68370 100644
--- a/arch/lkl/kernel/threads.c
+++ b/arch/lkl/kernel/threads.c
@@ -169,7 +169,7 @@ void inline lkl_restore_register(struct task_struct *task)
        asm("mov %0, %%rsp" :: "m"(newrsp));

        RESTORE_REG(r15);
-       RESTORE_REG(r14);
+//     RESTORE_REG(r14);
        RESTORE_REG(r13);
        RESTORE_REG(r12);
 //     RESTORE_REG(bx);

[retrage]

3026e15 will fix this issue.