File creation fails in binary-compatibility mode for `9P2000.L`

Describe the bug

Using open() or openat() to create a file under binary-compatibility mode (i.e. using the O_CREAT flag) fails when using 9PFS. This is caused by PR unikraft/unikraft#914, that changes the 9P version from 2000.U to 2000.L.

This doesn't happen under native mode, by invoking the open() call (see details below).

My investigation led me to locating the issue at line 216 in lib/uk9p/9preq.c in the uk_9preq_error() function. The call trace is:

Breakpoint 4, uk_9preq_error (req=req@entry=0x47f96a020) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9preq.c:223
223     UK_BUGON(req->recv.offset != UK_9P_HEADER_SIZE);
(gdb) bt
#0  uk_9preq_error (req=req@entry=0x47f96a020) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9preq.c:223
unikraft/unikraft#1  0x000000000014a720 in uk_9preq_waitreply (req=req@entry=0x47f96a020) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9preq.c:203
unikraft/unikraft#2  0x000000000014cbb2 in send_and_wait_zc (dev=0x47f900020, zc_dir=UK_9PREQ_ZCDIR_NONE, zc_buf=0x0, zc_size=0, zc_offset=0, req=0x47f96a020) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9p.c:62
unikraft/unikraft#3  send_and_wait_zc (zc_offset=0, zc_size=0, zc_buf=0x0, zc_dir=UK_9PREQ_ZCDIR_NONE, req=0x47f96a020, dev=0x47f900020) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9p.c:48
unikraft/unikraft#4  send_and_wait_no_zc (req=0x47f96a020, dev=0x47f900020) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9p.c:72
unikraft/unikraft#5  uk_9p_lcreate (dev=0x47f900020, fid=0x47f911020, name=name@entry=0x40005e9d1 "a.txt", flags=flags@entry=513, mode=mode@entry=33152, gid=gid@entry=0) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/uk9p/9p.c:378
unikraft/unikraft#6  0x0000000000119851 in uk_9pfs_create (dvp=<optimized out>, name=0x40005e9d1 "a.txt", mode=33152) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/9pfs/9pfs_vnops.c:397
unikraft/unikraft#7  0x000000000016d371 in sys_open (path=path@entry=0x40005e9d0 "/a.txt", flags=67, flags@entry=66, mode=<optimized out>, mode@entry=384, fpp=fpp@entry=0x40005e9c8) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/vfscore/syscalls.c:138
#8  0x000000000016f7d9 in __uk_syscall_r_open (pathname=pathname@entry=0x400602000 "a.txt", flags=flags@entry=66, mode=mode@entry=384) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/vfscore/main.c:143
unikraft/unikraft#9  0x0000000000171142 in uk_syscall_r_open (pathname=17186168832, flags=66, mode=384) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/vfscore/main.c:115
unikraft/unikraft#10 0x0000000000140152 in uk_syscall6_r (nr=<optimized out>, arg1=<optimized out>, arg2=<optimized out>, arg3=<optimized out>, arg4=<optimized out>, arg5=<optimized out>, arg6=<optimized out>) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/apps/app-elfloader/build/libsyscall_shim/uk_syscall6_r.c:981
unikraft/unikraft#11 0x0000000000143370 in ukplat_syscall_handler (r=0x40005fdd0) at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/lib/syscall_shim/uk_syscall_binary.c:94
unikraft/unikraft#12 0x0000000000105548 in _ukplat_syscall () at /media/razvan/c4f6765a-efa5-4ebd-9cf0-7da9908a0189/razvan/unikraft/scripts/workdir/unikraft/plat/common/x86/syscall.S:93
unikraft/unikraft#13 0x00000004004011d1 in _start ()
unikraft/unikraft#14 0x0000000000000001 in ?? ()
unikraft/unikraft#15 0x000000047f90e021 in ?? ()
unikraft/unikraft#16 0x0000000000000000 in ?? ()

(gdb) p/d req->recv.type
$2 = 7

The req->recv.type is 7 (UK_9P_LRERROR) causing the comparison in line 216 to fail.

Steps to reproduce

open-static-pie.zip consists of the source code files and prebuilt PIE binaries that can be used with run-app-elfloader. Use the run.sh script:

$ unzip open-static-pie.zip

$ ./run.sh -r open-static-pie/ /open_asm

$ ./run.sh -r open-static-pie/ /open_static_c

$ ./run.sh -r open-static-pie/ /open_dyamic_c

You would then capture an error message:

open("a.txt", O_RDONLY|O_CREAT|0x2) = Operation not permitted (-1)

Note that using open.c in a native Unikraft build works. You have to enable 9PFS + VFSCore support in the native build to make it run; Musl is not required.

Expected behavior

The open() or openat() system calls should succeed. They would return a file descriptor:

open("a.txt", O_RDONLY|O_CREAT|0x2) = fd:3

Which architectures were you using or does this bug affect?

x86_64

Which platforms were you using or does this bug affect?

kvm

Relevant log output

[    1.096561] dbg:  [libsyscall_shim] <uk_syscall_binary.c @   89> Binary system call request "open" (2) at ip:0x4004011d1 (arg0=0x400602000, arg1=0x42, ...)
[    1.105242] dbg:  [libvfscore] <main.c @  116> (int) uk_syscall_r_open((const char*) 0x400602000, (int) 0x42, (mode_t) 0x180)
[    1.110643] dbg:  [libuk9p] <9p.c @  223> TWALK fid 0 newfid 2 nwname 1 name a.txt
[    1.114167] dbg:  [libkvmvirtio] <virtio_ring.c @  368> Old head:0, new head:2, total_desc:2
[    1.118859] dbg:  [libkvmvirtio9p] <virtio_9p.c @  285> notify queue 0
[    1.123464] dbg:  [libuk9p] <9preq.c @  236> RLERROR 2
[    1.126771] dbg:  [libuk9p] <9p.c @  223> TWALK fid 0 newfid 2 nwname 0 name <NULL>
[    1.131217] dbg:  [libkvmvirtio] <virtio_ring.c @  368> Old head:0, new head:2, total_desc:2
[    1.136670] dbg:  [libkvmvirtio9p] <virtio_9p.c @  285> notify queue 0
[    1.140195] dbg:  [libuk9p] <9p.c @  244> RWALK nwqid 0
[    1.142817] dbg:  [libuk9p] <9p.c @  371> TLCREATE fid 2 mode 33152
[    1.145823] dbg:  [libkvmvirtio] <virtio_ring.c @  368> Old head:0, new head:2, total_desc:2
[    1.150348] dbg:  [libkvmvirtio9p] <virtio_9p.c @  285> notify queue 0
[    1.153947] dbg:  [libuk9p] <9preq.c @  236> RLERROR 1
open("a.txt", O_RDONLY|O_CREAT|0x2) = Operation not permitted (-1)

unikraft / run-app-elfloader