deislabs / mystikos

Tools and runtime for launching unmodified container images in Trusted Execution Environments
143 stars 49 forks source link

Interactive bash support issues #1423

Open vtikoo opened 2 years ago

vtikoo commented 2 years ago

vitikoo@vitikoo-icelake:/mnt/code/mystikos$ mkdir scratch
vitikoo@vitikoo-icelake:/mnt/code/mystikos$ cd scratch/
vitikoo@vitikoo-icelake:/mnt/code/mystikos/scratch$ ../scripts/appbuilder -i ubuntu
<logs-elided>
vitikoo@vitikoo-icelake:/mnt/code/mystikos/scratch$ ../build/bin/myst mkext2 appdir rootfs
vitikoo@vitikoo-icelake:/mnt/code/mystikos/scratch$ ../build/bin/myst exec rootfs /bin/bash
mystikos: warn: exec.c(1149): myst_exec(): 
    The thread stack size may be too small for the given program interpreter
    (link loader), which could result in stack overflows. Consider changing
    the thread stack size to at least 1048576 bytes, using the --thread-stack-size
    option or the ThreadStackSize configuration setting.
    [interpreter=/lib64/ld-linux-x86-64.so.2]
    [program=/bin/bash]

bash: cannot set terminal process group (-1): Not supported
bash: no job control in this shell
bash: fork: Not supported
root@TEE:/# *** kernel panic: syscall.c(7703): _syscall(): unhandled syscall: SYS_pselect6()
0x1007f4fd2: __myst_panic()
0x1008075a0: _syscall()
0x10082bb02: __morestack()
*** Kernel segmentation fault 
0x1007bcd82: myst_signal_process()
0x1008075ac: _syscall()
0x10082bb02: __morestack()
0x10080773e: myst_syscall()
0x104938b0c: <unknown address>
0x10499edc0: <unknown address>
0x10499ede8: <unknown address>
0x1046c2288: <unknown address> ```
isil-oz commented 2 years ago

@vtikoo If we have the pselect6 implemented in mystikos, what is the expected behavior? We have the implementation of select which is similar to pselect, I thought we can have it in a similar way. As a first attempt, when I duplicate the select syscall implementation for pselect6 I have the following output:

isil@isil-HP-ENVY:~/Documents/Code/mystikos/scratch$ ../build/bin/myst exec-linux --thread-stack-size 1048575 rootfs /bin/bash mystikos: info: enter.c(805): myst_enter_kernel(): Entered Mystikos kernel. mystikos: info: exec.c(1241): myst_exec(): Entering CRT. bash: cannot set terminal process group (-1): Not supported bash: no job control in this shell bash: fork: Not supported root@TEE:/#

vtikoo commented 2 years ago

Thanks for looking into this @isil-oz . The job control and terminal process group errors are due to ioctl requests which the current console device implementation(kernel/ttydev.c) does not support. You can view the ioctl system calls made by bash by adding --strace-filter=SYS_ioctl to the command line. I didn't look too closely, but failing ioctl's which seem relevant are get/set process group requests: TIOCGPGRP(0x540f), TIOCSPGRP(0x5410).

Why the process is exiting after printing the prompt is not clear to me as well. I put a breakpoint on the exit group syscall handler function _SYS_exit_group. Frame #9 and 10 are from bash source code. Also I am using the --unhandled-syscall-enosys=true option to avoid the unhandled syscall segfault. With this option SYS_pselect6 returns an ENOSYS return code.

../build/bin/myst-gdb --args ../build/bin/myst exec rootfs /bin/bash \
--unhandled-syscall-enosys=true \ 
--thread-stack-size=1M
...
(gdb) bt
#0  _SYS_exit_group (n=n@entry=231, thread=0x100467a00 <_main_thread>, process=process@entry=0x104611010, args=<optimized out>, params=<optimized out>) at syscall.c:4305
#1  0x000000010041c118 in _syscall (args_=0x10434fc40) at syscall.c:6887
#2  0x0000000100441fcd in __morestack () at ../asm/callonstack.s:42
#3  0x00000001004163d1 in myst_syscall (n=231, params=0x10434fd40) at /home/vitikoo/code/mystikos/include/myst/kstack.h:38
#4  0x000000010436a2f5 in myst_syscall (n=231, params=0x10434fd40) at enter.c:111
#5  0x000000010436a10e in __syscall1 (a1=254, n=231) at /home/vitikoo/code/mystikos/third_party/musl/crt/musl/arch/x86_64/syscall_arch.h:16
#6  _Exit (ec=254) at exit.c:98
#7  0x000000010437e9ca in do_crt_exit (code=code@entry=254) at src/exit/exit.c:33
#8  0x000000010436a217 in exit (code=254) at exit.c:80
#9  0x000000010409d479 in exit_shell ()
#10 0x000000010409bc14 in main ()
#11 0x000000010437dd50 in libc_start_main_stage2 (main=0x10409a340 <main>, argc=1, argv=0x104350010) at src/env/__libc_start_main.c:94
#12 0x000000010409bed5 in _start ()
isil-oz commented 2 years ago

Thanks for guidance @vtikoo. TIOCGPGRP gets, and TIOCSPGRP sets the foreground process group ID of the terminal. We have the myst_process_self() inside mystikos, is it the foreground process?

Additionally, there are other ioctl requests/commands not supported: TCGETS (0x5401), TCSETSW(0x5403), TIOCSWINSZ (0x5414)