search

QuarkContainer / Quark

A secure container runtime with CRI/OCI interface
Apache License 2.0
318 stars 47 forks source link

Different Flags in Different Arch #1323

Closed mhomidi closed 3 months ago

mhomidi commented 3 months ago

Hi,

I encountered an issue when trying to run Nginx with Quark on my ARM machine. The problem comes from the flags we use in our syscalls (these flags are defined in qlib/linux_def.rs:2260). After searching online, I figure out that some of these flags differ between ARM and x86 architectures. Here is the list I found:

ARM:

O_DIRECT: 0x10000
O_DIRECTORY: 0x4000
O_NOFOLLOW: 0x8000
O_TMPFILE: 0x404000

X86:

O_DIRECT: 0x4000
O_DIRECTORY: 0x10000
O_NOFOLLOW: 0x20000
O_TMPFILE: 0x410000

I believe we need to address this discrepancy.

QuarkContainer commented 3 months ago

@mhomidi I put a fix in https://github.com/QuarkContainer/Quark/pull/1319's 4th commit. Would you please have a try?

mhomidi commented 3 months ago

Thanks! it works.

mhomidi commented 3 months ago

Hello @shrik3,

I still have some issue on some testcases in the performance tests on ARM.

First, we have an issue when we want to run mariadb. Somewhere in the application, it calls syscall 435 which is NotImplementSyscall in ARM. I changed this to SysNoSys. Application runs but in somewhere it encounter /usr/local/bin/docker-entrypoint.sh: line 214: /proc/self/cgroup: No such file or directory. I have to check it. Probably it is similar to problem we have with nginx.

Secondly and the main one, it seems that signal handing on ARM has some issues. When I run Redis, the error log is this (last lines):

[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 2: call id sys_fcntl 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_clock_gettime arg0:1
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_clock_gettime 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_rt_sigprocmask arg0:2
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_rt_sigprocmask 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_clone arg0:50f00
[INFO] [1/45400c0000|0] Clone opts is CloneOptions { sharingOption: SharingOptions { NewAddressSpace: false, NewSignalHandlers: false, NewThreadGroup: false, TerminationSignal: Signal(0), NewPIDNamespace: false, NewUserNamespace: false, NewNetworkNamespace: false, NewFiles: false, NewFSContext: false, NewUTSNamespace: false, NewIPCNamespace: false }, Stack: 14000040000, SetTLS: false, TLS: 8, ChildClearTID: false, ChildSetTID: false, ChildTID: 8, ParentSetTID: false, ParentTID: 14000040000, Vfork: false, Untraced: false, InheritTracer: false }
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 6: call id sys_clone 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_rt_sigprocmask arg0:2
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_rt_sigprocmask 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_newfstatat arg0:ffffffffffffff9c
[INFO] [1/45400c0000|0] Fstatat path is /proc/self/exe dirPath false, flags & ATType::AT_SYMLINK_NOFOLLOW 0
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_newfstatat 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_getuid arg0:0
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_getuid 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_getgid arg0:0
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_getgid 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_openat arg0:ffffffffffffff9c
[INFO] [1/45400c0000|0] openat path is /etc/passwd, the perm is PermMask { read: true, write: false, execute: false }, current is /data
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 4: call id sys_openat 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_epoll_create1 arg0:80000
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 5: call id sys_epoll_create1 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_pipe2 arg0:14000052ac0
[INFO] [1/45400c0000|0] Pipe2 the fds is [6, 7]
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_pipe2 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_epoll_ctl arg0:5
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_epoll_ctl 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_epoll_ctl arg0:5
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is ffffffffffffffff: call id sys_epoll_ctl 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_openat arg0:ffffffffffffff9c
[INFO] [1/45400c0000|0] openat path is /etc/group, the perm is PermMask { read: true, write: false, execute: false }, current is /data
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 8: call id sys_openat 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_epoll_ctl arg0:5
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is ffffffffffffffff: call id sys_epoll_ctl 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_read arg0:4
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_nanosleep 
[INFO] [1/4540120000|0] (4/1)------get call id sys_clock_gettime arg0:1
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_clock_gettime 
[INFO] [1/4540120000|0] (4/1)------get call id sys_clock_gettime arg0:1
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_clock_gettime 
[INFO] [1/4540120000|0] (4/1)------get call id sys_epoll_pwait arg0:5
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_epoll_pwait 
[INFO] [1/4540120000|0] (4/1)------get call id sys_nanosleep arg0:14000043ef8
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 36c: call id sys_read 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_read arg0:4
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_read 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_read arg0:8
[DEBUG] [1/4540160000|0] entering child task: kernelSp/PtRegs @ 454017fed0
[INFO] [1/4540160000|0] (6/1)------get call id sys_gettid arg0:4d2
[INFO] [1/4540160000|0] (6/1)------Return[0] res is 6: call id sys_gettid 
[INFO] [1/4540160000|0] (6/1)------get call id sys_sigaltstack arg0:0
[INFO] [1/4540160000|0] (6/1)------Return[0] res is 0: call id sys_sigaltstack 
[INFO] [1/4540160000|0] (6/1)------get call id sys_sigaltstack arg0:1400003ff00
[INFO] [1/4540160000|0] (6/1)------Return[0] res is 0: call id sys_sigaltstack 
[INFO] [1/4540160000|0] (6/1)------get call id sys_rt_sigprocmask arg0:2
[INFO] [1/4540160000|0] (6/1)------Return[0] res is 0: call id sys_rt_sigprocmask 
[INFO] [1/4540160000|0] (6/1)------get call id sys_gettid arg0:0
[INFO] [1/4540160000|0] (6/1)------Return[0] res is 6: call id sys_gettid 
[INFO] [1/4540160000|0] (6/1)------get call id sys_futex arg0:2588d8
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_nanosleep 
[INFO] [1/4540120000|0] (4/1)------get call id sys_clock_gettime arg0:1
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_clock_gettime 
[INFO] [1/4540120000|0] (4/1)------get call id sys_clock_gettime arg0:1
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 0: call id sys_clock_gettime 
[INFO] [1/4540120000|0] (4/1)------get call id sys_nanosleep arg0:14000043ef8
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 1bf: call id sys_read 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_read arg0:8
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_read 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_close arg0:8
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_close 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_close arg0:4
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_close 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_setgroups arg0:0
[INFO] [1/45400c0000|0] SetExtraGIDs 1
[INFO] [1/45400c0000|0] SetExtraGIDs 2
[INFO] [1/45400c0000|0] SetExtraGIDs 3
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_setgroups 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_setgid arg0:3e7
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_setgid 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_getpid arg0:14000034800
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 1: call id sys_getpid 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_tgkill arg0:1
[INFO] [1/45400c0000|0] sendsignalLocked, signal:SignalInfo { Signo: 33, Errno: 0, Code: -6 }, group:false
[INFO] [1/45400c0000|0] sendsignalTimerLocked, signal:SignalInfo { Signo: 33, Errno: 0, Code: -6 }
[INFO] [1/45400c0000|0] Thread[6] Notified of signal Signal(33)
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_tgkill 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_getpid arg0:14000034400
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 1: call id sys_getpid 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_tgkill arg0:1
[INFO] [1/45400c0000|0] sendsignalLocked, signal:SignalInfo { Signo: 33, Errno: 0, Code: -6 }, group:false
[INFO] [1/45400c0000|0] sendsignalTimerLocked, signal:SignalInfo { Signo: 33, Errno: 0, Code: -6 }
[INFO] [1/45400c0000|0] Thread[5] Notified of signal Signal(33)
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_tgkill 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_getpid arg0:14000034000
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 1: call id sys_getpid 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_tgkill arg0:1
[INFO] [1/45400c0000|0] sendsignalLocked, signal:SignalInfo { Signo: 33, Errno: 0, Code: -6 }, group:false
[INFO] [1/45400c0000|0] sendsignalTimerLocked, signal:SignalInfo { Signo: 33, Errno: 0, Code: -6 }
[INFO] [1/45400c0000|0] Thread[4] Notified of signal Signal(33)
[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_tgkill 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_sched_yield arg0:0
[INFO] [1/4540120000|0] RunInterrupt[4540120000] ...
[DEBUG] [1/4540120000|0] task_signals: thread deliver signal SigAction { 
handler: 70f40, 
flag : 18000004, 
flags::HasRestorer: false, 
flags::IsOnStack: true, 
flags::IsRestart: true, 
flags::IsResetHandler: false, 
flags::IsNoDefer: false, 
flags::IsSigInfo: true, 
restorer : 0,  
mask: fffffffffffbfeff,  
}
[INFO] [1/4540120000|0] Signal 33: delivering to handler
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 21: call id sys_nanosleep 
[INFO] [1/4540120000|0] (4/1)------get call id sys_write arg0:2
[ERROR] [1/4540120000|0] (Data) Write fail: fatal: bad g in signal handler

[INFO] [1/4540160000|0] RunInterrupt[4540160000] ...
[DEBUG] [1/4540160000|0] task_signals: thread deliver signal SigAction { 
handler: 70f40, 
flag : 18000004, 
flags::HasRestorer: false, 
flags::IsOnStack: true, 
flags::IsRestart: true, 
flags::IsResetHandler: false, 
flags::IsNoDefer: false, 
flags::IsSigInfo: true, 
restorer : 0,  
mask: fffffffffffbfeff,  
}
[INFO] [1/4540160000|0] Signal 33: delivering to handler
[INFO] [1/4540160000|0] (6/1)------Return[0] res is 21: call id sys_futex 
[INFO] [1/4540160000|0] (6/1)------get call id sys_write arg0:2
[ERROR] [1/4540160000|0] (Data) Write fail: fatal: bad g in signal handler

[INFO] [1/4540140000|0] RunInterrupt[4540140000] ...
[DEBUG] [1/4540140000|0] task_signals: thread deliver signal SigAction { 
handler: 70f40, 
flag : 18000004, 
flags::HasRestorer: false, 
flags::IsOnStack: true, 
flags::IsRestart: true, 
flags::IsResetHandler: false, 
flags::IsNoDefer: false, 
flags::IsSigInfo: true, 
restorer : 0,  
mask: fffffffffffbfeff,  
}
[INFO] [1/4540140000|0] Signal 33: delivering to handler
[INFO] [1/4540140000|0] (5/1)------Return[0] res is 21: call id sys_futex 
[INFO] [1/4540140000|0] (5/1)------get call id sys_write arg0:2
[ERROR] [1/4540140000|0] (Data) Write fail: fatal: bad g in signal handler

[INFO] [1/45400c0000|0] (1/1)------Return[0] res is 0: call id sys_sched_yield 
[INFO] [1/45400c0000|0] (1/1)------get call id sys_sched_yield arg0:0
[INFO] [1/4540120000|0] (4/1)------Return[0] res is 1f: call id sys_write 
[INFO] [1/4540120000|0] (4/1)------get call id sys_exit_group arg0:2
[INFO] [1/4540120000|0] RunExit[4540120000] ...
[INFO] [1/4540120000|0] advanceExitStateLocked[4] TaskExitNone=>TaskExitInitiated
[INFO] [1/4540120000|0] RunExitNotify ...
[INFO] [1/4540120000|0] advanceExitStateLocked[4] TaskExitInitiated=>TaskExitZombie
[INFO] [1/4540120000|0] living task count:3
[INFO] [1/4540120000|0] advanceExitStateLocked[4] TaskExitZombie=>TaskExitDead
[INFO] [1/4540160000|0] RunInterrupt[4540160000] ...
[DEBUG] [1/4540160000|0] task_signals: thread deliver signal SigAction { 
handler: 0, 
flag : 0, 
flags::HasRestorer: false, 
flags::IsOnStack: false, 
flags::IsRestart: false, 
flags::IsResetHandler: false, 
flags::IsNoDefer: false, 
flags::IsSigInfo: false, 
restorer : 0,  
mask: 0,  
}
[INFO] [1/4540160000|0] Signal 9: terminating thread group
[INFO] [1/4540160000|0] RunExit[4540160000] ...
[INFO] [1/4540160000|0] advanceExitStateLocked[6] TaskExitNone=>TaskExitInitiated
[INFO] [1/4540160000|0] RunExitNotify ...
[INFO] [1/4540160000|0] advanceExitStateLocked[6] TaskExitInitiated=>TaskExitZombie
[INFO] [1/4540160000|0] living task count:2
[INFO] [1/4540160000|0] advanceExitStateLocked[6] TaskExitZombie=>TaskExitDead
[INFO] [1/4540140000|0] RunInterrupt[4540140000] ...
[DEBUG] [1/4540140000|0] task_signals: thread deliver signal SigAction { 
handler: 0, 
flag : 0, 
flags::HasRestorer: false, 
flags::IsOnStack: false, 
flags::IsRestart: false, 
flags::IsResetHandler: false, 
flags::IsNoDefer: false, 
flags::IsSigInfo: false, 
restorer : 0,  
mask: 0,  
}
[INFO] [1/4540140000|0] Signal 9: terminating thread group
[INFO] [1/4540140000|0] RunExit[4540140000] ...
[INFO] [1/4540140000|0] advanceExitStateLocked[5] TaskExitNone=>TaskExitInitiated
[INFO] [1/4540140000|0] RunExitNotify ...
[INFO] [1/4540140000|0] advanceExitStateLocked[5] TaskExitInitiated=>TaskExitZombie
[INFO] [1/4540140000|0] living task count:1
[INFO] [1/4540140000|0] advanceExitStateLocked[5] TaskExitZombie=>TaskExitDead
[INFO] [1/45400c0000|0] RunInterrupt[45400c0000] ...
[DEBUG] [1/45400c0000|0] task_signals: thread deliver signal SigAction { 
handler: 0, 
flag : 0, 
flags::HasRestorer: false, 
flags::IsOnStack: false, 
flags::IsRestart: false, 
flags::IsResetHandler: false, 
flags::IsNoDefer: false, 
flags::IsSigInfo: false, 
restorer : 0,  
mask: 0,  
}
[INFO] [1/45400c0000|0] Signal 9: terminating thread group
[INFO] [1/45400c0000|0] RunExit[45400c0000] ...
[INFO] [1/45400c0000|0] advanceExitStateLocked[1] TaskExitNone=>TaskExitInitiated
[INFO] [1/45400c0000|0] RunExitNotify ...
[INFO] [1/45400c0000|0] advanceExitStateLocked[1] TaskExitInitiated=>TaskExitZombie
[INFO] [1/45400c0000|0] living task count:0
[INFO] [1/45400c0000|0] advanceExitStateLocked[1] TaskExitZombie=>TaskExitDead
[INFO] [1/45400c0000|0]  sending exit notification for CID:1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a, execID:
[INFO] [1/45400c0000|0] ExitNotify shutdown
[ERROR] [1/45400c0000|0] PerfPrint::Kernel not ready ....
[ERROR] [1/45400c0000|0] PerfPrint::Threads not ready ....
[DEBUG] [1/45400c0000|0] VM: Start clean VMAs - KEY.0 ->
vseg:VMA { offset: 0, realPerms: AccessType(5), effectivePerms: AccessType(5), maxPerms: AccessType(7), private: true, growsDown: false, kernel: false, hint: "" }
vgap:Range { start: 0, len: 65536 }
[ERROR] [1/305757] PerfPrint::Kernel not ready ....
[INFO] [1/305791] cpu#1 finish
[INFO] [0/305799] cpu0 finish
[INFO] [0/306224] unmap ptr is 4000000000, len is 2ba000
[INFO] [0/308496] unmap ptr is 40002ba000, len is 1000
[INFO] [0/308528] unmap ptr is 40002bb000, len is 3000
[INFO] [0/33531000153197] commandline args is /usr/local/bin/quark_d --root /var/run/docker/runtime-quark_d/moby --log /run/containerd/io.containerd.runtime.v1.linux/moby/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/log.json --log-format json delete 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a 
[INFO] [0/33531000157712] Container:: Delete ....
[INFO] [0/33531000157801] Load metadata for container /var/run/docker/runtime-quark_d/moby 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000189727] metadatafile is /var/run/docker/runtime-quark_d/moby/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/meta.json
[INFO] [0/33531000242807] sandbox is not running, marking container as stopped...
[INFO] [0/33531000242908] Destroy container 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000242939] Find sandbox id for container 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000242977] Destroying container 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000243005] destroying root container by destroying sandbox
[INFO] [0/33531000243032] Destroy sandbox 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000243057] Killing sandbox 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a with signal 9
[INFO] [0/33531000243085] self child is false, pid is 3483498
[INFO] [0/33531000243112] Destroy sandbox 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000243138] Killing sandbox 1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a with signal 9
[INFO] [0/33531000243161] self child is false, pid is 3483498
[INFO] [0/33531000243190] container process stopped
[INFO] [0/33531000243827] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/proc
[WARN] [0/33531000243923] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/proc fail, error is -22
[INFO] [0/33531000244021] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev
[WARN] [0/33531000244121] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev fail, error is -22
[INFO] [0/33531000244359] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev/pts
[WARN] [0/33531000244403] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev/pts fail, error is -22
[INFO] [0/33531000244471] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/sys
[WARN] [0/33531000244510] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/sys fail, error is -22
[INFO] [0/33531000244758] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev/shm
[WARN] [0/33531000244801] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev/shm fail, error is -22
[INFO] [0/33531000244871] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/data
[WARN] [0/33531000244911] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/data fail, error is -22
[INFO] [0/33531000244981] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/etc/resolv.conf
[WARN] [0/33531000245022] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/etc/resolv.conf fail, error is -22
[INFO] [0/33531000245083] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/etc/hostname
[WARN] [0/33531000245122] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/etc/hostname fail, error is -22
[INFO] [0/33531000245183] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/etc/hosts
[WARN] [0/33531000245222] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/etc/hosts fail, error is -22
[INFO] [0/33531000245265] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev
[WARN] [0/33531000245303] MountContainerFs: unmount container fs /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/dev fail, error is -22
[INFO] [0/33531000245339] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000251851] deleting container root directory.../var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000252560] unmount sandbox root: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000252693] unmount destination: /var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a
[INFO] [0/33531000256154] deleting sandbox root directory.../var/lib/quark/1abc057abe5197b525863f490af83b70fb7f94d373231e5f9b9d9be729b6731a

Which I could not understand what the meaning of fatal: bad g in signal handler is. I search about it but I did not find anything that explain what the problem is. Could you please help us to solve it?

My branch is QuarkContainer:non_uring merged with shrik3:arm_dev_base

shrik3 commented 3 months ago

hi,

On Wed Jun 26, 2024 at 1:59 AM CEST, Hadi Omidi wrote:

Secondly and the main one, it seems that signal handing on ARM has some issues. When I run Redis, the error log is this (last lines):

this is known issue, see

https://github.com/QuarkContainer/Quark/pull/1217#issuecomment-2104144397

tl;dr, the signal handler for arm is not setting up the signal stack correctly. It works (magically) for other applications, but go runtime expects something different. I don't have time to look into this though, feel free to explore :D

Somewhere in the application, it calls sys call 435 which is NotImplementSyscall in ARM

https://github.com/QuarkContainer/Quark/pull/1270

This PR sets those to SysNoSys

/proc/self/cgroup

AFAIK Quark doesn't support cgroup (or anything related to user management) . This doesn't exist for the x86 build either.

mhomidi commented 3 months ago

Thanks for the clarification. In such case, I think I can close this issue.