Podman and DragonflyDB - Githubissues

Akaame commented 2 years ago

Describe the bug

podman run --network=host docker.dragonflydb.io/dragonflydb/dragonfly fails with proactor.cc:432] Error initializing io_uring: (38) Function not implemented which, I think, indicates that io_uring is not supported. In this case, can we have a user friendly error here instead of the stack trace:

podman run --network=host docker.dragonflydb.io/dragonflydb/dragonfly
I20220617 13:37:19.163322     1 init.cc:56] dragonfly running in opt mode.
I20220617 13:37:19.164311     1 dfly_main.cc:179] maxmemory has not been specified. Deciding myself....
I20220617 13:37:19.164657     1 dfly_main.cc:184] Found 1.63GiB available memory. Setting maxmemory to 1.30GiB
F20220617 13:37:19.165297     4 proactor.cc:432] Error initializing io_uring: (38) Function not implemented
*** Check failure stack trace: ***
    @     0xaaaad1850394  google::LogMessage::SendToLog()
    @     0xaaaad1849478  google::LogMessage::Flush()
    @     0xaaaad184adb4  google::LogMessageFatal::~LogMessageFatal()
    @     0xaaaad1582d40  util::uring::Proactor::Init()
    @     0xaaaad1590ff0  _ZNSt17_Function_handlerIFvvEZN4util12ProactorPool14SetupProactorsEvEUlvE_E9_M_invokeERKSt9_Any_data.lto_priv.0
    @     0xaaaad1577c3c  _ZN4baseL18start_cpp_functionEPv.lto_priv.0
    @     0xffff83618624  start_thread
    @     0xffff832bb49c  (unknown)
    @              (nil)  (unknown)
*** SIGABRT received at time=1655473039 on cpu 0 ***
PC: @     0xffff8321dd78  (unknown)  raise
    @     0xaaaad1892468       4976  absl::lts_20211102::AbslFailureSignalHandler()
    @     0xffff836867dc        304  (unknown)
    @     0xffff8320aaac        336  abort
    @     0xaaaad1856054        192  google::DumpStackTraceAndExit()
    @     0xaaaad1849a74         16  google::LogMessage::Fail()
    @     0xaaaad1850394        208  google::LogMessage::SendToLog()
    @     0xaaaad1849478         80  google::LogMessage::Flush()
    @     0xaaaad184adb4         16  google::LogMessageFatal::~LogMessageFatal()
    @     0xaaaad1582d40        512  util::uring::Proactor::Init()
    @     0xaaaad1590ff0         32  std::_Function_handler<>::_M_invoke()
    @     0xaaaad1577c3c        144  base::start_cpp_function()
    @     0xffff83618624  (unknown)  start_thread
[failure_signal_handler.cc : 331] RAW: Signal 5 raised at PC=0xffff8320ab7c while already in AbslFailureSignalHandler()
*** SIGTRAP received at time=1655473039 on cpu 0 ***
PC: @     0xffff8320ab7c  (unknown)  abort
    @     0xaaaad1892468       4976  absl::lts_20211102::AbslFailureSignalHandler()
    @     0xffff836867dc        336  (unknown)
    @     0xaaaad1856054        192  google::DumpStackTraceAndExit()
    @     0xaaaad1849a74         16  google::LogMessage::Fail()
    @     0xaaaad1850394        208  google::LogMessage::SendToLog()
    @     0xaaaad1849478         80  google::LogMessage::Flush()
    @     0xaaaad184adb4         16  google::LogMessageFatal::~LogMessageFatal()
    @     0xaaaad1582d40        512  util::uring::Proactor::Init()
    @     0xaaaad1590ff0         32  std::_Function_handler<>::_M_invoke()
    @     0xaaaad1577c3c        144  base::start_cpp_function()
    @     0xffff83618624  (unknown)  start_thread

To Reproduce Steps to reproduce the behavior:

Install podman
podman machine init
podman machine start
podman run --network=host docker.dragonflydb.io/dragonflydb/dragonfly

Expected behavior A more readable/helpful error message that indicated io_uring is not supported/enabled and exits the program. Similar to the one when not enough memory is available io_uring does not have enough memory. That can happen when your max locked memory is too limited. If you run me via docker, try adding '--ulimit memlock=-1' to docker run command

Desktop (please complete the following information):

OS: Fedora CoreOS 36.20220511.dev.0
Kernel: 5.17.5-300.fc36.aarch64
Containerized?: Fedora VM created by podman-machine

Version 0.3.0

Additional Context

grep io_uring_setup /proc/kallsyms

0000000000000000 t io_uring_setup
0000000000000000 T __arm64_sys_io_uring_setup
0000000000000000 d __event_exit__io_uring_setup
0000000000000000 d __event_enter__io_uring_setup
0000000000000000 d __p_syscall_meta__io_uring_setup
0000000000000000 d _eil_addr___arm64_sys_io_uring_setup
0000000000000000 d event_exit__io_uring_setup
0000000000000000 d event_enter__io_uring_setup
0000000000000000 d __syscall_meta__io_uring_setup
0000000000000000 d args__io_uring_setup
0000000000000000 d types__io_uring_setup

Follow up Questions How can I concretely check if io_uring is supported / if it is compiled out or not?

Cheers.

romange commented 2 years ago

interesting. I've never run it via podman. I wonder why io_uring is disabled there. @axboe do you know maybe?

romange commented 2 years ago

@Akaame did you try running dragonfly natively on that host? Does it work then?

axboe commented 2 years ago

I have no idea, sorry, never used it.

romange commented 2 years ago

Confirmed that it fails on Ubuntu as well. @Akaame you can still run it with --privileged flag. I do not know why podman in rootless mode can not call io_uring APIs.

Tieger commented 2 years ago

I had the same problem with WSL2. uname -a Linux XSJ-20210114XOJ 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

./dragonfly --alsologtostderr I20220623 17:28:24.142984 30367 init.cc:56] ./dragonfly running in opt mode. I20220623 17:28:24.143221 30367 dfly_main.cc:179] maxmemory has not been specified. Deciding myself.... I20220623 17:28:24.143266 30367 dfly_main.cc:184] Found 11.17GiB available memory. Setting maxmemory to 8.93GiB F20220623 17:28:24.143419 30368 proactor.cc:432] Error initializing io_uring: (38) Function not implemented Check failure stack trace: ...

axboe commented 2 years ago

Newer WSL2 kernels have it enabled.

mybigman commented 2 years ago

same issue

I20220630 01:42:54.135711     1 init.cc:56] dragonfly running in opt mode.
I20220630 01:42:54.135757     1 dfly_main.cc:179] maxmemory has not been specified. Deciding myself....
I20220630 01:42:54.135784     1 dfly_main.cc:184] Found 3.12GiB available memory. Setting maxmemory to 2.49GiB
F20220630 01:42:54.136021     4 proactor.cc:432] Error initializing io_uring: (38) Function not implemented
*** Check failure stack trace: ***
    @     0xaaaaaf88e394  google::LogMessage::SendToLog()
    @     0xaaaaaf887478  google::LogMessage::Flush()
    @     0xaaaaaf888db4  google::LogMessageFatal::~LogMessageFatal()
F20220630 01:42:54.137372     5 proactor.cc:432] Error initializing io_uring: (38) Function not implemented
    @     0xaaaaaf5c2c48  util::uring::Proactor::Init()
*** Check failure stack trace: ***

dragonflydb / dragonfly

Podman and DragonflyDB #169