search

unikraft / kraftkit

Build and use highly customized and ultra-lightweight unikernel VMs.
https://unikraft.org/docs/cli
BSD 3-Clause "New" or "Revised" License
237 stars 63 forks source link

Redis does not properly start (config file is not read) #722

Closed razvand closed 1 year ago

razvand commented 1 year ago

Describe the bug

When running app-redis with KraftKit it hangs, and no connection is possible. This is very likely due because the command line argument (the configuration file) is not properly passed. If Redis were to read the configuration file, it would provide an ASCII art logo.

Steps to reproduce

Run [app-redis]() using KraftKit. Sample run command:

kraft run --target redis-qemu-x86_64-initrd --initrd ./fs0 -p 6379:6379 -- /redis.conf

The output is frozen, as if the configuration file is not read:

en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Atlas 0.13.1~67ccbbf6
1:C 18 Aug 2015 03:52:31.020 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 18 Aug 2015 03:52:31.024 # Redis version=7.0.11, bits=64, commit=c5ee3442, modified=1, pid=1, just started
1:C 18 Aug 2015 03:52:31.029 # Configuration loaded
[    0.161082] ERR:  [libposix_process] <deprecated.c @  348> Ignore updating resource 7: cur = 10032, max = 10032
1:M 18 Aug 2015 03:52:31.066 * Increased maximum number of open files to 10032 (it was originally set to 1024).
1:M 18 Aug 2015 03:52:31.071 * monotonic clock: POSIX clock_gettime

Expected behavior

The output should be full. See the difference between running KraftKit vs qemu-system-x86_64 directly:

$ kraft run --initrd ./fs0 -p 6379:6379 .unikraft/build/redis-qemu-x86_64-initrd_qemu-x86_64 -- /redis.conf
[...]
en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Atlas 0.13.1~67ccbbf6
1:C 18 Aug 2015 03:52:31.020 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 18 Aug 2015 03:52:31.024 # Redis version=7.0.11, bits=64, commit=c5ee3442, modified=1, pid=1, just started
1:C 18 Aug 2015 03:52:31.029 # Configuration loaded
[    0.161082] ERR:  [libposix_process] <deprecated.c @  348> Ignore updating resource 7: cur = 10032, max = 10032
1:M 18 Aug 2015 03:52:31.066 * Increased maximum number of open files to 10032 (it was originally set to 1024).
1:M 18 Aug 2015 03:52:31.071 * monotonic clock: POSIX clock_gettime

$ cat run-qemu-x86_64-initrd.sh 
#!/bin/sh

kernel_image="build/redis_qemu-x86_64"

if test $# -eq 1; then
    kernel_image="$1"
fi

cd fs0
find -depth -print | tac | bsdcpio -o --format newc > ../fs0.cpio
cd ..

sudo ip link set dev virbr0 down 2> /dev/null
sudo ip link del dev virbr0 2> /dev/null
sudo ip link add dev virbr0 type bridge
sudo ip address add 172.44.0.1/24 dev virbr0
sudo ip link set dev virbr0 up

sudo qemu-system-x86_64 \
    -enable-kvm \
    -cpu host \
    -kernel "$kernel_image" -nographic \
    -netdev bridge,id=en0,br=virbr0 -device virtio-net-pci,netdev=en0 \
    -append "netdev.ipv4_addr=172.44.0.2 netdev.ipv4_gw_addr=172.44.0.1 netdev.ipv4_subnet_mask=255.255.255.0 -- /redis.conf" \
    -initrd fs0.cpio

$ ./run-qemu-x86_64-initrd.sh .unikraft/build/redis-qemu-x86_64-initrd_qemu-x86_64
[...]
Booting from ROM..1: Set IPv4 address 172.44.0.2 mask 255.255.255.0 gw 172.44.0.1
en1: Added
en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Atlas 0.13.1~67ccbbf6
1:C 18 Aug 2015 03:24:30.028 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 18 Aug 2015 03:24:30.033 # Redis version=7.0.11, bits=64, commit=c5ee3442, modified=1, pid=1, just started
1:C 18 Aug 2015 03:24:30.040 # Configuration loaded
[    0.168579] ERR:  [libposix_process] <deprecated.c @  348> Ignore updating resource 7: cur = 10032, max = 10032
1:M 18 Aug 2015 03:24:30.076 * Increased maximum number of open files to 10032 (it was originally set to 1024).
1:M 18 Aug 2015 03:24:30.084 * monotonic clock: POSIX clock_gettime
                _._                          
           _.-``__ ''-._                     
      _.-``    `.  `_.  ''-._           Redis 7.0.11 (c5ee3442/1) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._           
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1
  `-._    `-._  `-./  _.-'    _.-'           
 |`-._`-._    `-.__.-'    _.-'_.-'|          
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'           
 |`-._`-._    `-.__.-'    _.-'_.-'|          
 |    `-._`-._        _.-'_.-'    |          
  `-._    `-._`-.__.-'_.-'    _.-'           
      `-._    `-.__.-'    _.-'               
          `-._        _.-'                   
              `-.__.-'                       

1:M 18 Aug 2015 03:24:30.223 # WARNING: The TCP backlog setting of 511 cannot be enforced because SOMAXCONN is set to the lower value of 128.
1:M 18 Aug 2015 03:24:30.238 # Server initialized
1:M 18 Aug 2015 03:24:30.258 * Ready to accept connections

Which architectures were you using or does this bug affect?

x86_64

Which operating system were you using or does this bug affect?

linux/debian

Relevant log output

en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                  Atlas 0.13.1~67ccbbf6
1:C 18 Aug 2015 03:52:31.020 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 18 Aug 2015 03:52:31.024 # Redis version=7.0.11, bits=64, commit=c5ee3442, modified=1, pid=1, just started
1:C 18 Aug 2015 03:52:31.029 # Configuration loaded
[    0.161082] ERR:  [libposix_process] <deprecated.c @  348> Ignore updating resource 7: cur = 10032, max = 10032
1:M 18 Aug 2015 03:52:31.066 * Increased maximum number of open files to 10032 (it was originally set to 1024).
1:M 18 Aug 2015 03:52:31.071 * monotonic clock: POSIX clock_gettime
jake-ciolek commented 1 year ago

Thanks for the report.

It seems that the config does get loaded but redis doesn't spin up.

If you were to change kraft args to:

kraft run --target redis-qemu-x86_64-initrd --initrd ./fs0 -p 6379:6379 -- /redis.doesntexist

Redis will complain about not being able to find the file:

en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
                   Atlas 0.13.1~513cee9
1:C 14 Sep 2015 22:56:46.005 # Fatal error, can't open config file '/redis.doesntexist': No such file or directory

I'll be diving deeper into this issue to identify the root cause.

jake-ciolek commented 1 year ago

This seems to be a networking configuration issue. The redis.conf file makes it bind to 172.44.0.2

kraft run works if redis is made to listen on all interfaces via config change to:

bind 0.0.0.0
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
protected-mode no

See:

[    0.000000] ERR:  [libukallocbbuddy] <bbuddy.c @  488> 12000: Failed to add memory region 42b0ae-42c000: Not enough space after applying page alignments
en1: Added
en1: Interface is up
Powered by
o.   .o       _ _               __ _
Oo   Oo  ___ (_) | __ __  __ _ ' _) :_
oO   oO ' _ `| | |/ /  _)' _` | |_|  _)
oOo oOO| | | | |   (| | | (_) |  _) :_
 OoOoO ._, ._:_:_,\_._,  .__,_:_, \___)
              Prometheus 0.14.0~513cee9
1:C 14 Sep 2015 23:18:08.005 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 14 Sep 2015 23:18:08.006 # Redis version=7.0.11, bits=64, commit=c5ee3442, modified=1, pid=1, just started
1:C 14 Sep 2015 23:18:08.008 # Configuration loaded
[    0.111319] ERR:  [libposix_process] <deprecated.c @  348> Ignore updating resource 7: cur = 10032, max = 10032
1:M 14 Sep 2015 23:18:08.012 * Increased maximum number of open files to 10032 (it was originally set to 1024).
1:M 14 Sep 2015 23:18:08.014 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 7.0.11 (c5ee3442/1) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

1:M 14 Sep 2015 23:18:08.032 # WARNING: The TCP backlog setting of 511 cannot be enforced because SOMAXCONN is set to the lower value of 128.
1:M 14 Sep 2015 23:18:08.034 # Server initialized
1:M 14 Sep 2015 23:18:08.036 * Ready to accept connections
en1: Set IPv4 address 10.0.2.15 mask 255.255.255.0 gw 10.0.2.2
razvand commented 1 year ago

@jake-ciolek , this looks good. I will try it out. If it works, I will close this PR.

craciunoiuc commented 1 year ago

Considering that there were no complaints with this, and redis was recently pushed to the registry, I will assume that this works now just fine and will close