search

unikraft / kraftkit

Build and use highly customized and ultra-lightweight unikernel VMs.
https://unikraft.org/docs/cli
BSD 3-Clause "New" or "Revised" License
238 stars 63 forks source link

`kraft rm --all` ends up with an error #820

Open razvand opened 1 year ago

razvand commented 1 year ago

Describe the bug

After doing KraftKit testing, kraft rm --all end up with an error.

Steps to reproduce

I used KraftKit and, then, at some point I wanted to remove the created VMs. I used a normal account to remove those created by the privileged account.

I'm using Ubuntu 22.04. KraftKit version kraft 0.6.7 (b139ae2a52608f2e4a910225be0c981f671fe75f) go1.20.6 2023-09-14T09:00:05Z.

Expected behavior

kraft rm --all should remove all Unikernel images / VMs, not end up in error.

Which architectures were you using or does this bug affect?

x86_64

Which operating system were you using or does this bug affect?

linux/debian

Relevant log output

razvan@yggdrasil:~/projects/unikraft/maintainer-tools/workdir/apps/elfloader$ sudo kraft ps
NAME                      KERNEL                                 ARGS                                                                                 CREATED     STATUS  MEM   PLAT
agitated_joemartin        project://elfloader:kraftcloud-x86_64  /bin/python3.11 -m http.server 8080                                                  2 days ago  exited  0MiB  fc/x86_64
agitated_lucy             project://elfloader:kraftcloud-x86_64  /python -m http.server 8080                                                          1 day ago   exited  0MiB  fc/x86_64
charming_faben            project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
charming_frodo            project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
crazy_edgar               project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 http_server.py       2 days ago  exited  0MiB  fc/x86_64
eager_ayumu               project://elfloader:kraftcloud-x86_64  /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf                      2 days ago  exited  0MiB  fc/x86_64
elegant_loon              project://elfloader:kraftcloud-x86_64  /python http_server.py                                                               1 day ago   exited  0MiB  fc/x86_64
exciting_timmy            project://elfloader:kraftcloud-x86_64  /python -m http.server 8080                                                          1 day ago   exited  0MiB  fc/x86_64
focused_ahmeng            project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
frosty_azy                project://elfloader:kraftcloud-x86_64  /http_server                                                                         1 day ago   exited  0MiB  fc/x86_64
funny_crystal             project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 http_server.py       2 days ago  exited  0MiB  fc/x86_64
gifted_sultan             project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
gracious_bintijua         project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
interesting_sultan        project://elfloader:kraftcloud-x86_64  /http_server                                                                         1 day ago   exited  0MiB  fc/x86_64
jolly_fanni               project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 --version            2 days ago  exited  0MiB  fc/x86_64
musing_ahmeng             project://elfloader:kraftcloud-x86_64  /http_server                                                                         2 days ago  exited  0MiB  fc/x86_64
musing_gargantua          project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
musing_panbanisha         project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  1 day ago   exited  0MiB  fc/x86_64
objective_davidgreybeard  project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
pedantic_washoe           project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  2 days ago  exited  0MiB  fc/x86_64
priceless_littlemama      project://elfloader:kraftcloud-x86_64  /python -m http.server 8080                                                          1 day ago   exited  0MiB  fc/x86_64
upbeat_crystal            project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11                      2 days ago  exited  0MiB  fc/x86_64
vigorous_ramu             project://elfloader:kraftcloud-x86_64  /python http_server.py                                                               1 day ago   exited  0MiB  fc/x86_64
xenodochial_jaccomacacco  project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 -m http.server 8080  1 day ago   exited  0MiB  fc/x86_64
youthful_viki             project://elfloader:kraftcloud-x86_64  /lib64/ld-linux-x86-64.so.2 --library-path /lib /bin/python3.11 http_server.py       2 days ago  exited  0MiB  fc/x86_64
zealous_flint             project://elfloader:kraftcloud-x86_64  /http_server                                                                         1 day ago   exited  0MiB  fc/x86_64

razvan@yggdrasil:~/projects/unikraft/maintainer-tools/workdir/apps/elfloader$ sudo kraft rm --all
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
kraftkit.sh/machine/network/bridge.(*v1alpha1Network).Get(0x0?, {0x0?, 0x0?}, 0xc000619680)
        /__w/kraftkit/kraftkit/machine/network/bridge/v1alpha1.go:450 +0x378
api%2ezip.(*MethodClient[...]).Do(0x1b22580, {0x1b2e308, 0xc00033ad80}, 0xc000619680)
        /go/pkg/mod/api.zip@v0.1.5/client.go:49 +0xe8
kraftkit.sh/api/network/v1alpha1.(*NetworkServiceHandler).Get(0x163fe00?, {0x1b2e308?, 0xc00033ad80?}, 0x6?)
        /__w/kraftkit/kraftkit/api/network/v1alpha1/network.zip.go:151 +0x2c
kraftkit.sh/cmd/kraft/rm.(*Rm).Run(0xc0000117d0, 0x0?, {0xc000120c10, 0x0, 0x0?})
        /__w/kraftkit/kraftkit/cmd/kraft/rm/rm.go:143 +0x1068
kraftkit.sh/cmdfactory.bind.func1(0xc000172000?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc000172000?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc000172000?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0x0?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc00059bd80?, {0xc000120c10, 0x0, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.execute(0xc000172000, {0xc0007a1120, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:231 +0x30b
kraftkit.sh/cmdfactory.executeC(0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:291 +0x205
kraftkit.sh/cmdfactory.Main({0x1b2e308?, 0xc00033ad80}, 0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:322 +0x73
main.main()
        /__w/kraftkit/kraftkit/cmd/kraft/kraft.go:153 +0x549
razvand commented 1 year ago

I also get this error when using kraft rm:

razvan@yggdrasil:~/projects/unikraft/maintainer-tools/workdir/apps/elfloader$ sudo kraft rm agitated_joemartin
panic: runtime error: index out of range [0] with length 0

goroutine 1 [running]:
kraftkit.sh/machine/network/bridge.(*v1alpha1Network).Get(0x0?, {0x0?, 0x0?}, 0xc000577440)
        /__w/kraftkit/kraftkit/machine/network/bridge/v1alpha1.go:450 +0x378
api%2ezip.(*MethodClient[...]).Do(0x1b22580, {0x1b2e308, 0xc0003cc000}, 0xc000577440)
        /go/pkg/mod/api.zip@v0.1.5/client.go:49 +0xe8
kraftkit.sh/api/network/v1alpha1.(*NetworkServiceHandler).Get(0x163fe00?, {0x1b2e308?, 0xc0003cc000?}, 0x6?)
        /__w/kraftkit/kraftkit/api/network/v1alpha1/network.zip.go:151 +0x2c
kraftkit.sh/cmd/kraft/rm.(*Rm).Run(0xc000483800, 0x0?, {0xc0004a3620, 0x1, 0x0?})
        /__w/kraftkit/kraftkit/cmd/kraft/rm/rm.go:143 +0x1068
kraftkit.sh/cmdfactory.bind.func1(0xc0007b8000?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc0007b8000?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc0007b8000?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0x0?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.bind.func1(0xc0008b9d80?, {0xc0004a3620, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:689 +0x19d
kraftkit.sh/cmdfactory.execute(0xc0007b8000, {0xc00033daa0, 0x1, 0x1})
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:231 +0x30b
kraftkit.sh/cmdfactory.executeC(0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:291 +0x205
kraftkit.sh/cmdfactory.Main({0x1b2e308?, 0xc0003cc000}, 0xc000004600)
        /__w/kraftkit/kraftkit/cmdfactory/builder.go:322 +0x73
main.main()
        /__w/kraftkit/kraftkit/cmd/kraft/kraft.go:153 +0x549
razvand commented 1 year ago

I may have caused this by doing the following:

  1. Create a network interface with:

    sudo kraft net create ...

  2. Create a Unikraft instance:

    sudo kraft run ...

  3. Remove the network interface with:

    sudo ip link set dev kraft0 down
sudo ip link del dev kraft0

  4. Run the kraft rm command:

    sudo kraft rm --all
craciunoiuc commented 1 year ago

Trying to remove a used interface is protected again in kraft thankfully

➜  app-nginx git:(staging) ✗ sudo kraft net rm kraftt1
 E  interface still in use: kraftt1@if0 (02:b0:b0:d3:64:01, 172.46.0.2)
craciunoiuc commented 1 year ago

Deleting also messes up the network list

➜  app-nginx git:(staging) ✗ sudo ip link set dev kraftt1 down
➜  app-nginx git:(staging) ✗ sudo ip link del dev kraftt1
➜  app-nginx git:(staging) ✗ curl 172.46.0.2
^C
➜  app-nginx git:(staging) ✗ sudo kraft net rm kraftt1
 E  getting bridge kraftt1 failed: Link not found
➜  app-nginx git:(staging) ✗ sudo kraft net ls
NAME             NETWORK           DRIVER  STATUS
br-4ee203c06b74  172.20.0.1/16     bridge  up
br-c0a894b2682d  192.168.96.1/20   bridge  up
docker0          172.17.0.1/16     bridge  up
kraftt1          172.46.0.1/24     bridge  up
virbr0           192.168.122.1/24  bridge  up
craciunoiuc commented 1 year ago

I got a lot of errors running kraft rm --all on it but no crash

➜  app-nginx git:(staging) ✗ sudo kraft rm --all
peaceful_fanni
 W  could not get network information for kraftt1: could not get link kraftt1: Link not found
xenodochial_ferdinand

And on the the machine side:

 E  could not get network information for kraftt1: could not get link kraftt1: Link not found
craciunoiuc commented 1 year ago

Might be because of firecracker, checking now

craciunoiuc commented 1 year ago

Not that, not sure what caused the crash, I can't reproduce

I will add a fix that removes networks if they were already removed by something else