nubificus / urunc

a simple container runtime that aspires to become `runc` for unikernels
Apache License 2.0
48 stars 3 forks source link

MirageOS and urunc compatibilty #75

Open DeftaSebastian opened 1 week ago

DeftaSebastian commented 1 week ago

I have been trying to run a unikernel created using MirageOS and run it with the help of urunc. I am aware that urunc isn't made to work with MirageOS unikernels in mind, but it would help me a lot if you could give me some pointers for the problems that I came across.

System Info

As mentioned above, I tried 2 unikernels, they are both used as examples in the mirageOS-skeleton. The guide on how they work can be found on the getting started page (Step1:Hello World! and Step 4: Networking) of mirageOS.

Before I get to the problems found with the networking unikernel, I would like first to show the errors with the hello world example found at Step1: Hello World! in the same getting started guide.

The steps I took to build the unikernel are: mirage configure -t virtio (Similar erros or different ones appear if I build the unikernel for different VMM's) make depends dune build

Running the unikernel simply with /usr/bin/qemu-system-x86_64 -m 256M -cpu host -enable-kvm -nographic -vga none -kernel dist/hello.virtio will result in the correct behaviour, which is to display hello 4 times.

After this, I tried to build a docker image with this unikernel with the following files: Dockerfile:

FROM scratch
COPY dist/hello.virtio /hello.virtio
COPY ./urunc.json /urunc.json

LABEL "com.urunc.unikernel.binary"="/hello.virtio"
LABEL "com.urunc.unikernel.cmdline"="dummy"
LABEL "com.urunc.unikernel.unikernelType"="unikraft"
LABEL "com.urunc.unikernel.hypervisor"="qemu"
urunc.json:
{
    "com.urunc.unikernel.binary":"aGVsbG8udmlydGlv",
    "com.urunc.unikernel.cmdline":"ZHVtbXk=",
    "com.urunc.unikernel.unikernelType":"dW5pa3JhZnQ=",
    "com.urunc.unikernel.hypervisor":"cWVtdQ=="
}

To deploy the image, I made a unikernel.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: hello-world
  name: hello-world
spec:
  replicas: 1
  selector:
    matchLabels:
      run: hello-world
  template:
    metadata:
      labels:
        run: hello-world
    spec:
      runtimeClassName: urunc
      containers:
      - image: localhost:5000/hello-world:unikernel
        command: ['/hello.virtio']
        imagePullPolicy: Always
        name: hello-world
        securityContext:
          privileged: true

And the final result is:

SeaBIOS (version 1.15.0-1)

iPXE (https://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8B360+0FECB360 C000

Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x1c8fff)
Solo5:     rodata @ (0x1c9000 - 0x1f9fff)
Solo5:       data @ (0x1fa000 - 0x29afff)
Solo5:       heap >= 0x29b000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
Solo5: PCI:00:02: virtio-net device, base=0xc000, irq=10
Solo5: Virtio net: PCI:00:02 not in manifest
Solo5: PCI enumeration failed. Aborting.
Solo5: solo5_abort() called
Solo5: Halted

The command that was given to qemu to launch this unikernel was:

/usr/bin/qemu-system-x86_64 -m 256M -cpu host -enable-kvm -nographic -vga none -kernel /run/k3s/containerd/io.containerd.runtime.v2.task/k8s.io/884d182a0a3dc6638ad6a6209832758fda261e076b4cfd445e10ecb986e3d98f/rootfs/hello.virtio -net nic,model=virtio -net tap,script=no,ifname=tap0_urunc -append  netdev.ip=10.42.1.33/24:10.42.1.1:8.8.8.8    --

Now, seeing this error led me to believe that this could be caused by a network component missing from my unikernel. To try and resolve this I tried building the network unikernel found in the mirageOS starting guide Step4: Networking.

The building process of the unikernel is similar: mirage configure -t virtio make depends dune build

I used this command to start the unikernel: /usr/bin/qemu-system-x86_64 -m 256M -cpu host -enable-kvm -nographic -vga none -kernel dist/network.virtio

Perhaps I am missing some arguments since this is the result:

SeaBIOS (version 1.15.0-1)

iPXE (https://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8B360+0FECB360 C000

Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x24dfff)
Solo5:     rodata @ (0x24e000 - 0x29afff)
Solo5:       data @ (0x29b000 - 0x3aefff)
Solo5:       heap >= 0x3af000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
Solo5: Device 'service' of type NET_BASIC not attached.Solo5: All declared devices must be attached. Aborting.
Solo5: solo5_abort() called
Solo5: Halted

I was hoping that urunc would solve the networking issues so I prepared a docker image to use. Dockerfile:

FROM scratch
COPY dist/network.virtio /network.virtio
COPY ./urunc.json /urunc.json

LABEL "com.urunc.unikernel.binary"="/network.virtio"
LABEL "com.urunc.unikernel.cmdline"="dummy"
LABEL "com.urunc.unikernel.unikernelType"="unikraft"
LABEL "com.urunc.unikernel.hypervisor"="qemu"

urunc.json:

{
    "com.urunc.unikernel.binary":"bmV0d29yay52aXJ0aW8=",
    "com.urunc.unikernel.cmdline":"ZHVtbXk=",
    "com.urunc.unikernel.unikernelType":"dW5pa3JhZnQ=",
    "com.urunc.unikernel.hypervisor":"cWVtdQ=="
}

unikernel.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: network
  name: network
spec:
  replicas: 1
  selector:
    matchLabels:
      run: network
  template:
    metadata:
      labels:
        run: network
    spec:
      runtimeClassName: urunc
      containers:
      - image: localhost:5000/network:unikernel
        command: ['/network.virtio']
        imagePullPolicy: Always
        name: network
        securityContext:
          privileged: true

And the final result found in the logs of the created pod is:


SeaBIOS (version 1.15.0-1)

iPXE (https://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8B360+0FECB360 C000

Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x24dfff)
Solo5:     rodata @ (0x24e000 - 0x29afff)
Solo5:       data @ (0x29b000 - 0x3aefff)
Solo5:       heap >= 0x3af000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
Solo5: PCI:00:02: virtio-net device, base=0xc000, irq=10
Solo5: PCI:00:02: configured, mac=52:54:00:12:34:56, features=0x79bf8064
network: too many arguments, don't know what to do with 'dummy', 'netdev.ip=10.42.1.44/24:10.42.1.1:8.8.8.8', 'dummy'
Usage: network [--port=VAL] [OPTION]…
Try 'network --help' for more information.
Hint: To pass a space, it needs to be escaped twice: --hello='Hello,\ world!'
      Another possibility is: --hello='"Hello, world!"'
Solo5: solo5_exit(64) called
Solo5: Halted

Removing the "dummy" word from the commandline in the dockerfile/urunc.json results in the same error.

I am hoping you could guide me where to look in order to find what is missing in my approach. Thank you in advance!

Firobe commented 1 day ago

Hi Sebastian! I'll let the urunc devs answer as well but this is probably not the approach you want to take. The easiest way to have native MirageOS support in urunc would probably be to try to adapt the Rumprun support for MirageOS, since both use solo5-hvt/spt. But this is non-trivial work, especially since MirageOS doesn't have established conventions for storage/networking.

What you're trying to do is using the virtio target with the unikraft support of urunc, which you will have a hard time making work. Mirage's virtio target is quite limited and here urunc is trying to pass devices and command-line arguments in a way that MirageOS doesn't understand (e.g. the netdev.ip=.... is Unikraft-specific).

The best way to run MirageOS with urunc is currently the in-development qemu target for MirageOS, which relies on Unikraft and so works well with urunc. That target is not yet released but we're making a tool that'll automate that process and produce urunc-compatible images from MirageOS applications.

cmainas commented 1 day ago

Hello @DeftaSebastian ,

first of all, sorry for the delayed response. We were busy with a few submissions.

Also, thank you for the information in the description of the issue. Unfortunately, for the time being urunc does not support MirageOS. Therefore, and as correctly noted @Firobe , you are trying to boot a MirageOS unikernel with the same way as Unikraft. As a result, MirageOS does not understand the Unikraft specific kernel arguments and it fails.

However, we want to support as many unikernel frameworks as possible and MirageOS is on top of our list. The reason that we do not immediately add support for more unikernels is that we need to refactor some parts of our implementation for unikernels in urunc. Nonetheless and especially in the case of MirageOS, adding support for a new unikernel framework is (and will remain) quite straightforward. The developer should simply implement the unikernel interface and update the New function. On the same time this is exactly the point of urunc that we want to change before adding more unikernels. We need to add more methods in the interface and change the constructor.

Having said that, please take a look at the mirage branch. Using this branch, you will be able to boot MiragOS unikernels with urunc. Keep in mind though that this is a quick implementation for a PoC and it is not tested thoroughly. We will add support for MirageOS after a small refactor.

Consequently, when using the mirage branch we were able to run the hello-key unikernel with urunc. In particular, after building it, we can package it for urunc with pun and the following Containerfile:

#syntax=harbor.nbfc.io/nubificus/pun:latest
FROM scratch

COPY hello-key.virtio /unikernel/hello-key.virtio

LABEL "com.urunc.unikernel.binary"="/unikernel/hello-key.virtio"
LABEL "com.urunc.unikernel.cmdline"="--hello='Mirage\ urunc'"
LABEL "com.urunc.unikernel.unikernelType"="mirage"
LABEL "com.urunc.unikernel.hypervisor"="qemu"

and we can build the container with:

sudo docker build -f Containerfile -t harbor.nbfc.io/nubificus/urunc/hello-mirage-qemu:test .

Note: the same Containerfile can be used for bima too.

After building it, we can execute it with ctr:

$ sudo ctr run --rm --runtime io.containerd.urunc.v2 harbor.nbfc.io/nubificus/urunc/hello-mirage-qemu:test mirage-urunc
SeaBIOS (version 1.13.0-1ubuntu1.1)

iPXE (http://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8C820+0FECC820 C000

Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x1c2fff)
Solo5:     rodata @ (0x1c3000 - 0x1f2fff)
Solo5:       data @ (0x1f3000 - 0x28ffff)
Solo5:       heap >= 0x290000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
2024-11-26T14:02:45-00:00: [INFO] [application] 'Mirage urunc'
2024-11-26T14:02:47-00:00: [INFO] [application] 'Mirage urunc'
2024-11-26T14:02:48-00:00: [INFO] [application] 'Mirage urunc'
2024-11-26T14:02:49-00:00: [INFO] [application] 'Mirage urunc'
Solo5: solo5_exit(0) called
Solo5: Halted

As you noticed by yourself, when we use nerdctl urunc will give network access to the unikernel. Therefore, since this particular MirageOS unikernel was not configured with networking support, it would not be able to properly run.

$ sudo nerdctl run --rm -it --runtime io.containerd.urunc.v2 harbor.nbfc.io/nubificus/urunc/hello-mirage-qemu:test
SeaBIOS (version 1.13.0-1ubuntu1.1)

iPXE (http://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8C820+0FECC820 C000

Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x1c2fff)
Solo5:     rodata @ (0x1c3000 - 0x1f2fff)
Solo5:       data @ (0x1f3000 - 0x28ffff)
Solo5:       heap >= 0x290000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
Solo5: PCI:00:02: virtio-net device, base=0xc000, irq=10
Solo5: Virtio net: PCI:00:02 not in manifest
Solo5: PCI enumeration failed. Aborting.
Solo5: solo5_abort() called
Solo5: Halted

However, if we disable networking we will be able to run it with nerdctl or docker:

$ sudo nerdctl run --rm -it --network none --runtime io.containerd.urunc.v2 harbor.nbfc.io/nubificus/urunc/hello-mirage-qemu:test
SeaBIOS (version 1.13.0-1ubuntu1.1)

iPXE (http://ipxe.org) 00:02.0 C000 PCI2.10 PnP PMM+0FF8C820+0FECC820 C000

Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x1c2fff)
Solo5:     rodata @ (0x1c3000 - 0x1f2fff)
Solo5:       data @ (0x1f3000 - 0x28ffff)
Solo5:       heap >= 0x290000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
2024-11-26T14:09:08-00:00: [INFO] [application] 'Mirage urunc'
2024-11-26T14:09:09-00:00: [INFO] [application] 'Mirage urunc'
2024-11-26T14:09:10-00:00: [INFO] [application] 'Mirage urunc'
2024-11-26T14:09:11-00:00: [INFO] [application] 'Mirage urunc'
Solo5: solo5_exit(0) called
Solo5: Halted

Regarding the networking example. After building it, we can package the unikernel in the similar way. The respective Containerfile will be:

#syntax=harbor.nbfc.io/nubificus/pun:latest
FROM scratch

COPY network.virtio /unikernel/network.virtio

LABEL "com.urunc.unikernel.binary"="/unikernel/network.virtio"
LABEL "com.urunc.unikernel.cmdline"="-l *:debug"
LABEL "com.urunc.unikernel.unikernelType"="mirage"
LABEL "com.urunc.unikernel.hypervisor"="qemu"

and the command to build it:

$ sudo docker build -f Containerfile -t harbor.nbfc.io/nubificus/urunc/network-mirage-qemu:test .

After building it, we can run it with:

$ sudo nerdctl run --rm -it --runtime io.containerd.urunc.v2 harbor.nbfc.io/nubificus/urunc/network-mirage-qemu:test
Booting from ROM..            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 255 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x247fff)
Solo5:     rodata @ (0x248000 - 0x293fff)
Solo5:     rodata @ (0x248000 - 0x293fff)
Solo5:       data @ (0x294000 - 0x3a9fff)
Solo5:       heap >= 0x3aa000 < stack < 0xffe0000
Solo5: Clock source: KVM paravirtualized clock
Solo5: PCI:00:02: virtio-net device, base=0xc000, irq=10
Solo5: PCI:00:02: configured, mac=52:54:00:12:34:56, features=0x79bf8064
hello tcp world
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.pcb] Closing connection remote 10.4.0.1,38936 to local 
10.4.0.107, 8080
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] ND: Received NS: src=:: dst=ff02::1:ff12:3456 tgt=fe
80::5054:ff:fe12:3456
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:36:25-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:36:25-00:00: [DEBUG] [ndpc6] SLAAC: fe80::5054:ff:fe12:3456 --> PREFERRED
2024-11-26T14:36:25-00:00: [INFO] [ipv6] IP6: Started with fe80::5054:ff:fe12:3456
2024-11-26T14:36:25-00:00: [INFO] [tcp.pcb] TCP layer connected on 10.4.0.107/24, fe80::5054:ff
:fe12:3456/64
2024-11-26T14:36:25-00:00: [INFO] [udp] UDP layer connected on 10.4.0.107/24, fe80::5054:ff:fe1
2:3456/64
2024-11-26T14:36:25-00:00: [INFO] [tcpip-stack-direct] Dual TCP/IP stack assembled: mac=52:54:0
0:12:34:56,ip=10.4.0.107/24, fe80::5054:ff:fe12:3456/64
2024-11-26T14:36:25-00:00: [DEBUG] [tcpip-stack-direct] Establishing or updating listener for s
tack mac=52:54:00:12:34:56,ip=10.4.0.107/24, fe80::5054:ff:fe12:3456/64
2024-11-26T14:36:25-00:00: [DEBUG] [tcpip-stack-direct] Establishing or updating listener for s
tack mac=52:54:00:12:34:56,ip=10.4.0.107/24, fe80::5054:ff:fe12:3456/64
2024-11-26T14:36:34-00:00: [DEBUG] [ARP] replying to ARP request for 10.4.0.107 from 10.4.0.1 (
mac 9a:59:56:c2:5a:ae)
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.pcb] process-syn: [channels=0 listens=0 connects=0]
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.pcb] new-server-connection: [channels=0 listens=0 conne
cts=0]
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Closed  - Passive_open -> Listen
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Listen  - Send_synack(113678528) -> Syn_rcvd(1
13678528)
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.tcptimer] timerloop
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.tcptimer] timerloop: sleeping for 667000000 ns
2024-11-26T14:36:34-00:00: [DEBUG] [ARP] ignoring ARP request for 10.4.0.1 from 10.4.0.107 (mac
 52:54:00:12:34:56)
2024-11-26T14:36:34-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 28 payload
 len 0, needed_bytes 48
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.pcb] process-ack: [channels=0 listens=1 connects=0]
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.window] sequence validation: seq=87147542 range=8714754
2[262140] res=true
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Syn_rcvd(113678528)  - Recv_ack(113678529) -> 
Established
2024-11-26T14:36:34-00:00: [INFO] [application] new tcp connection from IP 10.4.0.1 on port 389
36
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.window] sequence validation: seq=87147542 range=8714754
2[262140] res=true
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Established  - Recv_ack(113678529) -> Establis
hed
2024-11-26T14:36:34-00:00: [DEBUG] [application] read: 15 bytes:
hello tcp world
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.pcb] Closing connection remote 10.4.0.1,38936 to local 
10.4.0.107, 8080
Solo5: Application acquired 'service' as network device
2024-11-26T14:36:24-00:00: [INFO] [netif] Plugging into service with mac 52:54:00:12:34:56 mtu 
1500
2024-11-26T14:36:24-00:00: [INFO] [ethernet] Connected Ethernet interface 52:54:00:12:34:56
2024-11-26T14:36:24-00:00: [INFO] [ARP] Sending gratuitous ARP for 10.4.0.107 (52:54:00:12:34:5
6)
2024-11-26T14:36:24-00:00: [INFO] [ipv6] IP6: Starting
...skipping...
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Established  - Send_fin(113678529) -> Fin_wa_)
2024-11-26T14:36:34-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 20 paylo 0
2024-11-26T14:36:34-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 20 paylo 0
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.window] sequence validation: seq=87147557 range=871477e
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] finwait2timer 10000000000
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Fin_wait_1(113678529)  - Recv_ack(113678530)>)
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] timewait 2000000000
2024-11-26T14:36:34-00:00: [DEBUG] [tcp.state] 0 Fin_wait_2(0)  - Recv_fin -> Time_wait
2024-11-26T14:36:34-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 20 paylo 0
2024-11-26T14:36:35-00:00: [DEBUG] [tcp.tcptimer] timerloop: stoptimer
2024-11-26T14:36:36-00:00: [DEBUG] [tcp.state] timewait on_close
2024-11-26T14:36:36-00:00: [DEBUG] [tcp.pcb] removing pcb from connection tables: [channels=1i]
2024-11-26T14:36:36-00:00: [DEBUG] [tcp.pcb] removed remote 10.4.0.1,38936 to local 10.4.0.10 s
2024-11-26T14:36:44-00:00: [DEBUG] [tcp.state] finwait2timer: Closed

Looking at logs, we can see the ip of the unikernel to be 10.4.0.107 and we can interact with it:

echo -n hello tcp world | nc -nw1 10.4.0.107 8080

Furthermore, as @Firobe mentioned, you might want to take a look at hvt/spt targets of MirageOS. Both of these targets are supported from urunc , due to Rumprun, but please make sure to use the mirage branch of urunc. In that context, after we build the network example of MirageOS we can package it for hvt execution with the following Containeffile:

#syntax=harbor.nbfc.io/nubificus/pun:latest
FROM scratch

COPY network.hvt /unikernel/network.hvt

LABEL "com.urunc.unikernel.binary"="/unikernel/network.hvt"
LABEL "com.urunc.unikernel.cmdline"="-l *:debug"
LABEL "com.urunc.unikernel.unikernelType"="mirage"
LABEL "com.urunc.unikernel.hypervisor"="hvt"

We can build the image as previously with:

$ sudo docker build -f Containerfile -t harbor.nbfc.io/nubificus/urunc/network-mirage-hvt:test . 

and we can run it with the exact same command as previosuly:

$ sudo nerdctl run --rm -it  --runtime io.containerd.urunc.v2 harbor.nbfc.io/nubificus/urunc/network-mirage-hvt:test
            |      ___|
  __|  _ \  |  _ \ __ \
\__ \ (   | | (   |  ) |
____/\___/ _|\___/____/
Solo5: Bindings version v0.9.0
Solo5: Memory map: 256 MB addressable:
Solo5:   reserved @ (0x0 - 0xfffff)
Solo5:       text @ (0x100000 - 0x245fff)
Solo5:     rodata @ (0x246000 - 0x290fff)
Solo5:       data @ (0x291000 - 0x39afff)
Solo5:       heap >= 0x39b000 < stack < 0x10000000
2024-11-26T14:50:50-00:00: [INFO] [netif] Plugging into service with mac e2:f8:29:d9:74:5e mtu 
1500
2024-11-26T14:50:50-00:00: [INFO] [ethernet] Connected Ethernet interface e2:f8:29:d9:74:5e
2024-11-26T14:50:50-00:00: [INFO] [ARP] Sending gratuitous ARP for 10.4.0.108 (e2:f8:29:d9:74:5
e)
2024-11-26T14:50:50-00:00: [INFO] [ipv6] IP6: Starting
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] ND6: Sending RS
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] ND6: Sending NS src=:: dst=ff02::1:ffd9:745e tgt=fe8
0::e0f8:29ff:fed9:745e
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:50-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:50-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] ND: Received NS: src=:: dst=ff02::1:ffd9:745e tgt=fe
80::e0f8:29ff:fed9:745e
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:50-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:50-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:50-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:50-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:50-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] ND6: Unsupported ND option in RA: ty=14 len=1
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] ND: Received NS: src=:: dst=ff02::1:ffac:399c tgt=fe
80::a05a:75ff:feac:399c
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:51-00:00: [INFO] [ndpc6] ND6: Unsupported ND option in RA: ty=14 len=1
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] ND: Received NS: src=:: dst=ff02::1:ff5a:35e9 tgt=fe
80::d0dc:5cff:fe5a:35e9
2024-11-26T14:50:51-00:00: [DEBUG] [ndpc6] SLAAC: fe80::e0f8:29ff:fed9:745e --> PREFERRED
2024-11-26T14:50:51-00:00: [INFO] [ipv6] IP6: Started with fe80::e0f8:29ff:fed9:745e
2024-11-26T14:50:51-00:00: [INFO] [tcp.pcb] TCP layer connected on 10.4.0.108/24, fe80::e0f8:29
ff:fed9:745e/64
2024-11-26T14:50:51-00:00: [INFO] [udp] UDP layer connected on 10.4.0.108/24, fe80::e0f8:29ff:f
ed9:745e/64
2024-11-26T14:50:51-00:00: [INFO] [tcpip-stack-direct] Dual TCP/IP stack assembled: mac=e2:f8:2
9:d9:74:5e,ip=10.4.0.108/24, fe80::e0f8:29ff:fed9:745e/64
2024-11-26T14:50:51-00:00: [DEBUG] [tcpip-stack-direct] Establishing or updating listener for s
tack mac=e2:f8:29:d9:74:5e,ip=10.4.0.108/24, fe80::e0f8:29ff:fed9:745e/64
2024-11-26T14:50:51-00:00: [DEBUG] [tcpip-stack-direct] Establishing or updating listener for s
tack mac=e2:f8:29:d9:74:5e,ip=10.4.0.108/24, fe80::e0f8:29ff:fed9:745e/64
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:52-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:52-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:50:53-00:00: [DEBUG] [ndpc6] IP6: Processing HOPOPT header
2024-11-26T14:50:53-00:00: [INFO] [ndpc6] IP6: Processing unknown option, MSB 5
2024-11-26T14:50:53-00:00: [DEBUG] [ndpc6] IP6: Processing PADN option
2024-11-26T14:50:53-00:00: [INFO] [ndpc6] ICMP6: Unknown packet type: ty=143
2024-11-26T14:51:20-00:00: [DEBUG] [ARP] replying to ARP request for 10.4.0.108 from 10.4.0.1 (
mac 9a:59:56:c2:5a:ae)
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.pcb] process-syn: [channels=0 listens=0 connects=0]
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.pcb] new-server-connection: [channels=0 listens=0 conne
cts=0]
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Closed  - Passive_open -> Listen
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Listen  - Send_synack(4124807740) -> Syn_rcvd(
4124807740)
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.tcptimer] timerloop
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.tcptimer] timerloop: sleeping for 667000000 ns
2024-11-26T14:51:20-00:00: [DEBUG] [ARP] ignoring ARP request for 10.4.0.1 from 10.4.0.108 (mac
 e2:f8:29:d9:74:5e)
2024-11-26T14:51:20-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 28 payload
 len 0, needed_bytes 48
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.pcb] process-ack: [channels=0 listens=1 connects=0]
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.window] sequence validation: seq=159316661 range=159316
661[262140] res=true
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Syn_rcvd(4124807740)  - Recv_ack(4124807741) -
> Established
2024-11-26T14:51:20-00:00: [INFO] [application] new tcp connection from IP 10.4.0.1 on port 356
82
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.window] sequence validation: seq=159316661 range=159316
661[262140] res=true
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Established  - Recv_ack(4124807741) -> Establi
shed
2024-11-26T14:51:20-00:00: [DEBUG] [application] read: 15 bytes:
hello tcp world
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.pcb] Closing connection remote 10.4.0.1,35682 to local
10.4.0.108, 8080
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Established  - Send_fin(4124807741) -> Fin_wait_1(4124807741)
2024-11-26T14:51:20-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 20 payload len 0, needed_bytes 40
2024-11-26T14:51:20-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 20 payload len 0, needed_bytes 40
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.window] sequence validation: seq=159316676 range=159316676[262140] res=true
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] finwait2timer 10000000000
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Fin_wait_1(4124807741)  - Recv_ack(4124807742) -> Fin_wait_2(0)
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] timewait 2000000000
2024-11-26T14:51:20-00:00: [DEBUG] [tcp.state] 0 Fin_wait_2(0)  - Recv_fin -> Time_wait
2024-11-26T14:51:20-00:00: [DEBUG] [ipv4] ip write: mtu is 1500, hdr_len is 20, size 20 payload len 0, needed_bytes 40
2024-11-26T14:51:21-00:00: [DEBUG] [tcp.tcptimer] timerloop: stoptimer
2024-11-26T14:51:22-00:00: [DEBUG] [tcp.state] timewait on_close
2024-11-26T14:51:22-00:00: [DEBUG] [tcp.pcb] removing pcb from connection tables: [channels=1 listens=0 connects=0]
2024-11-26T14:51:22-00:00: [DEBUG] [tcp.pcb] removed remote 10.4.0.1,35682 to local 10.4.0.108, 8080 from active channels

Similarly, we interact with it:

echo -n hello tcp world | nc -nw1 10.4.0.108 8080

We are also able to deploy the MirageOS unikernel in k3s with the following yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: network-mirage-qemu-deployment
  labels:
    app: network-mirage-qemu
spec:
  replicas: 1
  selector:
    matchLabels:
      app: network-mirage-qemu
  template:
    metadata:
      labels:
        app: network-mirage-qemu
    spec:
      runtimeClassName: urunc
      containers:
      - name: network-mirage-qemu
        image: harbor.nbfc.io/nubificus/urunc/network-mirage-qemu:test
        ports:
        - containerPort: 8080
        resources:
          limits:
            memory: "64M"

and it gets deployed succesfuly:

$ kubectl get pods
NAME                                              READY   STATUS    RESTARTS   AGE
network-mirage-qemu-deployment-5d845df9f7-jxtcj   1/1     Running   0          3m4s

We can also get its ip address:

$ kubectl describe pods network-mirage-qemu-deployment-5d845df9f7-jxtcj | grep IP
                     cni.projectcalico.org/podIP: 10.244.58.169/32
                     cni.projectcalico.org/podIPs: 10.244.58.169/32
IP:                  10.244.58.169
IPs:
  IP:           10.244.58.169

and interract with it:

$ ping 10.244.58.169
PING 10.244.58.169 (10.244.58.169) 56(84) bytes of data.
64 bytes from 10.244.58.169: icmp_seq=1 ttl=64 time=0.165 ms
64 bytes from 10.244.58.169: icmp_seq=2 ttl=64 time=0.129 ms
64 bytes from 10.244.58.169: icmp_seq=3 ttl=64 time=0.157 ms
^C
--- 10.244.58.169 ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2042ms
rtt min/avg/max/mdev = 0.129/0.150/0.165/0.015 ms

@Firobe We are very happy that you will try to create urunc-compatible images for MirageOS. The next unikernel we add support for will be MirageOS and we can support hvt, spt, virtio and probably the qemu target too. Please let us know if you need any help regarding urunc-compatible images.