lima-vm / lima

Linux virtual machines, with a focus on running containers
https://lima-vm.io/
Apache License 2.0
15.49k stars 608 forks source link

Monitor guest activity using the agent #835

Open afbjorklund opened 2 years ago

afbjorklund commented 2 years ago

Description

I am looking for a feature, where I can see the CPU usage and RAM usage of an instance. Preferrably disk usage too, if possible.

However, I could not find this to be easilly accessible in QEMU the same way that it is in VirtualBox or Docker or what have you ?

One workaround that I was considering earlier, would be to run a small program in the guest to output such information.

It would not be as accurate ("inside") as quering the hypervisor ("outside"), but it would be "good enough" for my purposes.


I made a proof-of-concept, using gopsutil. What you do you think about the idea ?

package main

import (
    "fmt"
    "path/filepath"
    "time"

    "github.com/shirou/gopsutil/v3/cpu"
    "github.com/shirou/gopsutil/v3/disk"
    "github.com/shirou/gopsutil/v3/host"
    "github.com/shirou/gopsutil/v3/mem"
)

func main() {
    _, _ = cpu.Percent(1*time.Second, true)
    h, _ := host.Info()
    p, _ := disk.Partitions(false)

    fmt.Printf("Host: %v\n", h.Hostname)

    mountpoint := "/"
    for _, part := range p {
        device, err := filepath.EvalSymlinks(part.Device)
        if err != nil {
            continue
        }
        if part.Fstype == "squashfs" { // skip /snap
            continue
        }
        fmt.Printf("%s %s\n", device, part.Mountpoint)
        if device == "/dev/vda1" {
            mountpoint = part.Mountpoint
        }
    }

    for {
        t := time.Now()
        c, _ := cpu.Percent(0, false)
        v, _ := mem.VirtualMemory()
        d, _ := disk.Usage(mountpoint)

        fmt.Printf("%v\n", t.Format(time.RFC3339Nano))
        fmt.Printf("Cpu: %.2f%%\n", c[0])
        fmt.Printf("Mem: %.2f%%\n", v.UsedPercent)
        fmt.Printf("Disk: %.2f%%\n", d.UsedPercent)

        time.Sleep(1 * time.Second)
    }
}

Probably would use JSON lines, rather than text. So that it can be parsed, and graphed, on the lima client side.

afbjorklund commented 2 years ago

The main reason to have this running in the agent, is that the cpu counters need to keep state from last time...

Otherwise there is a pause (it is 1 second in the above program), where it accumulates enough relative values.

AkihiroSuda commented 2 years ago

Will this be used for GUI dashboard or something similar?

afbjorklund commented 2 years ago

Will this be used for GUI dashboard or something similar?

That was the idea, but I guess you could view it in the console too

Something similar to the output you can get from e.g. lima vmstat 1

(Another workaround would be to run top, over the lima connection.)

But the main idea was for some graphs, like in https://virt-manager.org/


I think I will move the "host" info to a separate feature, from this one.

It grew a bit, from the original idea of just parsing /etc/os-release

afbjorklund commented 2 years ago

We used a command-line tool similar to this feature, for benchmarking minikube:

https://github.com/tstromberg/cstat

It was featured in one of the talks, on how to make Kubernetes start up faster:

https://kccnceu20.sched.com/event/Zerq/improving-the-performance-of-your-kubernetes-cluster-priya-wadhwa-google