search

siderolabs / talos

Talos Linux is a modern Linux distribution built for Kubernetes.
https://www.talos.dev
Mozilla Public License 2.0
6.45k stars 514 forks source link

QEMU Guest Agent integration #3630

Closed sarkoziadam closed 3 years ago

sarkoziadam commented 3 years ago

Feature Request

Description

Hello,

Is it possible to integrate the QEMU Guest Agent into the OS?

smira commented 3 years ago

It should work if deployed as a daemonset in Kubernetes

sarkoziadam commented 3 years ago

Hello,

Thanks for the idea. I’m still a beginner in the world of Kubernetes, I found a script that works with it.

Ref: https://github.com/openshift/okd/issues/126#issuecomment-619446932 Only the image need to replaced from: registry.access.redhat.com/rhev4/ovirt-guest-agent to: eleh/qemu-guest-agent

andrewrynhard commented 3 years ago

I wonder if we should document this

Y0ngg4n commented 2 years ago

@smira so qemu-guest-agent is allready deployed? So i can use it for example with proxmox?

smira commented 2 years ago

@smira so qemu-guest-agent is allready deployed? So i can use it for example with proxmox?

qemu-guest-agent is not part of Talos itself, but you could deploy it as a daemonset as described above

sarkoziadam commented 2 years ago

@Y0ngg4n check my site: https://learn.sarkoziadam.hu/learn-kontenerizacio/kubernetes/pods/daemonsets/qemu-guest-agent-kubernetes

reitermarkus commented 2 years ago

While I am able to run it as a daemonset, I'm not quite sure how to get it work fully, i.e. I am not able to shut down or reboot the VMs via Proxmox anymore, since there is no /sbin/shutdown on Talos. When trying to run /sbin/shutdown in a pod, it says it doesn't work from within chroot.

How do you shut down or reboot a node from within a pod?

reitermarkus commented 2 years ago

Okay, so after some experimentation, with the eleh/qemu-guest-agent image, neither of shutdown, poweroff, halt or reboot work, all just output Running in chroot, ignoring request..

I then tried the linuxkit/qemu-ga:v0.7 image, which is Alpine-based and does not have shutdown. poweroff and halt did not do anything, however reboot actually shut down the node, i.e. Talos says shutdown via SIGTERM received. So reboot actually behaves like kill -TERM 1 I guess.

That leaves the question, how do you actually reboot a node?

smira commented 2 years ago

Talos has APIs to perform host-level actions. In fact in master we have /sbin/shutdown on the host which talks to the machined API to initiate the shutdown.

Talos also acts on the ACPI power event.

reitermarkus commented 2 years ago

In fact in master we have /sbin/shutdown on the host

Good to know.

Talos also acts on the ACPI power event.

It does in fact do that. However, once I enable the “QEMU Guest Agent” option for a VM in Proxmox, Proxmox will only use QEMU commands, not ACPI.

windowsrefund commented 1 year ago

@Y0ngg4n check my site: https://learn.sarkoziadam.hu/learn-kontenerizacio/kubernetes/pods/daemonsets/qemu-guest-agent-kubernetes

You may want to format the yaml.

jabbas commented 1 year ago

Thanks for idea @sarkoziadam! Basing on your manifest i've created one that is using alpine, so the size is as small as possible:

https://gist.github.com/jabbas/cc97ee0430c5c537c9b440263d7aa9bb

crisobal commented 1 year ago

I created a container which can be run as daemonset and uses talosctl to shutdown / reboot hosts.

https://github.com/crisobal/qemu-guest-agent-talos

jabbas commented 1 year ago

I think the best way to implement that is using talos plugins (or whatever it's called), like the iscsi one, but i didn't have time to poke around yet.

reitermarkus commented 1 year ago

In fact in master we have /sbin/shutdown on the host which talks to the machined API to initiate the shutdown.

@smira, I remember checking where this was in master when you first mentioned this and couldn't find it. There seems to be a /sbin/poweroff now, is this what you meant?

If it is, how would you call this from within a pod? I tried mounting it, and it gets stuck after 

/ # /sbin2/poweroff
2023/06/20 00:05:07 limited GOMAXPROCS to 4
2023/06/20 00:05:07 waiting 1 second(s) for USB storage
2023/06/20 00:05:08 initialize sequence: 11 phase(s)
2023/06/20 00:05:08 phase logger (1/11): 1 tasks(s)
2023/06/20 00:05:08 task setupLogger (1/1): starting

I assume it also needs some config from the host mounted, not sure what exactly though.

frezbo commented 1 year ago

In fact in master we have /sbin/shutdown on the host which talks to the machined API to initiate the shutdown.

@smira, I remember checking where this was in master when you first mentioned this and couldn't find it. There seems to be a /sbin/poweroff now, is this what you meant?

If it is, how would you call this from within a pod? I tried mounting it, and it gets stuck after

/ # /sbin2/poweroff
2023/06/20 00:05:07 limited GOMAXPROCS to 4
2023/06/20 00:05:07 waiting 1 second(s) for USB storage
2023/06/20 00:05:08 initialize sequence: 11 phase(s)
2023/06/20 00:05:08 phase logger (1/11): 1 tasks(s)
2023/06/20 00:05:08 task setupLogger (1/1): starting

I assume it also needs some config from the host mounted, not sure what exactly though.

you can check the nut-client extension that calls /sbin/poweroff https://github.com/siderolabs/extensions/blob/main/power/nut-client/nut-client.yaml#L40