Remove Dependency from the `Virsh`

[hardikdr]

Summary

This is an issue to discuss and explore the feasibility of removing the dependency on the virsh CLI in the implementation of Machine Exec functionality.

One such alternative is the DomainSerialConsole() API provided by DigitalOcean's go-libvirt library. However, this API is unidirectional, which might not fully cater to our requirements. The potential helper there is using DomainSendKeys for sending KeyCodes, but this approach seems to be cumbersome, keyboard-type/os-specific, especially for the start.

The purpose here is to initiate a discussion and research into the available options for the future.

Motivation

Better maintainability, and less dependency on the host.

[lukas016]

we can use psuedo TTY directly with Character Device File (c), which is automatically created for all machines and path is defined in domain /dev/pts/0.

    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
      <address type='pci' domain='0x0000' bus='0x20' slot='0x01' function='0x0'/>
    </console>

[lukas016]

For more options as TCP, unix socket, UDP pls read docs: https://libvirt.org/formatdomain.html#host-interface

[so-sahu]

Possible Solutions for Bidirectional Communication

1. Pseudo TTY (pty):

   • Description: Allocates a Pseudo TTY using /dev/ptmx, allowing local console access.
   • Pros: Standard way for local console access.
   • Cons: May require additional handling for client-server communication.

2. TCP client/server:

   • Description: Acts as a TCP client connecting to a remote server or as a TCP server waiting for a client connection.
   • Pros: Enables network-based communication.
   • Cons: Requires network setup and security considerations.

3. UNIX domain socket client/server:

   • Description: Acts as a UNIX domain socket server, accepting connections from local clients.
   • Pros: Provides local communication without network overhead.
   • Cons: Limited to communication within the same host.

4. Named pipe:

   • Description: Writes output to a named pipe (FIFO file) for inter-process communication.
   • Pros: Simple to implement for local communication.
   • Cons: Limited to communication within the same host.

5. Spice channel:

   • Description: Accesses the character device through a Spice connection under a specified channel name.
   • Pros: Integrates with Spice for remote access and additional features.
   • Cons: Requires Spice support and more complex setup.

6. Nmdm device (FreeBSD):

   • Description: Provides two tty devices connected together by a virtual null modem cable.
   • Pros: Simulates a direct serial connection between two endpoints.
   • Cons: Limited to FreeBSD and specific device driver support.

Suggested Solution:

I recommend moving forward with the tty option because we already have <serial type="pty"> in our existing code, which can be easily leveraged. As mentioned, we may need to implement some functionality optimizations that we currently rely on the virsh tool for.

[lukas016]

i would like choose solution which will work with virsh. Our ops guys use this tool a lot. Currently i prefer unix socket or pseudo tty. Ideally solution hasn't dependency to libvirtd daemon and it can work without it.

[so-sahu]

Data Flow and Dependencies on libvirtd Service for Console Connections

When it comes to data flow and dependencies on the libvirtd service for console connections, there are key differences between using a pseudo-terminal (pty) and a Unix domain socket.

Using a Pseudo-Terminal (pty):

• Data Flow:

  • Your application interacts with the pty master, sending input and receiving output as if it were interacting with a physical terminal.
  • The pty master communicates with the pty slave, which is connected to the VM's console.
  • The libvirtd service is not directly involved in the data flow between your application and the VM's console when using a pty.

• Dependency on libvirtd Service:

  • If the libvirtd service fails or stops, the existing pty connections to VM consoles should continue to work.
  • However, you may not be able to establish new pty connections or perform management operations on VMs through libvirt until the service is restored.

Using a Unix Domain Socket:

• Data Flow:

  • Your application communicates directly with the Unix domain socket file, which is managed by libvirtd.
  • libvirtd forwards the data between the Unix domain socket and the VM's console.

• Dependency on libvirtd Service:

  • If the libvirtd service fails or stops, the Unix domain socket connections to VM consoles will be disrupted.
  • You will not be able to establish new Unix domain socket connections or perform management operations on VMs through libvirt until the service is restored.

In summary, when using a pty for console connections, the libvirtd service is not directly involved in the data flow, and existing connections should continue to work if the service fails. However, when using a Unix domain socket, libvirtd manages the socket, and a failure of the service will disrupt console connections until it is restored.

[lukas016]

pty looks better for me. I have little problem with few words:

If the libvirtd service fails or stops, the existing pty connections to VM consoles should continue to work. However, you may not be able to establish new pty connections or perform management operations on VMs through libvirt until the service is restored.

Both cases be very easy tested and i want to know, how it will work. Primary on one side you said libvirt isn't directly involved into communication on other side you said maybe i cannot connect to tty withot working libvirtd.

It doesn't sound very clear for me. I recommend do real test, it can be very easy simulated. If you don't know how, pls contact me and i will show

[so-sahu]

Tested the behavior of pty connections when the libvirtd service fails or stops. Below are the results:

1. Took the pty console -> Stopped the libvirtd service -> The pty console is still there: It shows, even after stopping the libvirtd service, the existing pty connections to VM consoles remained functional. This aligns with the understanding that libvirtd is not directly involved in the data flow for pty connections.

2. Stopped the libvirtd service -> Able to take pty console of a machine: Similarly, I was able to establish new pty connections and take pty consoles of VMs even when the libvirtd service was stopped. This indicates that libvirtd does not seem to be a prerequisite for establishing or maintaining pty connections.

Based on these tests and observations, it appears that pty is indeed a reliable choice for our requirements, as it allows for continued operation even when the libvirtd service is not available.