Virtual GIC Design

[pengzechen]

Part 1: Virtual GIC Configuration

1. vgicd-ctrl reg:

For cases 1, 2, and 3, IPI communication is not required. Case 4 requires IPI communication.

2. vgicd-iid reg, vgicd-type reg :

These save some attributes of the GIC and the number of PE (processing elements).

• The vgicd provided by virtualization should be configured according to the number of VPEs.

3. vgicd-isenable reg

getenable: Directly read the content from the structure.

setenable: Set the GIC according to the vtop and ptov settings. For cases 1, 2, and 3, IPI communication is not needed. Case 4 requires IPI communication.

4. other emu reg

Other vgicd-emu registers is similar to isenabler.

Part 2: Vint Handle

0-15 (sgi): Case 2 does not need IPI, Case 4 requires IPI.

16-1020 (hw): Case 2 does not need IPI, Case 4 requires IPI.

Processing Flow:

1. An interrupt causes the execution flow to return from vcpu.run(), carrying exit information. According to the current interrupt handling architecture, we need to handle the IRQ at the outermost level, as shown in the diagram:

• Here, separating host and guest interrupts: In Case 4 of the diagram, if the active VPE is vpe5 and the interrupt belongs to vm5, the vgic of vm5 needs to be called for handling. In the app, you can call get_vm_by_id(). The app can depend on axhal to access GIC resources and other resources.
• Then call vm.inject() to inject the interrupt into the virtual machine.

​2.Implement an int_inject function in the VM to handle physical vectors. Pseudocode is shown in the diagram.

Part 3: Problems to Solve

1. How should IPI communication be designed?
2. Why isn’t VGIC placed in Vec<> like other devices?

[luodeb]

Problem 1: Acquire the Vcpu when implement Vgicd. (eg: gicd_isenabler)

Solution 1: Crate Interface

let curr_vcpu_id = call_interface!(VcpuIf::current_vcpu_id());
error!("current vcpu id: {}", curr_vcpu_id);

Solution 2: Vcpu dyn trait

Passes the vCPU when the interrupt occurs, Before this, we deed define the VCpuIf trait and implement it in AxVCpu

[pengzechen]

More details about Vint Handle: Processing Flow(1): The most important is function find_vm_need_inject() , when pyh irq is not current vm find_vm_need_inject will find a vm that need to be injected.

1. Find current phy PE's all VPEs.
2. Find vms that contain these VPEs.
3. Check these vms if they register specific irq.

[pengzechen]

@luodeb find_vm_need_inject this function should be implented in umhv(app).

[luodeb]

Part 3 Software Generated Interrupts (SGI) Virtualization

Software Generated Interrupts (SGIs) are special types of interrupts that originate from software and are typically used to facilitate communication between cores in multi-core systems. The target CPU is specified by the sender, and SGIs can be routed to one or more cores.

In virtualized environments, due to multiple virtual CPUs (vCPUs) potentially sharing the same physical CPU, the hypervisor must virtualize SGIs to ensure isolation and transparency between virtual machines (VMs).

3.1 Hypervisor Interception of SGIs

In a virtualized setting, when a VM attempts to send an SGI, it usually does so by modifying the guest's Generic Interrupt Controller (GIC) related registers. The VM itself cannot directly access the physical GIC Distributor (GICD) registers, so these write operations are intercepted by the hypervisor.

• vCPU to vCPU SGI: When a VM sends an SGI to its own vCPU or to other vCPUs within the same VM.

3.2 Handling and Routing of SGIs

During SGI virtualization, the hypervisor is responsible for the following:

• Interception and Parsing of SGIs: When a VM writes to the GICD_SGIR register (used to trigger an SGI), the hypervisor intercepts this write operation. It parses out the target vCPU as well as the SGI ID.
• Redirection of SGIs: Based on the parsed SGI information, the hypervisor re-routes the SGI to the target vCPU. There should be an interface capable of injecting the specified interrupt based on the vcpu_id.
  • Active vCPU: The SGI is sent directly to the designated vCPU.
  • Sleeping vCPU: The vCPU is awakened before the SGI is sent to it.
• Management of Interrupt Priorities and States: The hypervisor needs to maintain the priority and state (such as pending, active, etc.) of virtual interrupts to ensure the interrupt behavior perceived by the VM matches that of physical hardware.

3.3 Support for Virtual GIC

To enable VMs to handle interrupts as if they were using a physical GIC, the hypervisor provides a virtual GIC interface (vGIC). The vGIC simulates the operations of the GICD and Generic Interrupt Controller Core (GICC) registers and maps these registers into the VM's address space.

Virtual GIC supports SGI management within the VM, including:

• Virtual GICD_SGIR Register: The VM uses this register to trigger an SGI, and the hypervisor redirects the interrupt upon receiving the write operation.
• Virtual Interrupt Routing Table: The hypervisor maintains a virtual interrupt routing table List Registers to track which vCPU the SGI should be routed to.

[luodeb]

Part 4 Privileged Peripheral Interrupts (PPI)

Privileged Peripheral Interrupts (PPIs) are typically used to manage processor-specific peripheral interrupts. In GICv2, each core has its dedicated PPI, usually including timer interrupts and other local peripheral interrupts. In a virtualized environment, the hypervisor must virtualize these interrupts to allow each VM transparent access and usage.

4.1 VM Initiating PPI Requests

When a vCPU within a VM needs to process a PPI, it is usually done through operations on the GIC registers. For example, a vCPU might read or clear the status of a particular PPI, an operation that requires interception by the hypervisor.

4.2 Hypervisor Intercepting Requests

• Intercepting Operations: The hypervisor monitors access requests to PPI-related registers, particularly the PPI registers of the GIC.
• Parsing Requests: The hypervisor parses the request to determine which vCPU or VM is performing the operation and processes these requests as needed.

4.3 Routing and Distribution of PPIs

• Managing PPI Status: The hypervisor needs to maintain the PPI status for each VM, ensuring that the state of the PPI correctly reflects on the corresponding vCPU while the VM is running.
• Interrupt Injection: When a PPI interrupt occurs, the hypervisor is responsible for injecting the interrupt into the appropriate vCPU. For PPIs, the hypervisor can directly send the interrupt request to the target vCPU without redirection.

4.4 Target vCPU Processing the Interrupt

• vCPU Processing PPI: Once the PPI is triggered, the target vCPU enters the interrupt handler and executes the corresponding processing logic.
• Status Restoration: After processing, the vCPU clears the PPI interrupt status and resumes its execution state.

4.5 Cleanup by the Hypervisor

• Status Management: After handling the PPI interrupt, the hypervisor is responsible for clearing the interrupt status and any associated registers, ensuring the correctness of subsequent interrupt requests.

[luodeb]

Part 5 Shared Peripheral Interrupts (SPI)

In a virtualized environment, SPI (Shared Peripheral Interrupt) is a type of interrupt designed to handle interrupts generated by peripherals shared among multiple processor cores. Unlike SGI and PPI, SPI pertains to interrupts for shared devices, allowing multiple CPUs to respond to interrupts generated by the same peripheral. When virtualizing SPI, the hypervisor must ensure isolation between VMs while providing correct interrupt management for shared peripherals.

SPI is typically used for peripherals within the system that can be accessed by multiple processors, such as network adapters and storage controllers. In GICv2, SPI is managed by the GIC Distributor (GICD), allowing multiple processor cores to receive interrupts from the same peripheral. In a virtualized environment, the hypervisor must virtualize SPIs in a form suitable for use by multiple VMs.

5.1 VM Initiating SPI Requests

When a peripheral generates an interrupt, it passes the SPI through the physical GIC to the corresponding processor cores. In a virtualized environment, the physical interrupt first reaches the hypervisor.

5.2 Hypervisor Interception and Management

• Intercepting Interrupts: The hypervisor intercepts SPI requests coming from peripherals and identifies the target VM (if mapped).
• Interrupt Mapping: The hypervisor maintains an interrupt mapping table that correlates physical interrupts with virtual interrupts within the VM.

5.3 Routing and Redirecting SPIs

• Identifying Target VM: The hypervisor determines, via the interrupt mapping table, which VM's vCPU the SPI should be routed to.
• Injecting Interrupt: The hypervisor injects the SPI into the corresponding vCPU of the target VM to trigger interrupt handling.

5.4 Target vCPU Processing the Interrupt

• Handling SPI Interrupt: Upon receiving the interrupt request, the target vCPU executes the corresponding interrupt handler. This process includes saving context, executing processing logic, and clearing the interrupt status.
• Restoring State: After processing, the vCPU returns to normal execution state and clears the interrupt flag.

5.5 Cleanup by the Hypervisor

• Updating Status: After handling the SPI interrupt, the hypervisor updates the interrupt status to reflect the current interrupt handling situation.

[luodeb]

Part 6 List Register

For a GIC with virtualization extensions, the hypervisor uses List Registers to maintain some context information about high-priority virtual interrupts.

struct gich_lr {
    uint32_t vid : 10;  // Virtual IRQ number
    uint32_t pid : 10;  // This field varies depending on the value of hw
                        // If hw=1, this indicates the virtual interrupt is associated with a physical interrupt, and pid is the actual physical IRQ number.
                        // If hw=0, bit19 indicates whether to signal EOI for maintenance interrupts (not discussed here).
                                 // Bits 12-10: If this is an SGI interrupt (i.e., virtual interrupt ID < 15), then this field represents the requesting CPU ID.

    uint32_t resv : 3;  // Reserved
    uint32_t pr : 5;    // Priority of the virtual interrupt
    uint32_t state : 2; // Indicates the state of the interrupt: invalid, pending, active, pending and active
    uint32_t grp1 : 1;  // Indicates whether the virtual interrupt is a Group 1 virtual interrupt
                        // 0 indicates this is a Group 0 virtual interrupt, representing a secure virtual interrupt, configurable to be sent to the vCPU as either a VIRQ or VFIRQ.
                        // 1 indicates this is a Group 1 virtual interrupt, representing a non-secure virtual interrupt, which triggers as a VIRQ rather than a VFIRQ.

    uint32_t hw : 1;    // Whether the virtual interrupt is associated with a hardware physical interrupt
                        // 0 indicates no, meaning it is triggered in software, and when deactivated, it will not notify the distributor.
                        // 1 indicates yes, meaning deactivating this virtual interrupt will also perform a deactivate operation on the corresponding physical interrupt.
                        // Specifically, the deactivate operation, if gicv_ctlr.eoimode=0, writing to the gicv_eoir register performs both a drop priority and deactivate operation simultaneously.
                        // If gicv_ctlr.eoimode=1, writing to the gicv_eoir register performs a drop priority, and writing to GICV_DIR performs a deactivate.
};

[hky1999]

Please move these comments to discussions. these can not be treated as Issues