Add CSRs to the hypervisor extension for more trap values, version 2

[jhauser-us]

This issue is a variation on #394. An earlier version of this proposal was #424.

There are reasons to suspect that hypervisor performance could substantially benefit if trap handlers received more information about page faults caused by guest-physical address translation (second-level translation).

To test this hypothesis, I propose adding two hypervisor CSRs, htval and htinst, plus two machine-level CSRs, mtval2 and mtinst. The abbreviation "tinst" is short for "trap instruction". Although the hypervisor extension would require all four CSRs to exist, a minimal implementation would be free to hardwire all of them to zeros. Like mtval and stval, htval and htinst must be written automatically on any trap into HS mode, and mtval2 and mtinst must be written on any trap into M mode.

When a guest-page fault trap is taken into HS mode, htval is automatically written either with zero or with the guest physical address that faulted, shifted right by 2 bits. For any other trap into HS mode, htval must be written with zero.

When a trap is taken into HS mode, htinst is automatically written with either

• zero;
• a transformation of the trapping instruction; or
• a special pseudoinstruction.

Except when a pseudoinstruction is required (described later), the value assigned to htinst may always be zero, indicating that the hardware isn't providing the transformed trapping instruction for this particular trap. For interrupts, and for the following synchronous exceptions, the value automatically written to htinst must be zero:

Instruction address misaligned
Instruction access fault
Illegal instruction
Breakpoint
Environment call
Instruction page fault

For these synchronous exceptions,

Load or store/AMO address misaligned
Load or store/AMO access fault
Load or store/AMO page fault

htinst may be automatically written with zero or with a transformation of the trapping instruction. If the trapping instruction is not a compressed instruction (it is 32-bit size or larger), the transformed instruction that may be written to htinst is as follows:

• If the trapping instruction is one of LB, LBU, LH, LHU, LW, LWU, LD, FLW, FLD, or FLQ, the transformed instruction has this structure:

  bits top:20   zeros
  bits 19:15    stval value minus the original virtual address
  bits 14:0     bits 14:0 of the trapping instruction
                  (fields funct3, rd, opcode)

• If the trapping instruction is one of SB, SH, SW, SD, FSW, FSD, or FSQ, the transformed instruction has this structure:

  bits top:25   zeros
  bits 24:20    bits 24:20 of the trapping instruction (rs2)
  bits 19:15    stval value minus the original virtual address
  bits 14:12    bits 14:12 of the trapping instruction (funct3)
  bits 11:7     zeros
  bits 6:0      bits 6:0 of the trapping instruction (opcode)

• If the trapping instruction is one of LR.x, SC.x, or AMO*, the transformed instruction has this structure:

  bits top:32   zeros
  bits 31:20    bits 31:20 of the trapping instruction
  bits 19:15    stval value minus the original virtual address
  bits 14:0     bits 14:0 of the trapping instruction

Observe that in all cases the instruction's rs1 field (bits 19:15) is replaced by the difference between the faulting virtual address (written to stval) and the original virtual address. This can be nonzero only for an access or page fault that occurs for a misaligned memory access. Note also, for basic loads and stores, the instruction's immediate offset is replaced by zero bits.

If the trapping instruction is a compressed instruction (16-bit size), the transformed instruction that may be written to htinst is found as follows:

1. Expand the compressed instruction to its 32-bit equivalent.
2. Transform as above.
3. Replace bit 1 with a 0.

Bits 1:0 of a transformed instruction in htinst will be 01 if the trapping instruction is compressed and 11 if not.

Traps for guest-page faults operate like regular page faults, except a special case applies. If a trap into HS mode is a guest-page fault, and htval is given a nonzero value (the faulting guest physical address), and furthermore if the guest memory access was for VS-level address translation, then htinst must be written with one of these pseudoinstructions:

0x00002000 = a 32-bit read for VS-level address translation (RV32)
0x00002020 = a 32-bit write for VS-level address translation (RV32)
0x00003000 = a 64-bit read for VS-level address translation (RV64)
0x00003020 = a 64-bit write for VS-level address translation (RV64)

For instruction guest-page faults, htinst is automatically written either with one of these pseudoinstructions or with zero. (No actual instruction is available, same as for regular instruction page faults.) For load or store/AMO guest-page faults, htinst may be written with a pseudoinstruction, with zero, or with the same transformed trapping instruction as a regular page fault. Pseudoinstructions in htinst are distinguished by having bits 1:0 equal to 00.

Note that, if an implementation uses htval to supply the faulting guest physical address to a trap handler, it cannot hardwire htinst entirely to zero. However, it may choose to support only values 0 and 0x00002000/0x00003000, and possibly 0x0x00002020/0x00003020. (A "write" pseudoinstruction, 0x0x00002020 or 0x00003020, is needed only if the hardware may automatically update bits A and D in VS-level page tables, else it won't arise.)

With these new CSRs, a trap handler for guest-page faults will have this information available to it:

• in stval, the original virtual address;
• in htval, if nonzero, the faulting guest physical address (shifted right by 2 bits); and
• in htinst, either a special pseudoinstruction indicating that the fault occurred due to a read or write by VS-level address translation, or, if nonzero, a transformation of the faulting instruction.

When maximally implemented, this should facilitate both the handling of guest-physical page faults and the emulation of I/O devices for a guest virtual machine.

Machine-level CSRs mtval2 and mtinst are of course defined identically to htval and htinst, except for M mode.

[avpatel]

Overall this approach looks good and it is much more detailed.

The existing instruction matching in OpenSBI, Xvisor and KVM can be easily adapted to the non-zero transformed value stored in htinst/mtinst CSR.

[fintelia]

Is it intentional that hinst and htval are written during traps from U-mode -> HS-mode? If so, shouldn't they start with s- prefixes instead?

[yyshen]

My understanding is that the hypervisor in HS-mode can fetch the faulting instruction (if it is not a instruction page fault) by setting the hstatus.sprv bit to 1 and performing a load with the faulting instruction address. Is that correct? If so, decoding the faulting instruction is relatively straightforward.

[avpatel]

My understanding is that the hypervisor in HS-mode can fetch the faulting instruction (if it is not a instruction page fault) by setting the hstatus.sprv bit to 1 and performing a load with the faulting instruction address. Is that correct? If so, decoding the faulting instruction is relatively straightforward.

Please refer previous discussion at https://github.com/riscv/riscv-isa-manual/issues/394

[jhauser-us]

Jonathan Behrens wrote:

Is it intentional that hinst and htval are written during traps from U-mode -> HS-mode? If so, shouldn't they start with s- prefixes instead?

We are proposing those CSRs for HS mode; they wouldn't exist for ordinary (non-hypervisor) S mode. As such, they are encoded in the space of hypervisor CSRs and have h- names the same as other hypervisor CSRs.

Yes, the fact that htinst may be written with a nonzero value on traps from U mode to HS mode (or from HS mode to itself) means an OS running in HS mode could use the information for traps that have no connection to any guest running in a virtual machine. If that additional utility bothers people, we could suppress it by saying htinst is written with a nonzero value only on transitions from VS or VU mode to HS mode. But I see nothing to be gained from ensuring HS mode gets no superfluous extra advantages over regular S mode.

[avpatel]

Any updates on this ? Can we have a PR for this ?

[jhauser-us]

Adopted with commit 899457caf7a9cdcb59a7d57421d249512b00de94.