search

rust-osdev / x86_64

Library to program x86_64 hardware.
https://docs.rs/x86_64
Apache License 2.0
797 stars 132 forks source link

add write_pcid_no_flush #472

Closed Freax13 closed 8 months ago

Freax13 commented 8 months ago

If Cr4.PCID is set and the 63rd bit is clear when moving into Cr3, the processor flushes all TLB entries for the given PCID (Intel SDM, Volume 3, 4.10.4.1 and AMD APM, Volume 2, 5.5.1). This is similar to the operation without PCID in which the processor also flushes all TLB entries every time a new value is moved into Cr3. Given that the entire idea behind PCIDs is to not flush all entries when Cr3 is changed, we should also provide a write method that sets the 63rd bit when moving into Cr3. This PR adds write_pcid_no_flush to do just that.

A very quick and dirty comparison between write_pcid and write_pcid_no_flush showed a 10% 25+% performance improvement.

It could be argued that setting the 63rd bit should be the default, however, I'm a bit concerned that users rely on the flushing behavior (either intentionally or unintentionally) and so suddenly changing to non-flushing behavior could break code (This seems to be the case for a codebase I'm working on). In a future breaking release, we may want to change write_pcid to take an additional parameter specifying whether flushing behavior is intended.

phil-opp commented 8 months ago

Took me a bit to find this behavior in the Intel/AMD manuals because it is not mentioned in the CR3 section. The relevant info can be found in section 5.5.1 in the AMD manual:

The current PCID is the value in CR3[11:0]. When PCIDs are enabled the system software can store 12-bit Process Context Identifiers in CR3 for different address spaces. Subsequently, when system software switches address spaces (by writing the page table base pointer in CR3[62:12]), the processor may use TLB mappings previously stored for that address space and PCID, providing that bit 63 of the source operand is set to 1. If bit 63 is set to 0, the legacy behavior of a move to CR3 is maintained, invalidating TLB entries but only non-global entries for the specified PCID. Note that this bit is not stored in the CR3 register itself.

(Emphasis mine)

For Intel, the behavior is documented in section 4.10.4.1:

  • MOV to CR3. The behavior of the instruction depends on the value of CR4.PCIDE:
    • If CR4.PCIDE = 0, the instruction invalidates all TLB entries associated with PCID 000H except those for global pages. It also invalidates all entries in all paging-structure caches associated with PCID 000H.
    • If CR4.PCIDE = 1 and bit 63 of the instruction’s source operand is 0, the instruction invalidates all TLB entries associated with the PCID specified in bits 11:0 of the instruction’s source operand except those for global pages. It also invalidates all entries in all paging-structure caches associated with that PCID. It is not required to invalidate entries in the TLBs and paging-structure caches that are associated with other PCIDs.
    • If CR4.PCIDE = 1 and bit 63 of the instruction’s source operand is 1, the instruction is not required to invalidate any TLB entries or entries in paging-structure caches.