enable memory management unit

isaac2-lee commented 1 year ago

make memory region on linker file (memory.x)
- .xlat_tables
make page translation table
- static ref MMU_PAGE
register setting changes:
- MAIR_EL2, TCR_EL2 (ref. tf-rmm)
- set HCR_EL2.E2H to 1
- set SCTLR_EL2.M to 1

isaac2-lee commented 1 year ago

Current : pause the system after setting the SCTLR_EL2 (refer branch/commit: 'dev/isaac2.lee/enable_mmu')

suspected points

physical address was wrong (why did old islet set the uart address)
not make translation table well (MMU_PAGE)
miss register settings such as ttbr1?

isaac2-lee commented 1 year ago

not make translation table well (MMU_PAGE)

making table process looks just wrong (* others should be checked)

below is from test with 512 pages I can check the overlapping address

[DEBUG]monitor::mm::page_table -- max enries: 512          
[DEBUG]monitor::mm::page_table -- page[0]: FDC16000     // LEVEL1
[DEBUG]monitor::mm::page_table -- max enries: 512
[DEBUG]monitor::mm::page_table -- page[224]: FDC17000 // LEVEL2
[DEBUG]monitor::mm::page_table -- max enries: 512
[DEBUG]monitor::mm::page_table -- page[160]: FDC00000 // LEVEL3
[DEBUG]monitor::mm::page_table -- page[161]: FDC01000
[DEBUG]monitor::mm::page_table -- page[162]: FDC02000
[DEBUG]monitor::mm::page_table -- page[163]: FDC03000
[DEBUG]monitor::mm::page_table -- page[164]: FDC04000
[DEBUG]monitor::mm::page_table -- page[165]: FDC05000
[DEBUG]monitor::mm::page_table -- page[166]: FDC06000
[DEBUG]monitor::mm::page_table -- page[167]: FDC07000
[DEBUG]monitor::mm::page_table -- page[168]: FDC08000
[DEBUG]monitor::mm::page_table -- page[169]: FDC09000
[DEBUG]monitor::mm::page_table -- page[170]: FDC0A000
[DEBUG]monitor::mm::page_table -- page[171]: FDC0B000
[DEBUG]monitor::mm::page_table -- page[172]: FDC0C000
[DEBUG]monitor::mm::page_table -- page[173]: FDC0D000
[DEBUG]monitor::mm::page_table -- page[174]: FDC0E000
[DEBUG]monitor::mm::page_table -- page[175]: FDC0F000
[DEBUG]monitor::mm::page_table -- page[176]: FDC10000
[DEBUG]monitor::mm::page_table -- page[177]: FDC11000
[DEBUG]monitor::mm::page_table -- page[178]: FDC12000
[DEBUG]monitor::mm::page_table -- page[179]: FDC13000
[DEBUG]monitor::mm::page_table -- page[180]: FDC14000
[DEBUG]monitor::mm::page_table -- page[181]: FDC15000
[DEBUG]monitor::mm::page_table -- page[182]: FDC16000
[DEBUG]monitor::mm::page_table -- page[183]: FDC17000
[DEBUG]monitor::mm::page_table -- page[184]: FDC18000
[DEBUG]monitor::mm::page_table -- page[185]: FDC19000
[DEBUG]monitor::mm::page_table -- page[186]: FDC1A000
....

it means I have to implement PageTranslation trait newly(translate address, mapping), not using old one.

isaac2-lee commented 1 year ago

최종 entry 에 입력될 때 RawPTE의 TABLE_OR_PAGE bit에 맞추어 주소가 set되고 있지 않은데 이것이 정상적인것인지 확인필요.. 임의로 bit를 올려 세팅시에 system hang 발생

mm/pte.rs

impl page_table::Entry for Entry {
...
    fn set(&mut self, addr: PhysAddr, flags: u64) {
        self.0
            .set(addr.as_u64() | flags) // <----------- ?
            .set_masked_value(RawPTE::SH, attr::shareable::INNER)
            .set_bits(RawPTE::AF)
            .set_bits(RawPTE::VALID);

        unsafe {
            core::arch::asm!(
                "dsb ishst",
                "dc civac, {}",
                "dsb ish",
                "isb",
                in(reg) &self.0 as *const _ as usize,
            );
        }

    fn set_with_page_table_flags(&mut self, addr: PhysAddr) {
        self.set(
            addr,  <-------------------------------- ?
            bits_in_reg(RawPTE::ATTR, attr::attribute::NORMAL) //?
                | bits_in_reg(RawPTE::TYPE, attr::page_type::TABLE_OR_PAGE),
        )
    }

isaac2-lee commented 1 year ago

[x] physical address was wrong (why did old islet set the uart address) -> in the table, inital physical address modified the rmm base address

let device_flags = helper::bits_in_reg(RawPTE::ATTR, attr::attribute::DEVICE_NGNRE)
    | helper::bits_in_reg(RawPTE::AP, attr::permission::RW);
let virt = Page::<BasePageSize, VirtualAddr>::range_with_size(
    VirtualAddr::from(PAGE_OFFSET),
    LARGE_PAGE_SIZE,
);
unsafe {
    let phys = Page::<BasePageSize, PhysAddr>::range_with_size(
        PhysAddr::from(rmm_base),
        LARGE_PAGE_SIZE,
    );
    root.set_pages(virt, phys, device_flags as u64);
}

[x] not make translation table well (MMU_PAGE) -> last commit looks good, IMO ref.
[ ] miss register settings such as ttbr1? -> ???

isaac2-lee commented 1 year ago

unit-test

check the unit test codes. on the previous unit-test, it was tested with only rmm/monitor when I checked rmm/armv9a path, it returned error such as below IMO, test developer can't fix this error and tested only rmm/montor path

error: the `#[alloc_error_handler]` in this crate conflicts with allocation error handler in: std

error: invalid register `x0`: unknown register
  --> rmm/armv9a/src/helper/mod.rs:82:9
   |
82 |         inlateout("x0") args[0] => ret[0],
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x1`: unknown register
  --> rmm/armv9a/src/helper/mod.rs:83:9
   |
83 |         inlateout("x1") args[1] => ret[1],
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x2`: unknown register
  --> rmm/armv9a/src/helper/mod.rs:84:9
   |
84 |         inlateout("x2") args[2] => ret[2],
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x3`: unknown register
  --> rmm/armv9a/src/helper/mod.rs:85:9
   |
85 |         inlateout("x3") args[3] => ret[3],
   |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x0`: unknown register
  --> rmm/armv9a/src/smc.rs:31:17
   |
31 |                 inlateout("x0") command => ret[0],
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x1`: unknown register
  --> rmm/armv9a/src/smc.rs:32:17
   |
32 |                 inlateout("x1") args[0] => ret[1],
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x2`: unknown register
  --> rmm/armv9a/src/smc.rs:33:17
   |
33 |                 inlateout("x2") args[1] => ret[2],
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x3`: unknown register
  --> rmm/armv9a/src/smc.rs:34:17
   |
34 |                 inlateout("x3") args[2] => ret[3],
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x4`: unknown register
  --> rmm/armv9a/src/smc.rs:35:17
   |
35 |                 inlateout("x4") args[3] => ret[4],
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: invalid register `x5`: unknown register
  --> rmm/armv9a/src/smc.rs:36:17
   |
36 |                 out("x5") ret[5],
   |                 ^^^^^^^^^^^^^^^^

error: invalid register `x6`: unknown register
  --> rmm/armv9a/src/smc.rs:37:17
   |
37 |                 out("x6") ret[6],
   |                 ^^^^^^^^^^^^^^^^

error: invalid register `x7`: unknown register
  --> rmm/armv9a/src/smc.rs:38:17
   |
38 |                 out("x7") ret[7],
   |                 ^^^^^^^^^^^^^^^^

error[E0152]: found duplicate lang item `panic_impl`
 --> rmm/armv9a/src/panic.rs:7:1
  |
7 | pub extern "C" fn panic_handler(_info: &core::panic::PanicInfo<'_>) -> ! {
  | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

isaac2-lee commented 1 year ago

Current page table descriptor look up following figure.

[Armv8-A Address Translation Version 1.1]

isaac2-lee commented 1 year ago

tf-rmm

TCR_EL2

    /* Recompute the value for TCR_EL2 */
    tcr = (uint64_t)t0sz << TCR_EL2_T0SZ_SHIFT;
    tcr |= (uint64_t)t1sz << TCR_EL2_T1SZ_SHIFT;

    /*
     * Set the cacheability and shareability attributes for memory
     * associated with translation table walks.
     */
    /* Inner & outer WBWA & shareable for both halfs. */
    tcr |= TCR_EL2_IRGN0_WBWA | TCR_EL2_ORGN0_WBWA | TCR_EL2_SH0_IS;
    tcr |= TCR_EL2_IRGN1_WBWA | TCR_EL2_ORGN1_WBWA | TCR_EL2_SH1_IS;

    /*
     * ASID and hierarchical permissions.
     */
    tcr |= TCR_EL2_AS | TCR_EL2_HPD0 | TCR_EL2_HPD1;

    /*
     * Granule size. Only 4K supported on both halfs.
     */
    tcr |= TCR_EL2_TG0_4K | TCR_EL2_TG1_4K;

accoding to specification, when HCR_EL2.E2H == 1, tcr register descriptor will be changed. tf-rmm is turned on HCR_EL2.E2H, islet don't. just follow up the IRGN0, ORGN0, SH0, HDP0.

isaac2-lee commented 1 year ago

MAIR_EL2 : tf-rmm

    mair = MAIR_ATTR_SET(ATTR_DEVICE, ATTR_DEVICE_INDEX);
    mair |= MAIR_ATTR_SET(ATTR_IWBWA_OWBWA_NTR, ATTR_IWBWA_OWBWA_NTR_INDEX);
    mair |= MAIR_ATTR_SET(ATTR_NON_CACHEABLE, ATTR_NON_CACHEABLE_INDEX);

00000000_01000100_00000100_11111111 (32bit)

일단은 tf-rmm 참고해서 세팅중

isaac2-lee commented 1 year ago

Current page table descriptor look up following figure.

Page table descriptor, rmm/page_table/entry.rs

define_bits!(
    PTDesc,
    Reserved[58 - 55],
    UXN[54 - 54],
    PXN[53 - 53],
    ADDR_BLK_L1[47 - 30],      // block descriptor; level 1
    ADDR_BLK_L2[47 - 21],      // block descriptor; level 2
    ADDR_TBL_OR_PAGE[47 - 12], // table descriptor(level 0-2) || page descriptor(level3)
    AF[10 - 10],               // access flag
    SH[9 - 8],                 // pte_shareable
    AP[7 - 6],                 // pte_access_perm
    NS[5 - 5],                 // security bit
    INDX[4 - 2],               // the index into the Memory Attribute Indirection Register MAIR_ELn
    TYPE[1 - 1],
    VALID[0 - 0]
);

attribute setting

    fn set(&mut self, addr: PhysAddr, flags: u64) {
        self.0
            .set(addr.as_u64() | flags)
            .set_masked_value(PTDesc::SH, attr::shareable::INNER)
            .set_bits(PTDesc::AF)
            .set_bits(PTDesc::VALID);
....

    fn set_with_page_table_flags(&mut self, addr: PhysAddr) {
        self.set(
            addr,
            bits_in_reg(PTDesc::TYPE, attr::page_type::TABLE_OR_PAGE),
        )
    }

in case of stage2_translation, it has one more bit setting bits_in_reg(RawPTE::ATTR, pte::attribute::NORMAL)

isaac2-lee commented 1 year ago

memory from EL3 should be set on NS[5-5] bit. in case of disable NS bit, can't access L3 table. (occured fault)

-> It's wrong analysis. NS bit should be not set

isaac2-lee commented 1 year ago

for log, UART address also should be set in page table.

islet has a uart physical address, 0x1c0c0000.

            self.set_pages(
                VirtAddr::from(uart_phys),
                PhysAddr::from(uart_phys),
                1,
                rw_flags,
            );

islet-project / islet