ACS: mm_unprotected_ipa_boundary testcase is failed

nook1208 commented 1 year ago

mm_unprotected_ipa_boundary testcase is failed because of Data abort in realm. let me check the issue in this page

// eom: mm_unprotected_ipa_boundary start
[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd RTT_INIT_RIPAS
[TRACE]islet_rmm::stat -- start Stat with command RTT_INIT_RIPAS
[TRACE]islet_rmm::rmi -- Dummy implementation.
[INFO]islet_rmm::stat -- RMI::RTT_INIT_RIPAS            TOTAL MemUsed:            0 byte, AVG MemUsed:          0 byte, CallCnt:     6418
[TRACE]islet_rmm::event -- RMI: RTT_INIT_RIPAS       [88405000, 7FFFFFF000, 3] > [0, 0]
[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd GRANULE_DELEGATE
[TRACE]islet_rmm::stat -- start Stat with command GRANULE_DELEGATE
[INFO]islet_rmm::stat -- RMI::GRANULE_DELEGATE          TOTAL MemUsed:       208944 byte, AVG MemUsed:         30 byte, CallCnt:     6909
[TRACE]islet_rmm::event -- RMI: GRANULE_DELEGATE     [8844C000] > [0]
[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd DATA_CREATE
[TRACE]islet_rmm::stat -- start Stat with command DATA_CREATE
[INFO]islet_rmm::stat -- RMI::DATA_CREATE               TOTAL MemUsed:       147456 byte, AVG MemUsed:         22 byte, CallCnt:     6417
[TRACE]islet_rmm::event -- RMI: DATA_CREATE          [8844C000, 88405000, 7FFFFFF000, 8844B000, 0] > [0]
[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd REALM_ACTIVATE
[TRACE]islet_rmm::stat -- start Stat with command REALM_ACTIVATE
[INFO]islet_rmm::stat -- RMI::REALM_ACTIVATE            TOTAL MemUsed:            0 byte, AVG MemUsed:          0 byte, CallCnt:       25
[TRACE]islet_rmm::event -- RMI: REALM_ACTIVATE       [88405000] > [0]
[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd RTT_MAP_UNPROTECTED
[TRACE]islet_rmm::stat -- start Stat with command RTT_MAP_UNPROTECTED
[INFO]islet_rmm::stat -- RMI::RTT_MAP_UNPROTECTED       TOTAL MemUsed:       106496 byte, AVG MemUsed:         16 byte, CallCnt:     6401
[TRACE]islet_rmm::event -- RMI: RTT_MAP_UNPROTECTED  [88405000, 8000000000, 3, 8844F3D8] > [0]
[TRACE]islet_rmm::rmi::rec::handlers -- rec::Run { entry::flags: 0x0, entry::gprs: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], entry::gicv3_hcr: 0x0, entry::gicv3_lrs: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], exit::exit_reason: 0, exit::imm: 0, exit::cntp_ctl: 0, exit::cntp_cval: 0, exit::cntv_ctl: 0, exit::cntv_cval: 0 }
[DEBUG]islet_rmm::realm::registry -- resuming: 0x400000
[TRACE]islet_rmm::realm::registry -- Switched to VCPU 0 on Realm 24
[TRACE]islet_rmm::rmi::rec::handlers -- REC_ENTER ret: [
    0x0,
    0xC4000196,
    0x0,
    0x0,
]
[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd REALM_CONFIG
[TRACE]islet_rmm::stat -- start Stat with command REALM_CONFIG
[INFO]islet_rmm::stat -- RSI::REALM_CONFIG              TOTAL MemUsed:            0 byte, AVG MemUsed:          0 byte, CallCnt:       25
[TRACE]islet_rmm::event -- RSI: REALM_CONFIG         [] > [4]
[DEBUG]islet_rmm::realm::registry -- resuming: 0x40159c
[TRACE]islet_rmm::realm::registry -- Switched to VCPU 0 on Realm 24
[DEBUG]islet_rmm::exception::trap -- Synchronous: InstructionAbort | DataAbort
[DEBUG]islet_rmm::exception::trap -- fipa: 100600000
[DEBUG]islet_rmm::exception::trap -- esr_el2: 93800005 93800005,

Problems

Data abort from realm in realm's entry function (val_realm_main())

esr_el2: 0x93800005
- EC: 0x24 0b100100 : Data Abort from a lower Exception level
- WnR: false : Abort caused by reading from memory
- DFSC: 0x05 0b000101 : Translation fault, level 1.

fault ipa : 0x100600000 The problem is that the fault ipa (0x100600000) is not mapped -> ipa_width == 33 // hard coded in islet. that's the problem. it should be set by host

0000000000400714 <val_get_curr_test_num>:
400714: a9bf7bfd        stp x29, x30, [sp, #-0x10]!
400718: 910003fd        mov x29, sp
40071c: 97ffff70        bl  0x4004dc <val_get_shared_region_base>
400720: b9400000        ldr w0, [x0] // << here is the problem. x0 == 0x100600000

/**
 *   @brief    Returns the current test number from shared region
 *   @param    Void
 *   @return   Current test number
 **/
uint32_t val_get_curr_test_num(void)
{
    return (*(uint32_t *)((val_get_shared_region_base() + TEST_NUM_OFFSET))); // here !
}

/**
 *   @brief    Returns the IPA address of the shared region
 *   @param    ipa_width      - Realm IPA_WIDTH
 *   @return   IPA address of the shared region
**/
void *val_get_shared_region_base_ipa(uint64_t ipa_width)
{
    if (realm_thread == 1)
        realm_ipa_width = ipa_width;

    // For realm, ns region must be mapped in unprotected space
    return ((void *)(VAL_NS_SHARED_REGION_IPA_OFFSET | (1ull << (ipa_width - 1)))); // This value is 0x100600000
}

//MSB is set at runtime based on ipa_width selected
#define VAL_NS_SHARED_REGION_IPA_OFFSET 0x600000

ipa_width == 33 // hard coded in islet. that's the problem. it should be set by host's argument

Although I fixed the ipa_width properly, the issue is reproduced :( WIP..

zpzigi754 commented 1 year ago

Good to use esr decoder in online! (I was not aware of the online version).

zpzigi754 commented 1 year ago

FYI, this issue might be resolved seamlessly (not sure though), if the new stage 2 page table is used.

nook1208 commented 1 year ago

That's a good news. Probably I can use Rd::ipa_bits temporarily for ipa_width before your PR to go further :)

zpzigi754 commented 1 year ago

Oh, if the main cause is about the hardcoded value like the above analysis (I am sorry for not fixing it directly), it would be orthogonal to the new stage 2 page table. Please go ahead.

nook1208 commented 1 year ago

[TRACE]islet_rmm::stat -- start Stat with command RTT_MAP_UNPROTECTED
[0;31m[ERROR]islet_rmm::realm::registry -- ipa : 8000600000, pa : c0000088200000
[0m[0;31m[ERROR]islet_rmm::realm::mm::stage2_translation -- after set_pages, print L1 all entries:
[0m[0;31m[ERROR]islet_rmm::realm::mm::stage2_translation -- idx:0, Entry(RawPTE(4257658651)) ptr: 0xfdc6c000
[0m[INFO]islet_rmm::stat -- RMI::RTT_MAP_UNPROTECTED       TOTAL MemUsed:         4096 byte, AVG MemUsed:       4096 byte, CallCnt:        1
[TRACE]islet_rmm::event -- RMI: RTT_MAP_UNPROTECTED  [88405000, 8000600000, 3, 882003D8] > [0]
...
[DEBUG]islet_rmm::exception::trap -- Synchronous: InstructionAbort | DataAbort
[DEBUG]islet_rmm::exception::trap -- fipa: 8000600000
[DEBUG]islet_rmm::exception::trap -- esr_el2: 93800005 93800005, 

# in tarmac log
441711990000 ps TTW DTLB LPAE 2:1 0000fdc6c000 00000000fdc6b71b : TABLE ADDR=0x00000000fdc6b000
441711990000 ps TTW DTLB LPAE 2:2 0000fdc6b010 00000000fdc7e71b : TABLE ADDR=0x00000000fdc7e000
441711990000 ps TTW DTLB LPAE 2:3 0000fdc7e190 00000000883327db : BLOCK MEMATTR=6 HAP=3 SH=3 AF=1 nT=1 16E=0 XN=0 AMEC=0 ADDR=0x0000000088332000
441711990000 ps TTW DTLB LPAE 1:3 000088332000 0060008000600767 : BLOCK ATTRIDX=1 NS=1 AP=1 SH=3 AF=1 nT=0 nG=0 16E=0 PXN=1 XN=1 ADDR=0x0000008000600000
441711990000 ps TTW DTLB LPAE 2:1 0000fdc6d000 0000000000000000 : FAULT
441711990000 ps MR4 0000008000600000 (ABORTED)

VTTBR = 0xfdc6c000
Unmapped page table entry address : 0xfdc6d000 (VTTBR + 0x1000)
IPA: 0x8000600000 => IPA[39] == 1
- In RTT_MAP_UNPROTECTED, It should use the next contiguous first-level translation table for the mapping (= VTTBR + 0x1000) like arm reference, but it uses VTTBR (0xfdc6c000)

→ resolved

nook1208 commented 1 year ago

0x412000 is unmapped in EL2, so the data abort occurs

[TRACE]islet_rmm::event -- let's get STATS.lock() with cmd HOST_CALL
[TRACE]islet_rmm::stat -- start Stat with command HOST_CALL
[0;31m[ERROR]islet_rmm::rsi -- eom: pa: 412000
[0m[0;31m[ERROR]islet_rmm::rsi -- eom: host_call is parsed
[0m[0;31m[ERROR]islet_rmm::panic -- RMM: panicked at rmm/src/exception/trap.rs:57:17:
Info { source: CurrentSPELx, kind: Synchronous } and esr 96000006, TrapFrame: TrapFrame { _res: 0, elr: 4257311548, spsr: 1610613705, regs: [4257733336, 1, 72, 4282512904, 2, 6, 2285965312, 4282514560, 4274180432, 375360, 375432, 112, 4257733296, 4257755136, 72, 4257733408, 4257733408, 1248, 4257734656, 4282513472, 4257350628, 4282513480, 4257231168, 4282513496, 4257316272, 1, 4257365256, 14, 4282513712, 4274180096, 4257311548] } on cpu::id (0, 0)

-> resolved after ipa to pa translation.

Although It can't pass the mm_unprotected_ipa_boundary testcase at the moment, Because the setup of realm was failed, other ACS testcases were also failed. So I think it's worth to push my patches first.

nook1208 commented 1 year ago

Now I've fixed a few problems and get pass on mm_unprotected_ipa_boundary testcase. All of passed ACS testcases are below :

cmd_granule_delegate
cmd_granule_undelegate
cmd_realm_create
cmd_rtt_read_entry
cmd_rmi_version
cmd_rsi_version
exception_rec_exit_hostcall
exception_realm_unsupported_smc
exception_rec_exit_hvc
exception_rec_exit_ripas 
exception_rec_exit_ia 
mm_hipas_assigned_da 
mm_unprotected_ipa_boundary 
mm_rtt_translation_table 
mm_feat_s2fwb_check_1 
mm_rtt_level_start

==================
   TOTAL TESTS     : 59
   TOTAL PASSED    : 16
   TOTAL FAILED    : 28
   TOTAL SKIPPED   : 2
   TOTAL SIM ERROR : 13

******* END OF ACS *******

nook1208 commented 1 year ago

Fixed with #197

islet-project / islet

ACS: mm_unprotected_ipa_boundary testcase is failed #188

Problems