search

HyperEnclave / hyperenclave

An Open and Cross-platform Trusted Execution Environment.
Apache License 2.0
141 stars 18 forks source link

Question about some parameters set in GRUB_CMD_LINE. #6

Closed Unik-lif closed 2 months ago

Unik-lif commented 1 year ago

Hello! I tried to build Hyperenclave for AMD R7-6800H, I changed the GRUB_CMDLINE_LINUX parameter into the following one:

memmap=4G\\\$0x100000000 amd_iommu=off mem_encrypt=off intremap=off no5lvl

Despite of some crash for general protection fault: 0000 [#1] SMP NOPTI, after the booting process of hyperenclave-driver, I stil successfully ran the sgx-code you offered in docker image with the aid of the hyperenclave. But the fault is still a bit of annoying, so I wonder if you can offer me the specific parameters of GRUB_CMDLINE_LINUX you are using in AMD processors. I've referred the version offered in atc-22 repos, but I don't know why after setting up hyperenclave for a while the crash will happen. So I doubt that maybe I've made some fault as I set the GRUB_CMDLINE_LINUX.

Besides, I am currently reading the source code of hyperenclave-driver and got some questions that I can't figure out now.

I wonder why we should turn amd_iommu and intremap to off mode, and why we should disable 5-level page mechanism? Could you please further explain the reason behind here?

Looking forward to your reply.

Unik-lif commented 1 year ago

Besides, in project hyperenclave-driver, I came across a function in function he_cmd_enable in hyperenclave-driver/driver/main.c.

    num_iomem = get_iomem_num();
    /*
     * memmap region should be removed from iomem regions,
     * so the max num of mem_regions is iomem_num + nr_rsrv_mem.
     */
    mem_regions = kvmalloc(sizeof(*mem_regions) * (num_iomem + nr_rsrv_mem),
                   GFP_KERNEL);

I am a little confused whether the flag amd_iommu=off or intel_iommu=off has some impacts on the function get_iomem_num, therefore triggering a wrong number of mem_regions?

int get_iomem_num(void)
{
    int num;
    struct resource *child;

    num = 0;
    child = iomem_resource.child;
    while (child) {
        num++;
        child = child->sibling;
    }

    return num;
}

Looking forward to your reply~

cz-chenzhou commented 1 year ago

Hello! I tried to build Hyperenclave for AMD R7-6800H, I changed the GRUB_CMDLINE_LINUX parameter into the following one:

memmap=4G\\\$0x100000000 amd_iommu=off mem_encrypt=off intremap=off no5lvl

Despite of some crash for general protection fault: 0000 [#1] SMP NOPTI, after the booting process of hyperenclave-driver, I stil successfully ran the sgx-code you offered in docker image with the aid of the hyperenclave. But the fault is still a bit of annoying, so I wonder if you can offer me the specific parameters of GRUB_CMDLINE_LINUX you are using in AMD processors. I've referred the version offered in atc-22 repos, but I don't know why after setting up hyperenclave for a while the crash will happen. So I doubt that maybe I've made some fault as I set the GRUB_CMDLINE_LINUX.

The GRUB_CMDLINE_LINUX you used is correct. Please provide more log about the hyperenclave setup and the crash.

Besides, I am currently reading the source code of hyperenclave-driver and got some questions that I can't figure out now.

I wonder why we should turn amd_iommu and intremap to off mode, and why we should disable 5-level page mechanism? Could you please further explain the reason behind here?

  1. turn amd_iommu and intremap to off In our design, hyperenclave restricts the physical memory accessed by the peripherals with the support of IOMMU. Hyperenclave is responsible for IOMMU hardware initialization, so we need to turn the amd_iommu and intremap to off for host linux.
  2. disable 5-level page Currently, hyperenclave doesn't support 5-level page table. Some hardware platform may use 5-level page table, such as，Intel Ice Lake. So we just add the no5lvl to disable it.

Looking forward to your reply.

cz-chenzhou commented 1 year ago

Besides, in project hyperenclave-driver, I came across a function in function he_cmd_enable in hyperenclave-driver/driver/main.c.

  num_iomem = get_iomem_num();
  /*
   * memmap region should be removed from iomem regions,
   * so the max num of mem_regions is iomem_num + nr_rsrv_mem.
   */
  mem_regions = kvmalloc(sizeof(*mem_regions) * (num_iomem + nr_rsrv_mem),
                 GFP_KERNEL);

I am a little confused whether the flag amd_iommu=off or intel_iommu=off has some impacts on the function get_iomem_num, therefore triggering a wrong number of mem_regions?

The IOMMU parameter amd_iommu=off has nothing to do with the function get_iomem_num. Function get_iomem_num is used to get the physical memory information. There may be some other issue triggering the crash. As i said above, we need the hyperenclave setup log and the crash log to figure out the reason.

int get_iomem_num(void)
{
  int num;
  struct resource *child;

  num = 0;
  child = iomem_resource.child;
  while (child) {
      num++;
      child = child->sibling;
  }

  return num;
}

Looking forward to your reply~

Unik-lif commented 1 year ago

Thank u for your answer and patience! Here is the log in dmesg, it seems that although it crashes at the beginning, it still successfully handled sgx code somehow:

[  229.301014] hyper_enclave: loading out-of-tree module taints kernel.
[  229.301162] hyper_enclave: module verification failed: signature and/or required key missing - tainting kernel
[  229.312632] HE: cpu_vendor_detect: 39. Vendor ID: AuthenticAMD
[  229.325575] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000000000000 -> 0x000000000009f000], type: System RAM
[  229.325577] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x000000000009f000 -> 0x00000000000c0000], type: Reserved
[  229.325578] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000000100000 -> 0x0000000009b00000], type: System RAM
[  229.325579] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009b00000 -> 0x0000000009e00000], type: Reserved
[  229.325580] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009e00000 -> 0x0000000009f00000], type: System RAM
[  229.325580] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009f00000 -> 0x0000000009f28000], type: ACPI Non-volatile Storage
[  229.325581] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009f28000 -> 0x00000000a07ff000], type: System RAM
[  229.325582] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a07ff000 -> 0x00000000a0800000], type: Reserved
[  229.325582] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a0800000 -> 0x00000000a2364000], type: System RAM
[  229.325583] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a2364000 -> 0x00000000a4564000], type: Reserved
[  229.325584] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a4564000 -> 0x00000000a456d000], type: System RAM
[  229.325584] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a456d000 -> 0x00000000a4570000], type: Reserved
[  229.325585] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a4570000 -> 0x00000000b077f000], type: System RAM
[  229.325586] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000b077f000 -> 0x00000000b2f7f000], type: Reserved
[  229.325587] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000b2f7f000 -> 0x00000000baf7f000], type: ACPI Non-volatile Storage
[  229.325587] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000baf7f000 -> 0x00000000bafff000], type: ACPI Tables
[  229.325588] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000bafff000 -> 0x00000000bb000000], type: System RAM
[  229.325589] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000bb000000 -> 0x00000000bc000000], type: Reserved
[  229.325589] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000bce00000 -> 0x00000000c0000000], type: Reserved
[  229.325590] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000fde00000 -> 0x00000000fdf00000], type: Reserved
[  229.325590] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000fed80000 -> 0x00000000fed81000], type: Reserved
[  229.325591] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000ff000000 -> 0x0000000100000000], type: Reserved
[  229.325592] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000100000000 -> 0x000000041e300000], type: System RAM
[  229.325593] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x000000041f340000 -> 0x0000000460200000], type: Reserved
[  229.325596] HE: get_convertible_memory: 213. Convertible Memory[ 0]: 0x0000000000000000 -> 0x000000000009f000
[  229.325596] HE: get_convertible_memory: 213. Convertible Memory[ 1]: 0x0000000000100000 -> 0x0000000009b00000
[  229.325597] HE: get_convertible_memory: 213. Convertible Memory[ 2]: 0x0000000009e00000 -> 0x0000000009f00000
[  229.325598] HE: get_convertible_memory: 213. Convertible Memory[ 3]: 0x0000000009f28000 -> 0x00000000a07ff000
[  229.325599] HE: get_convertible_memory: 213. Convertible Memory[ 4]: 0x00000000a0800000 -> 0x00000000a2364000
[  229.325599] HE: get_convertible_memory: 213. Convertible Memory[ 5]: 0x00000000a4564000 -> 0x00000000a456d000
[  229.325600] HE: get_convertible_memory: 213. Convertible Memory[ 6]: 0x00000000a4570000 -> 0x00000000b077f000
[  229.325600] HE: get_convertible_memory: 213. Convertible Memory[ 7]: 0x00000000bafff000 -> 0x00000000bb000000
[  229.325601] HE: get_convertible_memory: 213. Convertible Memory[ 8]: 0x0000000100000000 -> 0x000000041e300000
[  229.325602] HE: get_convertible_memory: 218. Convertible Memory size: 0x3cc4f3000
[  229.325603] HE: get_valid_rsrv_mem: 285. Reserved Memory[ 0]: 0x100000000 -> 0x200000000
[  229.325604] HE: get_valid_rsrv_mem: 290. Reserved Memory size: 0x100000000
[  229.325606] HE: get_sme_mask: 63. CPU does not enable SME
[  229.325649] HE: mem_test: 48. Memory[0x100000000 - 0x200000000] test begin
[  230.196819] HE: mem_test: 78. Memory[0x100000000 - 0x200000000] test pass
[  230.211666] HE: get_hv_heap_size: 375. Hypervisor heap size: 0x43800000
[  230.211668] HE: get_hv_cmrm_size: 387. Hypervisor cmrm size: 0x62d5000
[  230.211669] HE: get_hv_frame_size: 400. Hypervisor frame size: 0x1c00000
[  230.211669] HE: get_hypervisor_size: 413. Hv_core_and_percpu_size: 0xe40000, Hypervisor size: 0x80000000
[  230.211670] HE: he_cmd_enable: 302. hypervisor size: 0x80000000
[  230.211672] HE: get_sme_mask: 63. CPU does not enable SME
[  230.394700] HE: he_cmd_enable: 352. config_size: 1860
[  230.417661] HE: add_epc_pages: 43. total_epc_pages: 0x80000, free_epc_pages: 0x80000
[  230.417663] HE: init_enclave_page: 317. epc ranges: [0x180000000-0x200000000], 0x80000000
[  230.417664] HE: init_enclave_page: 333. Initialized EPC ranges size: 0x80000000
[  230.417665] HE: he_cmd_enable: 383. config_header load_addr: 0xffffff0000e40000
[  230.417695] HE: he_cmd_enable: 404. mem_region load_addr: 0xffffff0000e40124
[  230.417696] HE: inspect_tpm: 206. using fake tpm
[  230.417697] HE: he_cmd_enable: 411. tpm mmio type=8,size=0 pa=ffffffff
[  230.574385] HE: init_cmrm: 448. Initialize [0x0 -> 0x41e300000]'s CMRM
[  230.574631] HE: he_cmd_enable: 483. The hyperenclave is opening.
[  240.645555] [0] Activating hypervisor on CPU 0...
[  240.645557] [1] Activating hypervisor on CPU 1...
[  240.645558] [2] Activating hypervisor on CPU 2...
[  240.645559] [3] Activating hypervisor on CPU 3...
[  240.645560] [4] Activating hypervisor on CPU 4...
[  240.645561] [5] Activating hypervisor on CPU 5...
[  240.645561] [6] Activating hypervisor on CPU 6...
[  240.645562] [7] Activating hypervisor on CPU 7...
[  240.645563] [8] Activating hypervisor on CPU 8...
[  240.645564] [9] Activating hypervisor on CPU 9...
[  240.645564] [10] Init HHBox log feature ok
[  240.645565] [10] Init HHBox crash feature ok
[  240.645565] [10] tpm_detect starting....
[  240.645565] [10] fake tpm is detected and initialized
[  240.645566] [10] FAKE TPM: tpm signing key pub x
[  240.645566] [10] C29974C9F1090FA4A10E9990620E91828B593A7211E2468450E3DC96DD5933FB
[  240.645567] [10] size= :0x20
[  240.645567] [10] FAKE TPM: tpm signing key pub y
[  240.645568] [10] 402206ECCC5479289F33668EAAB85527ABBBB9F7B41CEB71551027D57AF28267
[  240.645568] [10] size= :0x20
[  240.645568] [10] FAKE TPM: root secret is generated and sealed
[  240.645569] [10] FAKE TPM: hypervisor AK pub x=
[  240.645569] [10] 3D9BB7BA028C5F97AC5AB1619336D9ED23E86858DDBDC23B510D5F0EBA8FF338
[  240.645569] [10] size= :0x20
[  240.645570] [10] FAKE TPM: hypervisor AK pub y=
[  240.645570] [10] 0B28428BDA30B2800FCB032ABCED81071B5F0DCB1A02B22AFF56B7DD22E52522
[  240.645571] [10] size= :0x20
[  240.645571] [10] FAKE TPM: hash of he_ak_pub extended to PCR 13:
[  240.645571] [10] AAA056CA1F030B7BD6C4089C2AEEC36D01173B46E0FD2B4C1BD2C14649B66539
[  240.645572] [10] size= :0x20
[  240.645572] [10] HyperEnclave: root of trust initialized!
[  240.645572] [10] Activating hypervisor on CPU 10...
[  240.645573] [11] Activating hypervisor on CPU 11...
[  240.645574] [12] Activating hypervisor on CPU 12...
[  240.645575] [13] Activating hypervisor on CPU 13...
[  240.645575] [14] Activating hypervisor on CPU 14...
[  240.645576] [15] Activating hypervisor on CPU 15...
[  254.042706] general protection fault: 0000 [#1] SMP NOPTI
[  254.042715] CPU: 0 PID: 1595 Comm: upowerd Tainted: G           OE     5.4.0-050400-generic #201911242031
[  254.042717] Hardware name: HONOR GLO-NX6/GLO-NX6-PCB, BIOS 1.10 06/13/2023
[  254.042724] RIP: 0010:acpi_ex_system_memory_space_handler+0x239/0x2b5
[  254.042728] Code: 02 00 00 00 00 41 83 fc 20 74 25 77 12 41 83 fc 08 74 17 41 83 fc 10 75 58 41 0f b7 06 eb 14 41 83 fc 40 75 4c 49 8b 06 eb 09 <41> 0f b6 06 eb 03 41 8b 06 49 89 02 eb 3c 41 83 fc 20 74 2d 77 15
[  254.042730] RSP: 0018:ffffbbb7c231f8a8 EFLAGS: 00010246
[  254.042733] RAX: ffffbbb7c03f937e RBX: 00000000fe80037e RCX: 0000000000000080
[  254.042734] RDX: 00000000fe800400 RSI: 00000000000000f4 RDI: 0000000000000033
[  254.042735] RBP: ffffbbb7c231f8d8 R08: 0000000000000000 R09: ffff9e4ccfa56b00
[  254.042736] R10: ffffbbb7c231fa08 R11: ffff9e4cc9c95038 R12: 0000000000000008
[  254.042737] R13: 0000000000000000 R14: ffffbbb7c03f937e R15: ffff9e4ccfa027e0
[  254.042739] FS:  00007f90743d1080(0000) GS:ffff9e4cd2400000(0000) knlGS:0000000000000000
[  254.042740] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  254.042742] CR2: 00007fc1bd15a9dc CR3: 00000003da4fa000 CR4: 0000000000740ef0
[  254.042743] PKRU: 55555554
[  254.042744] Call Trace:
[  254.042751]  acpi_ev_address_space_dispatch+0x2f7/0x39f
[  254.042754]  ? acpi_ex_prep_field_value+0x513/0x513
[  254.042756]  acpi_ex_access_region+0x454/0x4ed
[  254.042759]  acpi_ex_field_datum_io+0x18a/0x42d
[  254.042762]  acpi_ex_extract_from_field+0xff/0x320
[  254.042764]  ? acpi_ev_acquire_global_lock+0x1de/0x1e6
[  254.042767]  ? acpi_ex_acquire_mutex_object+0x115/0x11f
[  254.042769]  acpi_ex_read_data_from_field+0x30f/0x361
[  254.042771]  acpi_ex_resolve_node_to_value+0x3a7/0x4dd
[  254.042773]  acpi_ex_resolve_to_value+0x3c3/0x472
[  254.042776]  acpi_ds_evaluate_name_path+0xb1/0x169
[  254.042779]  ? acpi_db_single_step+0x1f/0x252
[  254.042781]  acpi_ds_exec_end_op+0x118/0x76b
[  254.042784]  acpi_ps_parse_loop+0x84b/0x920
[  254.042786]  acpi_ps_parse_aml+0x1af/0x550
[  254.042789]  acpi_ps_execute_method+0x208/0x2ca
[  254.042791]  acpi_ns_evaluate+0x34e/0x4f0
[  254.042793]  acpi_evaluate_object+0x18e/0x3b4
[  254.042796]  acpi_battery_get_state+0x94/0x220
[  254.042798]  acpi_battery_get_property+0x4f/0x3e2
[  254.042803]  power_supply_get_property.part.0+0x15/0x20
[  254.042805]  power_supply_get_property+0x18/0x30
[  254.042807]  power_supply_show_property+0x9d/0x300
[  254.042811]  dev_attr_show+0x1d/0x40
[  254.042815]  sysfs_kf_seq_show+0xa1/0x100
[  254.042817]  kernfs_seq_show+0x27/0x30
[  254.042820]  seq_read+0xdc/0x430
[  254.042822]  kernfs_fop_read+0x35/0x190
[  254.042826]  __vfs_read+0x1b/0x40
[  254.042828]  vfs_read+0xab/0x160
[  254.042830]  ksys_read+0x67/0xe0
[  254.042832]  __x64_sys_read+0x1a/0x20
[  254.042836]  do_syscall_64+0x57/0x190
[  254.042841]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  254.042843] RIP: 0033:0x7f9074f4b3cc
[  254.042846] Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 89 fc ff ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 30 44 89 c7 48 89 44 24 08 e8 bf fc ff ff 48
[  254.042848] RSP: 002b:00007ffcd424da40 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[  254.042850] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9074f4b3cc
[  254.042851] RDX: 0000000000001000 RSI: 000055ab9cd11f00 RDI: 000000000000000a
[  254.042852] RBP: 000055ab9ccdbb30 R08: 0000000000000000 R09: 0000000000001000
[  254.042853] R10: 000055ab9cc56010 R11: 0000000000000246 R12: 00007ffcd424db70
[  254.042854] R13: 0000000000000000 R14: 0000000000000000 R15: 000000000000000a
[  254.042856] Modules linked in: sm3_generic hyper_enclave(OE) rfcomm xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_addrtype iptable_filter bpfilter br_netfilter bridge stp llc ccm cmac algif_hash algif_skcipher af_alg overlay bnep kvm_amd ccp joydev kvm irqbypass snd_hda_codec_generic snd_hda_codec_hdmi nls_iso8859_1 snd_hda_intel snd_intel_nhlt rt2800usb snd_hda_codec rt2x00usb rt2800lib snd_hda_core rt2x00lib snd_hwdep crct10dif_pclmul uvcvideo videobuf2_vmalloc ghash_clmulni_intel snd_pcm mac80211 videobuf2_memops videobuf2_v4l2 btusb videobuf2_common btrtl btbcm snd_seq_midi btintel snd_seq_midi_event bluetooth snd_rawmidi cfg80211 aesni_intel snd_seq videodev huawei_wmi crypto_simd snd_seq_device cryptd ecdh_generic ledtrig_audio glue_helper snd_timer mc hid_multitouch libarc4 input_leds ecc sparse_keymap serio_raw wmi_bmof snd soundcore snd_pci_acp3x mac_hid acpi_tad sch_fq_codel
[  254.042902]  parport_pc ppdev lp parport ramoops drm reed_solomon efi_pstore ip_tables x_tables autofs4 hid_generic crc32_pclmul nvme i2c_piix4 nvme_core wmi video i2c_hid hid
[  254.042917] ---[ end trace 174b1af698bdf677 ]---
[  257.158924] RIP: 0010:acpi_ex_system_memory_space_handler+0x239/0x2b5
[  257.158932] Code: 02 00 00 00 00 41 83 fc 20 74 25 77 12 41 83 fc 08 74 17 41 83 fc 10 75 58 41 0f b7 06 eb 14 41 83 fc 40 75 4c 49 8b 06 eb 09 <41> 0f b6 06 eb 03 41 8b 06 49 89 02 eb 3c 41 83 fc 20 74 2d 77 15
[  257.158935] RSP: 0018:ffffbbb7c231f8a8 EFLAGS: 00010246
[  257.158938] RAX: ffffbbb7c03f937e RBX: 00000000fe80037e RCX: 0000000000000080
[  257.158940] RDX: 00000000fe800400 RSI: 00000000000000f4 RDI: 0000000000000033
[  257.158941] RBP: ffffbbb7c231f8d8 R08: 0000000000000000 R09: ffff9e4ccfa56b00
[  257.158942] R10: ffffbbb7c231fa08 R11: ffff9e4cc9c95038 R12: 0000000000000008
[  257.158943] R13: 0000000000000000 R14: ffffbbb7c03f937e R15: ffff9e4ccfa027e0
[  257.158945] FS:  00007f90743d1080(0000) GS:ffff9e4cd2400000(0000) knlGS:0000000000000000
[  257.158947] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  257.158948] CR2: 00007fc1bd15a9dc CR3: 00000003da4fa000 CR4: 0000000000740ef0
[  257.158949] PKRU: 55555554
[  261.125550] [0] [WARN][0] #VMEXIT(NPF) @ NptViolationInfo {
[  261.125551]     guest_paddr: 0xfe80037e,
[  261.125552]     present: false,
[  261.125552]     write: false,
[  261.125552]     user_mode: true,
[  261.125553]     reserved_bits_used: false,
[  261.125553]     execute: false,
[  261.125554]     shadow_stack_access: false,
[  261.125554]     final_translation: true,
[  261.125554] } RIP(0xffffffff9cbf91e5)
[  261.125555] [0] [WARN][0] #VMEXIT handler returned Err([src/arch/x86_64/amd/vmexit.rs:167:9] Function not implemented):
[  261.125555] VmExitInfo {
[  261.125556]     exit_code: Ok(
[  261.125556]         NPF,
[  261.125556]     ),
[  261.125557]     exit_info_1: 0x100000004,
[  261.125557]     exit_info_2: 0xfe80037e,
[  261.125558]     guest_rip: 0xffffffff9cbf91e5,
[  261.125558] }

[  261.125558] Guest State Dump:
[  261.125559] Vcpu {
[  261.125559]     guest_regs: GuestRegisters {
[  261.125559]         rax: 0xffffbbb7c03f937e,
[  261.125560]         rcx: 0x80,
[  261.125560]         rdx: 0xfe800400,
[  261.125560]         rbx: 0xfe80037e,
[  261.125561]         _unused_rsp: 0x0,
[  261.125561]         rbp: 0xffffbbb7c231f8d8,
[  261.125561]         rsi: 0xf4,
[  261.125562]         rdi: 0x33,
[  261.125562]         r8: 0x0,
[  261.125562]         r9: 0xffff9e4ccfa56b00,
[  261.125563]         r10: 0xffffbbb7c231fa08,
[  261.125563]         r11: 0xffff9e4cc9c95038,
[  261.125563]         r12: 0x8,
[  261.125564]         r13: 0x0,
[  261.125564]         r14: 0xffffbbb7c03f937e,
[  261.125564]         r15: 0xffff9e4ccfa027e0,
[  261.125565]     },
[  261.125565]     rip: 0xffffffff9cbf91e5,
[  261.125565]     rsp: 0xffffbbb7c231f8a8,
[  261.125566]     rflags: INTERRUPT_FLAG | ZERO_FLAG | PARITY_FLAG | 0x0x2,
[  261.125566]     cr0: PROTECTED_MODE_ENABLE | MONITOR_COPROCESSOR | NUMERIC_ERROR | WRITE_PROTECT | ALIGNMENT_MASK | PAGING | 0x0x10,
[  261.125567]     cr3: 0x3da4fa000,
[  261.125568]     cr4: PAGE_SIZE_EXTENSION | PHYSICAL_ADDRESS_EXTENSION | MACHINE_CHECK_EXCEPTION | PAGE_GLOBAL | OSFXSR | OSXMMEXCPT_ENABLE | USER_MODE_INSTRUCTION_PREVENTION | OSXSAVE | SUPERVISOR_MODE_EXECUTION_PROTECTION | SUPERVISOR_MODE_ACCESS_PREVENTION | PROTECTION_KEY,
[  261.125568]     cs: VmcbSegment {
[  261.125569]         selector: 0x10,
[  261.125569]         attr: 0x29b,
[  261.125569]         limit: 0xffffffff,
[  261.125570]         base: 0x0,
[  261.125570]     },
[  261.125570] }
[  261.125571] [0] [ERROR][0] Failed to handle VM exit, inject fault to guest...
[  261.125571] [src/arch/x86_64/amd/vmexit.rs:167:9] Function not implemented
[  261.125572] [0] [WARN][0] VCPU fault: PerCpu {
[  261.125572]     cpu_id: 0x0,
[  261.125572]     state: HvEnabled,
[  261.125572]     vcpu: Vcpu {
[  261.125573]         guest_regs: GuestRegisters {
[  261.125573]             rax: 0xffffbbb7c03f937e,
[  261.125574]             rcx: 0x80,
[  261.125574]             rdx: 0xfe800400,
[  261.125574]             rbx: 0xfe80037e,
[  261.125575]             _unused_rsp: 0x0,
[  261.125575]             rbp: 0xffffbbb7c231f8d8,
[  261.125575]             rsi: 0xf4,
[  261.125576]             rdi: 0x33,
[  261.125576]             r8: 0x0,
[  261.125576]             r9: 0xffff9e4ccfa56b00,
[  261.125577]             r10: 0xffffbbb7c231fa08,
[  261.125577]             r11: 0xffff9e4cc9c95038,
[  261.125577]             r12: 0x8,
[  261.125578]             r13: 0x0,
[  261.125578]             r14: 0xffffbbb7c03f937e,
[  261.125578]             r15: 0xffff9e4ccfa027e0,
[  261.125579]         },
[  261.125579]         rip: 0xffffffff9cbf91e5,
[  261.125579]         rsp: 0xffffbbb7c231f8a8,
[  261.125580]         rflags: INTERRUPT_FLAG | ZERO_FLAG | PARITY_FLAG | 0x0x2,
[  261.125580]         cr0: PROTECTED_MODE_ENABLE | MONITOR_COPROCESSOR | NUMERIC_ERROR | WRITE_PROTECT | ALIGNMENT_MASK | PAGING | 0x0x10,
[  261.125581]         cr3: 0x3da4fa000,
[  261.125582]         cr4: PAGE_SIZE_EXTENSION | PHYSICAL_ADDRESS_EXTENSION | MACHINE_CHECK_EXCEPTION | PAGE_GLOBAL | OSFXSR | OSXMMEXCPT_ENABLE | USER_MODE_INSTRUCTION_PREVENTION | OSXSAVE | SUPERVISOR_MODE_EXECUTION_PROTECTION | SUPERVISOR_MODE_ACCESS_PREVENTION | PROTECTION_KEY,
[  261.125582]         cs: VmcbSegment {
[  261.125583]             selector: 0x10,
[  261.125583]             attr: 0x29b,
[  261.125583]             limit: 0xffffffff,
[  261.125584]             base: 0x0,
[  261.125584]         },
[  261.125584]     },
[  261.125585]     enclave_thread: Inactive,
[  261.125585] }
[  472.325810] HE: he_cmd_encl_create: 226. encl: 0xffff9e4cc8094000
[  472.325837] HE: he_cmd_encl_create: 259. encl: 0xffff9e4cc8094000, encl.start_gva=0x7fd49ec15000, encl_size: 0x1000000
[  472.427682] HE: shared_memory_destroy: 327. mmu_notifier_unregister
[  472.428057] HE: he_encl_cleanup: 966. nr_free_epc_page: 0x80000, encl: 0xffff9e4cc8094000
[  875.504580] HE: he_cmd_encl_create: 226. encl: 0xffff9e4c120e4000
[  875.504594] HE: he_cmd_encl_create: 259. encl: 0xffff9e4c120e4000, encl.start_gva=0x7fb9a4f4e000, encl_size: 0x200000000
[  917.053466] HE: shared_memory_destroy: 327. mmu_notifier_unregister
[  917.094283] HE: he_encl_cleanup: 966. nr_free_epc_page: 0x80000, encl: 0xffff9e4c120e4000
[ 1097.502777] HE: he_cmd_encl_create: 226. encl: 0xffff9e4c1a218000
[ 1097.502792] HE: he_cmd_encl_create: 259. encl: 0xffff9e4c1a218000, encl.start_gva=0x7f7a57d3e000, encl_size: 0x200000000
[ 1115.746853] HE: shared_memory_destroy: 327. mmu_notifier_unregister
[ 1115.788011] HE: he_encl_cleanup: 966. nr_free_epc_page: 0x80000, encl: 0xffff9e4c1a218000
cz-chenzhou commented 1 year ago

Thank u for your answer and patience! Here is the log in dmesg, it seems that although it crashes at the beginning, it still successfully handled sgx code somehow:

[  229.301014] hyper_enclave: loading out-of-tree module taints kernel.
[  229.301162] hyper_enclave: module verification failed: signature and/or required key missing - tainting kernel
[  229.312632] HE: cpu_vendor_detect: 39. Vendor ID: AuthenticAMD
[  229.325575] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000000000000 -> 0x000000000009f000], type: System RAM
[  229.325577] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x000000000009f000 -> 0x00000000000c0000], type: Reserved
[  229.325578] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000000100000 -> 0x0000000009b00000], type: System RAM
[  229.325579] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009b00000 -> 0x0000000009e00000], type: Reserved
[  229.325580] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009e00000 -> 0x0000000009f00000], type: System RAM
[  229.325580] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009f00000 -> 0x0000000009f28000], type: ACPI Non-volatile Storage
[  229.325581] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000009f28000 -> 0x00000000a07ff000], type: System RAM
[  229.325582] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a07ff000 -> 0x00000000a0800000], type: Reserved
[  229.325582] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a0800000 -> 0x00000000a2364000], type: System RAM
[  229.325583] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a2364000 -> 0x00000000a4564000], type: Reserved
[  229.325584] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a4564000 -> 0x00000000a456d000], type: System RAM
[  229.325584] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a456d000 -> 0x00000000a4570000], type: Reserved
[  229.325585] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000a4570000 -> 0x00000000b077f000], type: System RAM
[  229.325586] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000b077f000 -> 0x00000000b2f7f000], type: Reserved
[  229.325587] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000b2f7f000 -> 0x00000000baf7f000], type: ACPI Non-volatile Storage
[  229.325587] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000baf7f000 -> 0x00000000bafff000], type: ACPI Tables
[  229.325588] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000bafff000 -> 0x00000000bb000000], type: System RAM
[  229.325589] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000bb000000 -> 0x00000000bc000000], type: Reserved
[  229.325589] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000bce00000 -> 0x00000000c0000000], type: Reserved
[  229.325590] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000fde00000 -> 0x00000000fdf00000], type: Reserved
[  229.325590] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000fed80000 -> 0x00000000fed81000], type: Reserved
[  229.325591] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x00000000ff000000 -> 0x0000000100000000], type: Reserved
[  229.325592] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x0000000100000000 -> 0x000000041e300000], type: System RAM
[  229.325593] HE: get_convertible_memory: 136. BIOS E820 table from firmware: [0x000000041f340000 -> 0x0000000460200000], type: Reserved
[  229.325596] HE: get_convertible_memory: 213. Convertible Memory[ 0]: 0x0000000000000000 -> 0x000000000009f000
[  229.325596] HE: get_convertible_memory: 213. Convertible Memory[ 1]: 0x0000000000100000 -> 0x0000000009b00000
[  229.325597] HE: get_convertible_memory: 213. Convertible Memory[ 2]: 0x0000000009e00000 -> 0x0000000009f00000
[  229.325598] HE: get_convertible_memory: 213. Convertible Memory[ 3]: 0x0000000009f28000 -> 0x00000000a07ff000
[  229.325599] HE: get_convertible_memory: 213. Convertible Memory[ 4]: 0x00000000a0800000 -> 0x00000000a2364000
[  229.325599] HE: get_convertible_memory: 213. Convertible Memory[ 5]: 0x00000000a4564000 -> 0x00000000a456d000
[  229.325600] HE: get_convertible_memory: 213. Convertible Memory[ 6]: 0x00000000a4570000 -> 0x00000000b077f000
[  229.325600] HE: get_convertible_memory: 213. Convertible Memory[ 7]: 0x00000000bafff000 -> 0x00000000bb000000
[  229.325601] HE: get_convertible_memory: 213. Convertible Memory[ 8]: 0x0000000100000000 -> 0x000000041e300000
[  229.325602] HE: get_convertible_memory: 218. Convertible Memory size: 0x3cc4f3000
[  229.325603] HE: get_valid_rsrv_mem: 285. Reserved Memory[ 0]: 0x100000000 -> 0x200000000
[  229.325604] HE: get_valid_rsrv_mem: 290. Reserved Memory size: 0x100000000
[  229.325606] HE: get_sme_mask: 63. CPU does not enable SME
[  229.325649] HE: mem_test: 48. Memory[0x100000000 - 0x200000000] test begin
[  230.196819] HE: mem_test: 78. Memory[0x100000000 - 0x200000000] test pass
[  230.211666] HE: get_hv_heap_size: 375. Hypervisor heap size: 0x43800000
[  230.211668] HE: get_hv_cmrm_size: 387. Hypervisor cmrm size: 0x62d5000
[  230.211669] HE: get_hv_frame_size: 400. Hypervisor frame size: 0x1c00000
[  230.211669] HE: get_hypervisor_size: 413. Hv_core_and_percpu_size: 0xe40000, Hypervisor size: 0x80000000
[  230.211670] HE: he_cmd_enable: 302. hypervisor size: 0x80000000
[  230.211672] HE: get_sme_mask: 63. CPU does not enable SME
[  230.394700] HE: he_cmd_enable: 352. config_size: 1860
[  230.417661] HE: add_epc_pages: 43. total_epc_pages: 0x80000, free_epc_pages: 0x80000
[  230.417663] HE: init_enclave_page: 317. epc ranges: [0x180000000-0x200000000], 0x80000000
[  230.417664] HE: init_enclave_page: 333. Initialized EPC ranges size: 0x80000000
[  230.417665] HE: he_cmd_enable: 383. config_header load_addr: 0xffffff0000e40000
[  230.417695] HE: he_cmd_enable: 404. mem_region load_addr: 0xffffff0000e40124
[  230.417696] HE: inspect_tpm: 206. using fake tpm
[  230.417697] HE: he_cmd_enable: 411. tpm mmio type=8,size=0 pa=ffffffff
[  230.574385] HE: init_cmrm: 448. Initialize [0x0 -> 0x41e300000]'s CMRM
[  230.574631] HE: he_cmd_enable: 483. The hyperenclave is opening.
[  240.645555] [0] Activating hypervisor on CPU 0...
[  240.645557] [1] Activating hypervisor on CPU 1...
[  240.645558] [2] Activating hypervisor on CPU 2...
[  240.645559] [3] Activating hypervisor on CPU 3...
[  240.645560] [4] Activating hypervisor on CPU 4...
[  240.645561] [5] Activating hypervisor on CPU 5...
[  240.645561] [6] Activating hypervisor on CPU 6...
[  240.645562] [7] Activating hypervisor on CPU 7...
[  240.645563] [8] Activating hypervisor on CPU 8...
[  240.645564] [9] Activating hypervisor on CPU 9...
[  240.645564] [10] Init HHBox log feature ok
[  240.645565] [10] Init HHBox crash feature ok
[  240.645565] [10] tpm_detect starting....
[  240.645565] [10] fake tpm is detected and initialized
[  240.645566] [10] FAKE TPM: tpm signing key pub x
[  240.645566] [10] C29974C9F1090FA4A10E9990620E91828B593A7211E2468450E3DC96DD5933FB
[  240.645567] [10] size= :0x20
[  240.645567] [10] FAKE TPM: tpm signing key pub y
[  240.645568] [10] 402206ECCC5479289F33668EAAB85527ABBBB9F7B41CEB71551027D57AF28267
[  240.645568] [10] size= :0x20
[  240.645568] [10] FAKE TPM: root secret is generated and sealed
[  240.645569] [10] FAKE TPM: hypervisor AK pub x=
[  240.645569] [10] 3D9BB7BA028C5F97AC5AB1619336D9ED23E86858DDBDC23B510D5F0EBA8FF338
[  240.645569] [10] size= :0x20
[  240.645570] [10] FAKE TPM: hypervisor AK pub y=
[  240.645570] [10] 0B28428BDA30B2800FCB032ABCED81071B5F0DCB1A02B22AFF56B7DD22E52522
[  240.645571] [10] size= :0x20
[  240.645571] [10] FAKE TPM: hash of he_ak_pub extended to PCR 13:
[  240.645571] [10] AAA056CA1F030B7BD6C4089C2AEEC36D01173B46E0FD2B4C1BD2C14649B66539
[  240.645572] [10] size= :0x20
[  240.645572] [10] HyperEnclave: root of trust initialized!
[  240.645572] [10] Activating hypervisor on CPU 10...
[  240.645573] [11] Activating hypervisor on CPU 11...
[  240.645574] [12] Activating hypervisor on CPU 12...
[  240.645575] [13] Activating hypervisor on CPU 13...
[  240.645575] [14] Activating hypervisor on CPU 14...
[  240.645576] [15] Activating hypervisor on CPU 15...
[  254.042706] general protection fault: 0000 [#1] SMP NOPTI
[  254.042715] CPU: 0 PID: 1595 Comm: upowerd Tainted: G           OE     5.4.0-050400-generic #201911242031
[  254.042717] Hardware name: HONOR GLO-NX6/GLO-NX6-PCB, BIOS 1.10 06/13/2023
[  254.042724] RIP: 0010:acpi_ex_system_memory_space_handler+0x239/0x2b5
[  254.042728] Code: 02 00 00 00 00 41 83 fc 20 74 25 77 12 41 83 fc 08 74 17 41 83 fc 10 75 58 41 0f b7 06 eb 14 41 83 fc 40 75 4c 49 8b 06 eb 09 <41> 0f b6 06 eb 03 41 8b 06 49 89 02 eb 3c 41 83 fc 20 74 2d 77 15
[  254.042730] RSP: 0018:ffffbbb7c231f8a8 EFLAGS: 00010246
[  254.042733] RAX: ffffbbb7c03f937e RBX: 00000000fe80037e RCX: 0000000000000080
[  254.042734] RDX: 00000000fe800400 RSI: 00000000000000f4 RDI: 0000000000000033
[  254.042735] RBP: ffffbbb7c231f8d8 R08: 0000000000000000 R09: ffff9e4ccfa56b00
[  254.042736] R10: ffffbbb7c231fa08 R11: ffff9e4cc9c95038 R12: 0000000000000008
[  254.042737] R13: 0000000000000000 R14: ffffbbb7c03f937e R15: ffff9e4ccfa027e0
[  254.042739] FS:  00007f90743d1080(0000) GS:ffff9e4cd2400000(0000) knlGS:0000000000000000
[  254.042740] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  254.042742] CR2: 00007fc1bd15a9dc CR3: 00000003da4fa000 CR4: 0000000000740ef0
[  254.042743] PKRU: 55555554
[  254.042744] Call Trace:
[  254.042751]  acpi_ev_address_space_dispatch+0x2f7/0x39f
[  254.042754]  ? acpi_ex_prep_field_value+0x513/0x513
[  254.042756]  acpi_ex_access_region+0x454/0x4ed
[  254.042759]  acpi_ex_field_datum_io+0x18a/0x42d
[  254.042762]  acpi_ex_extract_from_field+0xff/0x320
[  254.042764]  ? acpi_ev_acquire_global_lock+0x1de/0x1e6
[  254.042767]  ? acpi_ex_acquire_mutex_object+0x115/0x11f
[  254.042769]  acpi_ex_read_data_from_field+0x30f/0x361
[  254.042771]  acpi_ex_resolve_node_to_value+0x3a7/0x4dd
[  254.042773]  acpi_ex_resolve_to_value+0x3c3/0x472
[  254.042776]  acpi_ds_evaluate_name_path+0xb1/0x169
[  254.042779]  ? acpi_db_single_step+0x1f/0x252
[  254.042781]  acpi_ds_exec_end_op+0x118/0x76b
[  254.042784]  acpi_ps_parse_loop+0x84b/0x920
[  254.042786]  acpi_ps_parse_aml+0x1af/0x550
[  254.042789]  acpi_ps_execute_method+0x208/0x2ca
[  254.042791]  acpi_ns_evaluate+0x34e/0x4f0
[  254.042793]  acpi_evaluate_object+0x18e/0x3b4
[  254.042796]  acpi_battery_get_state+0x94/0x220
[  254.042798]  acpi_battery_get_property+0x4f/0x3e2
[  254.042803]  power_supply_get_property.part.0+0x15/0x20
[  254.042805]  power_supply_get_property+0x18/0x30
[  254.042807]  power_supply_show_property+0x9d/0x300
[  254.042811]  dev_attr_show+0x1d/0x40
[  254.042815]  sysfs_kf_seq_show+0xa1/0x100
[  254.042817]  kernfs_seq_show+0x27/0x30
[  254.042820]  seq_read+0xdc/0x430
[  254.042822]  kernfs_fop_read+0x35/0x190
[  254.042826]  __vfs_read+0x1b/0x40
[  254.042828]  vfs_read+0xab/0x160
[  254.042830]  ksys_read+0x67/0xe0
[  254.042832]  __x64_sys_read+0x1a/0x20
[  254.042836]  do_syscall_64+0x57/0x190
[  254.042841]  entry_SYSCALL_64_after_hwframe+0x44/0xa9
[  254.042843] RIP: 0033:0x7f9074f4b3cc
[  254.042846] Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 89 fc ff ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d 00 f0 ff ff 77 30 44 89 c7 48 89 44 24 08 e8 bf fc ff ff 48
[  254.042848] RSP: 002b:00007ffcd424da40 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
[  254.042850] RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f9074f4b3cc
[  254.042851] RDX: 0000000000001000 RSI: 000055ab9cd11f00 RDI: 000000000000000a
[  254.042852] RBP: 000055ab9ccdbb30 R08: 0000000000000000 R09: 0000000000001000
[  254.042853] R10: 000055ab9cc56010 R11: 0000000000000246 R12: 00007ffcd424db70
[  254.042854] R13: 0000000000000000 R14: 0000000000000000 R15: 000000000000000a
[  254.042856] Modules linked in: sm3_generic hyper_enclave(OE) rfcomm xt_conntrack xt_MASQUERADE nf_conntrack_netlink nfnetlink xfrm_user xfrm_algo iptable_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c xt_addrtype iptable_filter bpfilter br_netfilter bridge stp llc ccm cmac algif_hash algif_skcipher af_alg overlay bnep kvm_amd ccp joydev kvm irqbypass snd_hda_codec_generic snd_hda_codec_hdmi nls_iso8859_1 snd_hda_intel snd_intel_nhlt rt2800usb snd_hda_codec rt2x00usb rt2800lib snd_hda_core rt2x00lib snd_hwdep crct10dif_pclmul uvcvideo videobuf2_vmalloc ghash_clmulni_intel snd_pcm mac80211 videobuf2_memops videobuf2_v4l2 btusb videobuf2_common btrtl btbcm snd_seq_midi btintel snd_seq_midi_event bluetooth snd_rawmidi cfg80211 aesni_intel snd_seq videodev huawei_wmi crypto_simd snd_seq_device cryptd ecdh_generic ledtrig_audio glue_helper snd_timer mc hid_multitouch libarc4 input_leds ecc sparse_keymap serio_raw wmi_bmof snd soundcore snd_pci_acp3x mac_hid acpi_tad sch_fq_codel
[  254.042902]  parport_pc ppdev lp parport ramoops drm reed_solomon efi_pstore ip_tables x_tables autofs4 hid_generic crc32_pclmul nvme i2c_piix4 nvme_core wmi video i2c_hid hid
[  254.042917] ---[ end trace 174b1af698bdf677 ]---
[  257.158924] RIP: 0010:acpi_ex_system_memory_space_handler+0x239/0x2b5
[  257.158932] Code: 02 00 00 00 00 41 83 fc 20 74 25 77 12 41 83 fc 08 74 17 41 83 fc 10 75 58 41 0f b7 06 eb 14 41 83 fc 40 75 4c 49 8b 06 eb 09 <41> 0f b6 06 eb 03 41 8b 06 49 89 02 eb 3c 41 83 fc 20 74 2d 77 15
[  257.158935] RSP: 0018:ffffbbb7c231f8a8 EFLAGS: 00010246
[  257.158938] RAX: ffffbbb7c03f937e RBX: 00000000fe80037e RCX: 0000000000000080
[  257.158940] RDX: 00000000fe800400 RSI: 00000000000000f4 RDI: 0000000000000033
[  257.158941] RBP: ffffbbb7c231f8d8 R08: 0000000000000000 R09: ffff9e4ccfa56b00
[  257.158942] R10: ffffbbb7c231fa08 R11: ffff9e4cc9c95038 R12: 0000000000000008
[  257.158943] R13: 0000000000000000 R14: ffffbbb7c03f937e R15: ffff9e4ccfa027e0
[  257.158945] FS:  00007f90743d1080(0000) GS:ffff9e4cd2400000(0000) knlGS:0000000000000000
[  257.158947] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[  257.158948] CR2: 00007fc1bd15a9dc CR3: 00000003da4fa000 CR4: 0000000000740ef0
[  257.158949] PKRU: 55555554
[  261.125550] [0] [WARN][0] #VMEXIT(NPF) @ NptViolationInfo {
[  261.125551]     guest_paddr: 0xfe80037e,
[  261.125552]     present: false,
[  261.125552]     write: false,
[  261.125552]     user_mode: true,
[  261.125553]     reserved_bits_used: false,
[  261.125553]     execute: false,
[  261.125554]     shadow_stack_access: false,
[  261.125554]     final_translation: true,
[  261.125554] } RIP(0xffffffff9cbf91e5)
[  261.125555] [0] [WARN][0] #VMEXIT handler returned Err([src/arch/x86_64/amd/vmexit.rs:167:9] Function not implemented):
[  261.125555] VmExitInfo {
[  261.125556]     exit_code: Ok(
[  261.125556]         NPF,
[  261.125556]     ),
[  261.125557]     exit_info_1: 0x100000004,
[  261.125557]     exit_info_2: 0xfe80037e,
[  261.125558]     guest_rip: 0xffffffff9cbf91e5,
[  261.125558] }

[  261.125558] Guest State Dump:
[  261.125559] Vcpu {
[  261.125559]     guest_regs: GuestRegisters {
[  261.125559]         rax: 0xffffbbb7c03f937e,
[  261.125560]         rcx: 0x80,
[  261.125560]         rdx: 0xfe800400,
[  261.125560]         rbx: 0xfe80037e,
[  261.125561]         _unused_rsp: 0x0,
[  261.125561]         rbp: 0xffffbbb7c231f8d8,
[  261.125561]         rsi: 0xf4,
[  261.125562]         rdi: 0x33,
[  261.125562]         r8: 0x0,
[  261.125562]         r9: 0xffff9e4ccfa56b00,
[  261.125563]         r10: 0xffffbbb7c231fa08,
[  261.125563]         r11: 0xffff9e4cc9c95038,
[  261.125563]         r12: 0x8,
[  261.125564]         r13: 0x0,
[  261.125564]         r14: 0xffffbbb7c03f937e,
[  261.125564]         r15: 0xffff9e4ccfa027e0,
[  261.125565]     },
[  261.125565]     rip: 0xffffffff9cbf91e5,
[  261.125565]     rsp: 0xffffbbb7c231f8a8,
[  261.125566]     rflags: INTERRUPT_FLAG | ZERO_FLAG | PARITY_FLAG | 0x0x2,
[  261.125566]     cr0: PROTECTED_MODE_ENABLE | MONITOR_COPROCESSOR | NUMERIC_ERROR | WRITE_PROTECT | ALIGNMENT_MASK | PAGING | 0x0x10,
[  261.125567]     cr3: 0x3da4fa000,
[  261.125568]     cr4: PAGE_SIZE_EXTENSION | PHYSICAL_ADDRESS_EXTENSION | MACHINE_CHECK_EXCEPTION | PAGE_GLOBAL | OSFXSR | OSXMMEXCPT_ENABLE | USER_MODE_INSTRUCTION_PREVENTION | OSXSAVE | SUPERVISOR_MODE_EXECUTION_PROTECTION | SUPERVISOR_MODE_ACCESS_PREVENTION | PROTECTION_KEY,
[  261.125568]     cs: VmcbSegment {
[  261.125569]         selector: 0x10,
[  261.125569]         attr: 0x29b,
[  261.125569]         limit: 0xffffffff,
[  261.125570]         base: 0x0,
[  261.125570]     },
[  261.125570] }
[  261.125571] [0] [ERROR][0] Failed to handle VM exit, inject fault to guest...
[  261.125571] [src/arch/x86_64/amd/vmexit.rs:167:9] Function not implemented
[  261.125572] [0] [WARN][0] VCPU fault: PerCpu {
[  261.125572]     cpu_id: 0x0,
[  261.125572]     state: HvEnabled,
[  261.125572]     vcpu: Vcpu {
[  261.125573]         guest_regs: GuestRegisters {
[  261.125573]             rax: 0xffffbbb7c03f937e,
[  261.125574]             rcx: 0x80,
[  261.125574]             rdx: 0xfe800400,
[  261.125574]             rbx: 0xfe80037e,
[  261.125575]             _unused_rsp: 0x0,
[  261.125575]             rbp: 0xffffbbb7c231f8d8,
[  261.125575]             rsi: 0xf4,
[  261.125576]             rdi: 0x33,
[  261.125576]             r8: 0x0,
[  261.125576]             r9: 0xffff9e4ccfa56b00,
[  261.125577]             r10: 0xffffbbb7c231fa08,
[  261.125577]             r11: 0xffff9e4cc9c95038,
[  261.125577]             r12: 0x8,
[  261.125578]             r13: 0x0,
[  261.125578]             r14: 0xffffbbb7c03f937e,
[  261.125578]             r15: 0xffff9e4ccfa027e0,
[  261.125579]         },
[  261.125579]         rip: 0xffffffff9cbf91e5,
[  261.125579]         rsp: 0xffffbbb7c231f8a8,
[  261.125580]         rflags: INTERRUPT_FLAG | ZERO_FLAG | PARITY_FLAG | 0x0x2,
[  261.125580]         cr0: PROTECTED_MODE_ENABLE | MONITOR_COPROCESSOR | NUMERIC_ERROR | WRITE_PROTECT | ALIGNMENT_MASK | PAGING | 0x0x10,
[  261.125581]         cr3: 0x3da4fa000,
[  261.125582]         cr4: PAGE_SIZE_EXTENSION | PHYSICAL_ADDRESS_EXTENSION | MACHINE_CHECK_EXCEPTION | PAGE_GLOBAL | OSFXSR | OSXMMEXCPT_ENABLE | USER_MODE_INSTRUCTION_PREVENTION | OSXSAVE | SUPERVISOR_MODE_EXECUTION_PROTECTION | SUPERVISOR_MODE_ACCESS_PREVENTION | PROTECTION_KEY,
[  261.125582]         cs: VmcbSegment {
[  261.125583]             selector: 0x10,
[  261.125583]             attr: 0x29b,
[  261.125583]             limit: 0xffffffff,
[  261.125584]             base: 0x0,
[  261.125584]         },
[  261.125584]     },
[  261.125585]     enclave_thread: Inactive,
[  261.125585] }
[  472.325810] HE: he_cmd_encl_create: 226. encl: 0xffff9e4cc8094000
[  472.325837] HE: he_cmd_encl_create: 259. encl: 0xffff9e4cc8094000, encl.start_gva=0x7fd49ec15000, encl_size: 0x1000000
[  472.427682] HE: shared_memory_destroy: 327. mmu_notifier_unregister
[  472.428057] HE: he_encl_cleanup: 966. nr_free_epc_page: 0x80000, encl: 0xffff9e4cc8094000
[  875.504580] HE: he_cmd_encl_create: 226. encl: 0xffff9e4c120e4000
[  875.504594] HE: he_cmd_encl_create: 259. encl: 0xffff9e4c120e4000, encl.start_gva=0x7fb9a4f4e000, encl_size: 0x200000000
[  917.053466] HE: shared_memory_destroy: 327. mmu_notifier_unregister
[  917.094283] HE: he_encl_cleanup: 966. nr_free_epc_page: 0x80000, encl: 0xffff9e4c120e4000
[ 1097.502777] HE: he_cmd_encl_create: 226. encl: 0xffff9e4c1a218000
[ 1097.502792] HE: he_cmd_encl_create: 259. encl: 0xffff9e4c1a218000, encl.start_gva=0x7f7a57d3e000, encl_size: 0x200000000
[ 1115.746853] HE: shared_memory_destroy: 327. mmu_notifier_unregister
[ 1115.788011] HE: he_encl_cleanup: 966. nr_free_epc_page: 0x80000, encl: 0xffff9e4c1a218000

It seems the normal vm access the memory with no npt mapping and then trigger the general protection fault. Please provide the physical memory layout of your machine with the following command：

sudo cat /proc/iomem

Thanks.

Unik-lif commented 1 year ago

Thank u 4 u reply~

The physical memory layout is shown below.

00000000-00000fff : Reserved
00001000-0009efff : System RAM
0009f000-000bffff : Reserved
  000a0000-000bffff : PCI Bus 0000:00
000c0000-000c3fff : PCI Bus 0000:00
000c4000-000c7fff : PCI Bus 0000:00
000c8000-000cbfff : PCI Bus 0000:00
000cc000-000cffff : PCI Bus 0000:00
000d0000-000d3fff : PCI Bus 0000:00
000d4000-000d7fff : PCI Bus 0000:00
000d8000-000dbfff : PCI Bus 0000:00
000dc000-000dffff : PCI Bus 0000:00
000e0000-000e3fff : PCI Bus 0000:00
000e4000-000e7fff : PCI Bus 0000:00
000e8000-000ebfff : PCI Bus 0000:00
000ec000-000effff : PCI Bus 0000:00
000f0000-000fffff : System ROM
00100000-09afffff : System RAM
09b00000-09dfffff : Reserved
09e00000-09efffff : System RAM
09f00000-09f27fff : ACPI Non-volatile Storage
09f28000-a07fefff : System RAM
a07ff000-a07fffff : Reserved
a0800000-a0d1c017 : System RAM
a0d1c018-a0d26e57 : System RAM
a0d26e58-a2363fff : System RAM
a2364000-a4563fff : Reserved
a4564000-a456cfff : System RAM
a456d000-a456ffff : Reserved
a4570000-b077efff : System RAM
b077f000-b2f7efff : Reserved
  b1677000-b16c1fff : AMDI0100:00
  b2ed2000-b2ed5fff : MSFT0101:00
    b2ed2000-b2ed5fff : MSFT0101:00
  b2ed6000-b2ed9fff : MSFT0101:00
    b2ed6000-b2ed9fff : MSFT0101:00
b2f7f000-baf7efff : ACPI Non-volatile Storage
baf7f000-baffefff : ACPI Tables
bafff000-baffffff : System RAM
bb000000-bbffffff : Reserved
bce00000-bfffffff : Reserved
c0000000-dfffffff : PCI Bus 0000:00
  c0000000-c01fffff : PCI Bus 0000:01
    c0000000-c01fffff : 0000:01:00.0
  c0200000-c04fffff : PCI Bus 0000:04
    c0200000-c02fffff : 0000:04:00.0
      c0200000-c02fffff : xhci-hcd
    c0300000-c03fffff : 0000:04:00.3
      c0300000-c03fffff : xhci-hcd
    c0400000-c04fffff : 0000:04:00.4
      c0400000-c04fffff : xhci-hcd
  c0500000-c08fffff : PCI Bus 0000:03
    c0500000-c05fffff : 0000:03:00.3
      c0500000-c05fffff : xhci-hcd
    c0600000-c06fffff : 0000:03:00.4
      c0600000-c06fffff : xhci-hcd
    c0700000-c07fffff : 0000:03:00.2
    c0800000-c087ffff : 0000:03:00.0
    c0880000-c08bffff : 0000:03:00.5
    c08c0000-c08c7fff : 0000:03:00.6
      c08c0000-c08c7fff : ICH HD audio
    c08c8000-c08cbfff : 0000:03:00.1
      c08c8000-c08cbfff : ICH HD audio
    c08cc000-c08cdfff : 0000:03:00.2
  c0900000-c09fffff : PCI Bus 0000:02
    c0900000-c0903fff : 0000:02:00.0
      c0900000-c0903fff : nvme
f0000000-fdc00000 : PCI Bus 0000:00
fde00000-fdefffff : Reserved
  fde00000-fdefffff : pnp 00:00
    fde10510-fde1053f : MSFT0101:00
fec00000-fec003ff : IOAPIC 0
fec01000-fec013ff : IOAPIC 1
fed00000-fed003ff : HPET 0
  fed00000-fed003ff : PNP0103:00
fed45000-fed814ff : PCI Bus 0000:00
  fed80000-fed80fff : Reserved
fed81500-fed818ff : AMDI0030:00
fed81900-fed81fff : PCI Bus 0000:00
fedc0000-fedc0fff : PCI Bus 0000:00
fedc2000-fedc2fff : AMDI0010:00
  fedc2000-fedc2fff : AMDI0010:00
fedc6000-fedc6fff : PCI Bus 0000:00
fee00000-fee00fff : Local APIC
  fee00000-fee00fff : pnp 00:00
ff000000-1ffffffff : Reserved
200000000-41e2fffff : System RAM
  2cfe00000-2d0c00e90 : Kernel code
  2d0c00e91-2d1655dff : Kernel data
  2d1913000-2d1dfffff : Kernel bss
41e300000-41f33ffff : RAM buffer
41f340000-4601fffff : Reserved
460200000-7effffffff : PCI Bus 0000:00
  7ee0000000-7ef01fffff : PCI Bus 0000:03
    7ee0000000-7eefffffff : 0000:03:00.0
      7ee0000000-7ee0bf3fff : efifb
    7ef0000000-7ef01fffff : 0000:03:00.0

BTW: I am running hyperenclave in a machine with dual system, I don't know whether this might have some impact on the result especially for the crash, though it sounds so bizarre.

Looking forward to your reply~

cz-chenzhou commented 1 year ago

The physical memory 0xfe800400 accessed by the power management is not in the e820 table. It may be the bios bug, i suggest you check if there is newer bios and upgrade the bios to the latest version.

Besides, the fault only occur when start hyperenclave, right?

Unik-lif commented 1 year ago

The physical memory 0xfe800400 accessed by the power management is not in the e820 table. It may be the bios bug, i suggest you check if there is newer bios and upgrade the bios to the latest version.

Thank u for u reply! So maybe there is something wrong with the bios of my HONOR magicbook 14. LMAO.

Besides, the fault only occur when start hyperenclave, right?

Yes. The bug only occurs when I start Hyperenclave.

cz-chenzhou commented 1 year ago

Hi Unik-lif,

about the crash issue, we propose another solution, which is to add mappings for the accessed memory not in the e820 table. You can use the hyperenclave patch 0001-Support-handling-NPF-for-linux-vm.patch to confirm if the patch solves the issue. Looking forward to your test result.

Thanks

Unik-lif commented 1 year ago

Ok, recently I am bit of busy. so long as I got some spare time, I will reply as soon as possible.

Bonjourz commented 2 months ago

Hi @Unik-lif ,

This repository has been archived, and moved to a new location: https://github.com/asterinas/hyperenclave.

If you still have such issue, please copy it there: https://github.com/asterinas/hyperenclave/issues and discuss there.