search

keystone-enclave / keystone

Keystone Enclave (QEMU + HiFive Unleashed)
Other
465 stars 134 forks source link

Moving to OpenSBI #212

Closed dayeol closed 3 years ago

dayeol commented 4 years ago

We're moving the security monitor to OpenSBI (https://github.com/riscv/opensbi) so that it's much easier to integrate Keystone with the latest RISC-V platforms. This will also resolve a lot of IPI-related issues (#127) and device tree bugs (#147, #148), and so on.

dayeol commented 4 years ago

I was able to make it work in a single-core machine, but it ended up having a bunch of interface changes. Thus, we will be bumping the major versions of components to v1.0.

Currently, our IPI does not properly work, thus we need to fix in order to run on multi-core machines. @ThaumicMekanism is working on this and will follow up in this thread.

dayeol commented 4 years ago

Note: OpenSBI v0.7 (or higher) requires Linux v5.7-rc1 (or higher) in order to support SMP. We may need to bump Linux v5.7-rc3 (cuz it's what FireMarshal uses).

yintianyu commented 4 years ago

Note: OpenSBI v0.7 (or higher) requires Linux v5.7-rc1 (or higher) in order to support SMP. We may need to bump Linux v5.7-rc3 (cuz it's what FireMarshal uses).

Why is branch dev-firemarshal removed?

dayeol commented 4 years ago

@yintianyu because it was merged #208

fanghuaqi commented 3 years ago

Hi @dayeol, if I want to test keystone with opensbi, which branch should I use?

dayeol commented 3 years ago

Hi Huaqi,

Sorry for the late reply, I've been busy last week.

We're still working on a few more bug fixes, so it's not testable with the public branch. However, if you really want to try it out ahead of time, you can run on single-core machine with this version of OpenSBI:

https://github.com/keystone-enclave/opensbi/tree/ab417a8b2c41aa388a8871c8255655b499bbf5a7

You need to update a few things in your keystone repo:

(1) Build linux image with make, not make vmlinux. I'd modify CMakeListst.txt to omit vmlinux from the Linux build commands. This is because bbl was using vmlinux image, whereas OpenSBI uses the raw kernel image as a payload.

(2) In the fork of openSBI, you can find the SM submodule under lib/utils/experimental/keystone. The submodule itself is in https://github.com/keystone-enclave/sm

Compile OpenSBI with the following commands: make O=build PLATFORM=generic CROSS_COMPILE=riscv64-unknown-elf- FW_PAYLOAD_PATH=/home/ubuntu/keystone/build/linux.build/arch/riscv/boot/Image -j4

and you will find the image in build/platform/generic/firmware/fw_payload.elf

Replace QEMU's -kernel with that image. (Modify <build directory>/scripts/run-qemu.sh)

(3) Finally, change the Linux driver to dev-opensbi branch ( https://github.com/keystone-enclave/linux-keystone-driver/tree/dev-opensbi)

These three will let you boot OpenSBI firmware with Keystone security monitor, and your updated driver will call correct SBIs compatible with the OpenSBI spec.

Let me know if face any problem!

Thanks Dayeol

On Thu, Dec 3, 2020 at 6:03 PM Huaqi Fang notifications@github.com wrote:

Hi @dayeol https://github.com/dayeol, if I want to test keystone with opensbi, which branch should I use?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/keystone-enclave/keystone/issues/212#issuecomment-738507655, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPWYJ6DRKGWTLTH35UVRATSTA7PVANCNFSM4SHZOLDA .

fanghuaqi commented 3 years ago

Thank you @dayeol for your detailed information, I will take a try later.

fanghuaqi commented 3 years ago

Hi @dayeol , I tried with opensbi + sm, but I met an issue here, as below:

 [keystone-test]Enclave init: eapp_file stack, rt_file eyrie-rt
Enclave::init: initFiles
Enclave::init: initDevice
Enclave::init: prepareEnclave
[ 5743.336639] keystone_enclave: keystone_ioctl execute: cmd 8088a400, ff905818
[ 5747.943603] keystone_enclave: epm allocated 8192 page(s) @ 00000000aa000000, CMA=1
[ 5747.958709] keystone_enclave: keystone_ioctl finish: cmd 8088a400, ff905818
[ 5747.973022] keystone_enclave: keystone_ioctl return: cmd 8088a400, ff905818
Enclave::init: mapElf for runtimeFile
Enclave::init: mapElf for enclaveFile
Enclave::init: allocUtm
[ 5748.129333] keystone_enclave: keystone_ioctl execute: cmd 8088a407, ff905828
[ 5748.145629] keystone_enclave: utm allocated 512 page(s) @ ffffffe008600000
[ 5748.159362] keystone_enclave: keystone_ioctl finish: cmd 8088a407, ff905828
[ 5748.173278] keystone_enclave: keystone_ioctl return: cmd 8088a407, ff905828
Enclave::init: loadUntrusted
allocPage ffffffff80000000
[ 5748.219787] test-runner[154]: unhandled signal 11 code 0x1 at 0xffffffffffffffff in libc-2.29.so[3fba995000+fb000]
[ 5748.241149] CPU: 0 PID: 154 Comm: test-runner Tainted: G           O      5.7.0+ #1
[ 5748.256530] epc: 0000003fba9f9a80 ra : 0000000000015fa0 sp : 0000003fff905800
[ 5748.270751]  gp : 000000000004fc40 tp : 0000003fba993b40 t0 : 0000003fff9053a0
[ 5748.285339]  t1 : 000000000001342c t2 : 0000000000000000 s0 : 000000000004fba8
[ 5748.299926]  s1 : 0000000000000fff a0 : 0000000000000001 a1 : 000000000004fba8
[ 5748.314544]  a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000000
[ 5748.329132]  a5 : 000000000004fba9 a6 : 0000000000000000 a7 : 00000000000000de
[ 5748.343719]  s2 : ffffffffffffffff s3 : 0000000000000000 s4 : 0000000000000000
[ 5748.358306]  s5 : 0000000000000008 s6 : 0000003fe06c1810 s7 : 00000000000be688
[ 5748.372924]  s8 : 0000000000000000 s9 : 000000000009ac80 s10: 00000000000c1785
[ 5748.387512]  s11: 000000000009acc0 t3 : 0000003fba9f9a4e t4 : 0000003fff9053d0
[ 5748.402099]  t5 : ffffffff80000000 t6 : 0000000000000010
[ 5748.412902] status: 0000000200000020 badaddr: ffffffffffffffff cause: 000000000000000f
[ 5748.527404] keystone_enclave: fatal: cannot destroy enclave: SBI failed

I didn't met with this issue using the bbl version, I am wondering whether there is something I missed.

BTW, wish you a happy new year in 2021.

Thanks Huaqi

dayeol commented 3 years ago

I am not sure, but did you use dev-opensbi branch for linux-keystone-driver? One thing I probably missed: you should also change SDK branch to dev-opensbi!

fanghuaqi commented 3 years ago

Yes, I am using that branch, it seems that the bug is happening in keystone-runtime executing.

fanghuaqi commented 3 years ago

Hi @dayeol, which keystone-sdk branch and keystone-runtime branch should I use?

dayeol commented 3 years ago

Hi Huaqi,

SDK's dev-opensbi branch will contain the right runtime branch in its CMakeLists.txt. Please refer to the latest commit of SDK's dev-opensbi branch: https://github.com/keystone-enclave/keystone-sdk/commit/eded0e02805f6aee7f07046620962e0c246032cf Let me know if that branch doesn't work. If you checkout the new SDK, you'll need to reinstall it by running make install in the sdk's build directory. Also, if you're testing with QEMU, you need to run make image to rebuild the disk image, because the old image will contain the enclave binaries built with older SDK.

Hopefully, I'll finish testing this OpenSBI version and publish the working version shortly. We are currently incorporating a few changes http://patchwork.ozlabs.org/project/opensbi/list/?series=222283 in OpenSBI for the security monitor. I'll let you know when it's done.

On Tue, Jan 5, 2021 at 12:34 AM Huaqi Fang notifications@github.com wrote:

Hi @dayeol https://github.com/dayeol, which keystone-sdk branch and keystone-runtime branch should I use?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/keystone-enclave/keystone/issues/212#issuecomment-754490060, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAPWYJYLVKLKV54U4HKISTDSYLFKLANCNFSM4SHZOLDA .

fanghuaqi commented 3 years ago

Ok, thank you for your information. @dayeol

dayeol commented 3 years ago

FYI, I'm deprecating riscv-pk with this PR https://github.com/keystone-enclave/keystone/pull/220

fanghuaqi commented 3 years ago

Hi @dayeol , I am able to reproduce the result, but when I integrated it into our SDK using riscv gnu toolchain(gcc 9.2, glibc 2.29), I faced issue show above. Do you have any changes to gcc toolchain?

Thanks

fanghuaqi commented 3 years ago

Hi @dayeol , I tried to use the same toolchain base you used in keystone, and rebuilt toolchain, and intergrated it in our SDK with keystone, and I am able to run keystone tests, and it works well, but when I changed to gcc 9.2, it didn't work. Any compatiable issue in different glibc library?

dayeol commented 3 years ago

I do remember that there was a compatibility issue that we needed to fix. (it was buildroot that causes some error as far as i remember) Could you let me know the exact toolchain commit hash that you're using? I'll try to bump the toolchain.

fanghuaqi commented 3 years ago

Hi @dayeol , I am using this commit in our own repo: https://github.com/riscv-mcu/riscv-gnu-toolchain/tree/8c13bb4f538863f9ddb545aa1c5ec9d0be65cb44

fanghuaqi commented 3 years ago

I find the fault caused running loadUntrusted function when using gcc 9.2 toolchain, and happened with memcpy function

dayeol commented 3 years ago

Could you try unmodified version of riscv-gnu-toolchain with gcc 9.2 to see if the same thing happens?

fanghuaqi commented 3 years ago

Hi, we didn't do any modification related to glibc library, but BTW I will take a try.

fanghuaqi commented 3 years ago

Hi @dayeol , I have tested with riscv gcc 9.2, glibc 2.29, the same error happened.

Create PTE for 3fffc000
PTE mode 1
Create PTE for 3fffd000
PTE mode 1
Create PTE for 3fffe000
PTE mode 1
Create PTE for 3ffff000
PTE mode 1
[   97.646026] keystone_enclave: utm allocated 512 page(s) @ ffffffe008c00000
allocPage ffffffff80000000, ffffffff80200000
Create PTE for ffffffff80000000
PTE mode 4
Map a8e00000, aa000000, 1000
Memcpy ffffffffffffffff, 4eb60, 1000 start
[   97.697052] test-runner[139]: unhandled signal 11 code 0x1 at 0xffffffffffffffff in libc-2.29.so[3ff66a1000+fb000]
[   97.718414] CPU: 0 PID: 139 Comm: test-runner Tainted: G           O      5.7.0+ #1
[   97.734283] epc: 0000003ff6706408 ra : 0000000000015c3a sp : 0000003fffe9f770
[   97.748352]  gp : 000000000004ebf8 tp : 0000003ff669fb40 t0 : 0000000000001000
[   97.762939]  t1 : 000000000001313c t2 : 0000000000000010 s0 : 000000000004eb60
[   97.777526]  s1 : 0000000000000fff a0 : 0000000000000001 a1 : 000000000004eb60
[   97.792114]  a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000000
[   97.806732]  a5 : 000000000004eb61 a6 : 7efefefefefefeff a7 : 0000000000000040
[   97.821319]  s2 : ffffffffffffffff s3 : 0000000000000000 s4 : 0000000000000000
[   97.835906]  s5 : 00000000000fe450 s6 : 0000000000000000 s7 : ffffffffffffffff
[   97.850494]  s8 : 0000000000000000 s9 : 00000000000feb20 s10: 00000000000ca140
[   97.865112]  s11: 0000000000000000 t3 : 0000003ff67063d6 t4 : 0000000000000000
[   97.879669]  t5 : 0000003fffe9f280 t6 : 0000000000000001
[   97.890502] status: 0000000200004020 badaddr: ffffffffffffffff cause: 000000000000000f
[   98.002044] keystone_enclave: fatal: cannot destroy enclave: SBI failed
Segmentation fault
fanghuaqi commented 3 years ago

Hi, I have done a few more tests, I build riscv toolchain with gcc 9.2, but using glibc 2.26, it works well for me, is there any incompatiable issue with glibc 2.26 and glibc 2.29 which might cause above issue.

Here is a full run log 20210119-ks-gcc9.2-glibc2.26-ok.txt

fanghuaqi commented 3 years ago

Failed with glibc 2.27, see 

[  195.021697] keystone_enclave: utm allocated 512 page(s) @ ffffffe008c00000
allocPage ffffffff80000000, ffffffff80200000
Create PTE for ffffffff80000000
PTE mode 4
Map a8e00000, aa000000, 1000
Memcpy ffffffffffffffff, 4eb60, 1000 start
[  195.072296] test-runner[137]: unhandled signal 11 code 0x1 at 0xffffffffffffffff in libc-2.27.so[3fe44fe000+fd000]
[  195.093658] CPU: 0 PID: 137 Comm: test-runner Tainted: G           O      5.7.0+ #1
[  195.109191] epc: 0000003fe456356a ra : 0000000000015c36 sp : 0000003fffacf770
[  195.123535]  gp : 000000000004ebf8 tp : 0000003fe44fcb30 t0 : 0000000000001000
[  195.138000]  t1 : 000000000001313c t2 : 0000000000000010 s0 : 000000000004eb60
[  195.152618]  s1 : 0000000000000fff a0 : 0000000000000001 a1 : 000000000004eb60
[  195.167175]  a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000000
[  195.181793]  a5 : 000000000004eb61 a6 : 7efefefefefefeff a7 : 0000000000000040
[  195.196380]  s2 : ffffffffffffffff s3 : 0000000000000000 s4 : 0000000000000000
[  195.210968]  s5 : 00000000000fcc00 s6 : 0000000000000000 s7 : ffffffffffffffff
[  195.225585]  s8 : 0000000000000000 s9 : 00000000000fca40 s10: 00000000000fe750
[  195.240173]  s11: 0000000000000000 t3 : 0000003fe4563538 t4 : 0000000000000000
[  195.254760]  t5 : 0000003fffacf280 t6 : 0000003fffacf2b0
[  195.265563] status: 0000000200004020 badaddr: ffffffffffffffff cause: 000000000000000f
[  195.381713] keystone_enclave: fatal: cannot destroy enclave: SBI failed
Segmentation fault
dayeol commented 3 years ago

Thanks a lot for the detailed log. I am trying to reproduce this. To clarify, the code is segfaulting at loadUntrusted's memcpy if you used glibc 2.27 and 2.29, and it works fine if you use glibc 2.26? And the device driver/sdk are at dev-opensbi branch?

fanghuaqi commented 3 years ago

Yes, but I am not using the latest version you just pushed several days ago.

For SDK, based on 67b4fa2303939f540477dc02864ca033acbcca62 For linux driver, based on 883cb47190227e7dcb5ee8a78cafdcad2ae75367

fanghuaqi commented 3 years ago

I just tried with keystone master branch using gcc 9.20 + glibc 2.29, and run with sifive qemu, repeated the same issue.

$ git diff conf/
diff --git a/conf/qemu_riscv64_virt_defconfig b/conf/qemu_riscv64_virt_defconfig
index 5478a5e..11c4508 100644
--- a/conf/qemu_riscv64_virt_defconfig
+++ b/conf/qemu_riscv64_virt_defconfig
@@ -8,8 +8,8 @@ BR2_GCC_TARGET_ABI="lp64d"
 BR2_TOOLCHAIN_EXTERNAL=y
 BR2_TOOLCHAIN_EXTERNAL_PATH="$(RISCV)"
 BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="$(ARCH)-unknown-linux-gnu"
-BR2_TOOLCHAIN_EXTERNAL_GCC_7=y
-BR2_TOOLCHAIN_EXTERNAL_HEADERS_4_13=y
+BR2_TOOLCHAIN_EXTERNAL_GCC_9=y
+BR2_TOOLCHAIN_EXTERNAL_HEADERS_5_0=y
 BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y
 BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y
 BR2_TOOLCHAIN_EXTERNAL_CXX=y

Run log:

$ ./scripts/run-qemu.sh 
**** Running QEMU SSH on port 3603 ****
overriding secure boot ROM (file: /home/hqfang/workspace/software/keystone/build_gen9.2/bootrom.build/bootrom.bin)
boot ROM size: 54093
fdt dumped at 58189
bbl loader
Relocated DTB from 000000000000e34d to 0000000083200000

                SIFIVE, INC.

         5555555555555555555555555
        5555                   5555
       5555                     5555
      5555                       5555
     5555       5555555555555555555555
    5555       555555555555555555555555
   5555                             5555
  5555                               5555
 5555                                 5555
5555555555555555555555555555          55555
 55555           555555555           55555
   55555           55555           55555
     55555           5           55555
       55555                   55555
         55555               55555
           55555           55555
             55555       55555
               55555   55555
                 555555555
                   55555
                     5

           SiFive RISC-V Core IP
initializing sm
initialized sm
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 5.3.0-dirty (hqfang@softserver) (gcc version 9.2.0 (GCC)) #1 SMP Thu Jan 21 11:35:46 CST 2021
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] initrd not found or empty - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x00000000ffffffff]
[    0.000000] cma: Reserved 1024 MiB at 0x00000000be000000
[    0.000000] software IO TLB: mapped [mem 0xba000000-0xbe000000] (64MB)
[    0.000000] elf_hwcap is 0x112d
[    0.000000] percpu: Embedded 18 pages/cpu s34520 r8192 d31016 u73728
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 516615
[    0.000000] Kernel command line: console=hvc earlycon=sbi cma=1024M@0-4G
[    0.000000] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes, linear)
[    0.000000] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 933016K/2095104K available (5969K kernel code, 380K rwdata, 1893K rodata, 7177K init, 305K bss, 113512K reserved, 1048576K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu:     RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS: 0, nr_irqs: 0, preallocated irqs: 0
[    0.000000] plic: mapped 53 interrupts with 1 handlers for 2 contexts.
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000116] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
[    0.003084] Console: colour dummy device 80x25
[    0.003640] printk: console [hvc0] enabled
[    0.003640] printk: console [hvc0] enabled
[    0.004238] printk: bootconsole [sbi0] disabled
[    0.004238] printk: bootconsole [sbi0] disabled
[    0.007658] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
[    0.008314] pid_max: default: 32768 minimum: 301
[    0.009606] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.010023] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.035135] rcu: Hierarchical SRCU implementation.
[    0.038031] smp: Bringing up secondary CPUs ...
[    0.038369] smp: Brought up 1 node, 1 CPU
[    0.067519] devtmpfs: initialized
[    0.074267] random: get_random_u32 called from bucket_table_alloc.isra.0+0x4e/0x15e with crng_init=0
[    0.076853] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.077716] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[    0.087538] NET: Registered protocol family 16
[    0.129172] vgaarb: loaded
[    0.130257] SCSI subsystem initialized
[    0.132719] usbcore: registered new interface driver usbfs
[    0.133361] usbcore: registered new interface driver hub
[    0.133920] usbcore: registered new device driver usb
[    0.141774] clocksource: Switched to clocksource riscv_clocksource
[    0.164050] NET: Registered protocol family 2
[    0.167982] tcp_listen_portaddr_hash hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.168644] TCP established hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.169581] TCP bind hash table entries: 16384 (order: 6, 262144 bytes, linear)
[    0.170275] TCP: Hash tables configured (established 16384 bind 16384)
[    0.171993] UDP hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.174677] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes, linear)
[    0.176307] NET: Registered protocol family 1
[    0.179666] RPC: Registered named UNIX socket transport module.
[    0.180109] RPC: Registered udp transport module.
[    0.180409] RPC: Registered tcp transport module.
[    0.180713] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.181396] PCI: CLS 0 bytes, default 64
[    0.359056] workingset: timestamp_bits=62 max_order=19 bucket_order=0
[    0.368817] NFS: Registering the id_resolver key type
[    0.370093] Key type id_resolver registered
[    0.370361] Key type id_legacy registered
[    0.370684] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.371693] 9p: Installing v9fs 9p2000 file system support
[    0.373020] NET: Registered protocol family 38
[    0.373721] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    0.374289] io scheduler mq-deadline registered
[    0.374648] io scheduler kyber registered
[    0.376865] pci-host-generic 30000000.pci: host bridge /soc/pci@30000000 ranges:
[    0.377994] pci-host-generic 30000000.pci:    IO 0x03000000..0x0300ffff -> 0x00000000
[    0.378883] pci-host-generic 30000000.pci:   MEM 0x40000000..0x7fffffff -> 0x40000000
[    0.380827] pci-host-generic 30000000.pci: ECAM at [mem 0x30000000-0x3fffffff] for [bus 00-ff]
[    0.382240] pci-host-generic 30000000.pci: PCI host bridge to bus 0000:00
[    0.382768] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.383185] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.383678] pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
[    0.384812] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000
[    0.388417] pci 0000:00:01.0: [1af4:1005] type 00 class 0x00ff00
[    0.389056] pci 0000:00:01.0: reg 0x10: [io  0x0000-0x001f]
[    0.389760] pci 0000:00:01.0: reg 0x20: [mem 0x00000000-0x00003fff 64bit pref]
[    0.391619] pci 0000:00:01.0: BAR 4: assigned [mem 0x40000000-0x40003fff 64bit pref]
[    0.392259] pci 0000:00:01.0: BAR 0: assigned [io  0x0000-0x001f]
[    0.503015] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.510045] 10000000.uart: ttyS0 at MMIO 0x10000000 (irq = 10, base_baud = 230400) is a 16550A
[    0.512152] [drm] radeon kernel modesetting enabled.
[    0.527543] loop: module loaded
[    0.529880] libphy: Fixed MDIO Bus: probed
[    0.535984] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    0.536310] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.536967] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.537662] ehci-pci: EHCI PCI platform driver
[    0.538130] ehci-platform: EHCI generic platform driver
[    0.538615] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.539020] ohci-pci: OHCI PCI platform driver
[    0.539456] ohci-platform: OHCI generic platform driver
[    0.540177] usbcore: registered new interface driver uas
[    0.540670] usbcore: registered new interface driver usb-storage
[    0.542423] mousedev: PS/2 mouse device common for all mice
[    0.543900] usbcore: registered new interface driver usbhid
[    0.544244] usbhid: USB HID core driver
[    0.546040] NET: Registered protocol family 10
[    0.551725] Segment Routing with IPv6
[    0.552315] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.554981] NET: Registered protocol family 17
[    0.556277] 9pnet: Installing 9P2000 support
[    0.556795] Key type dns_resolver registered
[    0.587205] Freeing unused kernel memory: 7176K
[    0.587523] This architecture does not have kernel memory protection.
[    0.587915] Run /init as init process
Starting syslogd: OK
Starting klogd: OK
Running sysctl: OK
Initializing random number generator... [    1.277988] random: dd: uninitialized urandom read (512 bytes read)
done.
Starting network: udhcpc: started, v1.31.0
udhcpc: sending discover
udhcpc: sending select for 192.168.100.128
udhcpc: lease of 192.168.100.128 obtained, lease time 86400
deleting routers
adding dns 192.168.100.3
OK
Starting dropbear sshd: [    2.061631] random: dropbear: uninitialized urandom read (32 bytes read)
OK

Welcome to Buildroot
buildroot login: root
Password: 
login[113]: root login on 'console'
# ls
keystone-driver.ko  tests.ke
# insmod keystone-driver.ko 
[   11.178268] keystone_driver: loading out-of-tree module taints kernel.
[   11.183820] keystone_enclave: keystone enclave v0.2
# 
# fi
fi    find  
# fi
fi    find  
# ./tests.ke --target tests --noexec
Creating directory tests
Verifying archive integrity... All good.
Uncompressing Keystone Enclave Packagedf: tests: can't find mount point
./tests.ke: line 526: test: Available: integer expression expected

# cd tests
# ./test-runner 
.options_log  eyrie-rt      long-nop      run-test.sh   untrusted
attestation   fib-bench     loop          stack         
data-sealing  fibonacci     malloc        test-runner   
# ./test-runner fibonacci eyrie-rt 
[   51.459090] test-runner[186]: unhandled signal 11 code 0x1 at 0xffffffffffffffff in libc-2.29.so[15557b4000+fb000]
[   51.460042] CPU: 0 PID: 186 Comm: test-runner Tainted: G           O      5.3.0-dirty #1
[   51.460558] sepc: 0000001555819408 ra : 0000000000015b8c sp : 0000003fff845770
[   51.460959]  gp : 000000000004ebf8 tp : 00000015558b9b40 t0 : 00000015557b7d58
[   51.461651]  t1 : 000000000001313c t2 : 000000000004dd20 s0 : 000000000004eb60
[   51.462044]  s1 : 0000000000000fff a0 : 0000000000000001 a1 : 000000000004eb60
[   51.462434]  a2 : 0000000000001000 a3 : 0000000000000000 a4 : 0000000000000000
[   51.462814]  a5 : 000000000004eb61 a6 : 0000000000000000 a7 : 00000000000000de
[   51.463198]  s2 : ffffffffffffffff s3 : 0000000000000000 s4 : 0000000000000000
[   51.463589]  s5 : 00000000000fe1f0 s6 : 0000000000000000 s7 : ffffffffffffffff
[   51.463976]  s8 : 0000000000000000 s9 : 00000000000fed90 s10: 00000000000fe220
[   51.464352]  s11: 0000000000000000 t3 : 00000015558193d6 t4 : 0000000000065088
[   51.464729]  t5 : 0000000000000006 t6 : 0000000000000000
[   51.465017] sstatus: 8000000000006020 sbadaddr: ffffffffffffffff scause: 000000000000000f
[   51.475183] keystone_enclave: fatal: cannot destroy enclave: SBI failed
Segmentation fault
# q
-bash: q: command not found
# 
# ^C
# ^C
# ^C
# ^C
# QEMU: Terminated
dayeol commented 3 years ago

I was able to reproduce the issue and it seems that this is a duplicate of https://github.com/keystone-enclave/keystone-sdk/issues/48. Please follow up in that thread!

dayeol commented 3 years ago

Resolved