Ddnirvana
commented
11 months ago Hi Sadiq, thanks for your comments!
@fengerhu1 @Fly0307 Erhu and Xi, please help check the issue and help Sadiq to fix it out. Thanks!
Closed Sadiq-techie closed 5 months ago
Ddnirvana
commented
11 months ago Hi Sadiq, thanks for your comments!
@fengerhu1 @Fly0307 Erhu and Xi, please help check the issue and help Sadiq to fix it out. Thanks!
Fly0307
commented
11 months ago Hi Sadiq, We're sorry you're having this problem with penglai. After our troubleshooting, the panic occurs because the kernel performs a scanning check on the memory, but it is unable to access the safe memory protected by penglai. Specifically, in openEuler 23.03, this issue may cause intermittent triggering of kernel panics and eventual system termination. However, in openEuler 23.09, this issue most of the time only prompts a kernel scan message, but does not affect program and terminal execution, then you can choose to ignore the prompt. We will fix this problem as soon as possible! If you need to use penglai, you can try openEuler 22.X (linux kernel is 5.X version) or lower version of openEuler, as far as we know currently in linux kernel 5.X did not occur kernel panic problem.
Fly0307
commented
11 months ago This is an issue we have fixed in the latest version. fix kernel panic
Sadiq-techie
commented
11 months ago Hi Dong Du, Fly0307. Thanks for the response and fix. Also there is issue while cross compiling sdk as follows: I was using gcc toolchain version 13.2.0 and was attempting to cross compile sdk but not succeeded.
make[2]: Entering directory '/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/sdk/lib/app'
riscv64-unknown-linux-gnu-gcc -Wall -shared -fPIC -Iinclude -c src/eret.S -o src/eret.o
riscv64-unknown-linux-gnu-gcc -Wall -shared -fPIC -Iinclude -c src/print.c -o src/print.o
src/print.c:92:6: warning: conflicting types for built-in function 'vprintf'; expected 'int(const char *, void *)' [-Wbuiltin-declaration-mismatch]
92 | void vprintf(const char*s, va_list vl)
| ^~~~~~~
src/print.c:6:1: note: 'vprintf' is declared in header '<stdio.h>'
5 | #include "print.h"
+++ |+#include <stdio.h>
6 |
riscv64-unknown-elf-ar rcs ../libpenglai-enclave-eapp.a src/eret.o src/print.o
make[2]: riscv64-unknown-elf-ar: No such file or directory
make[2]: *** [Makefile:24: ../libpenglai-enclave-eapp.a] Error 127
make[2]: Leaving directory '/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/sdk/lib/app'
make[1]: *** [Makefile:2: all] Error 2
make[1]: Leaving directory '/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/sdk/lib'
make: *** [Makefile:3: all] Error 2As per the above log should I use multilib toolchain? When I used multilib toolchain version 12.2.0 facing below issue
Entering directory '/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/demo/gm_test_enclaves'
riscv64-unknown-linux-gnu-ld -static -L/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib -L/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/musl/lib -L/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm -lpenglai-enclave-eapp -lc -o test_sm2 test_sm2.o /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/libpenglai-enclave-eapp.a /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm/sm2.a /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm/miracl.a /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/musl/lib/libc.a /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/libgcc.a -T /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/app.lds
riscv64-unknown-linux-gnu-ld: /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm/sm2.a(Random.o):(.bss+0x0): multiple definition of `g_mem'; /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm/sm2.a(SM2_sv.o):(.bss+0x0): first defined here
riscv64-unknown-linux-gnu-ld: /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm/sm2.a(Random.o):(.bss+0x278): multiple definition of `g_mem_point'; /home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/lib/gm/sm2.a(SM2_sv.o):(.bss+0x278): first defined here
riscv64-unknown-linux-gnu-ld: warning: test_sm2 has a LOAD segment with RWX permissions
make[2]: *** [Makefile:55: test_sm2] Error 1
make[2]: Leaving directory '/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/demo/gm_test_enclaves'
make[1]: *** [Makefile:9: all] Error 2
make[1]: Leaving directory '/home/mdsadiq/qemu-virt/Penglai-openEuler-23.09-qemu-virt/Penglai-Enclave-sPMP/sdk/demo'
make: *** [Makefile:4: all] Error 2 Please let me know your suggestions or comments.
Thanks, MD Sadiq
Fly0307
commented
11 months ago For sdk compiling, we've provided a docker image that contains most of the toolchain for penglai, the toolchain doesn't need to support multilib. you can start the image with this command:
docker run -v $(pwd):/home/penglai/penglai-enclave -w /home/penglai/penglai-enclave --network=host --rm -it ddnirvana/penglai-enclave:v0.5 bash
The version information of the toolchain in the container is as follows:
Target: riscv64-unknown-linux-gnu
Configured with: /home/dd/ddPC5/home/dd/devlop/penglai-enclave/penglai-u-sdk/riscv-gnu-toolchain/riscv-gcc/configure --target=riscv64-unknown-linux-gnu --prefix=/home/dd/ddPC5/home/dd/devlop/penglai-enclave/penglai-u-sdk/toolchain --with-sysroot=/home/dd/ddPC5/home/dd/devlop/penglai-enclave/penglai-u-sdk/toolchain/sysroot --with-system-zlib --enable-shared --enable-tls --enable-languages=c,c++,fortran --disable-libmudflap --disable-libssp --disable-libquadmath --disable-nls --disable-bootstrap --src=/home/dd/ddPC5/home/dd/devlop/penglai-enclave/penglai-u-sdk/riscv-gnu-toolchain/riscv-gcc --enable-checking=yes --disable-multilib --with-abi=lp64d --with-arch=rv64imafdc
Thread model: posix
gcc version 7.2.0 (GCC)You can compile the sdk by executing the following command in the penglai-enclave-sPMP directory:
./docker_cmd.sh docker
# In the docker image
cd sdk
PENGLAI_SDK=$(pwd) make -j8Hi, Thanks for the quick response. Any way to cross compile sdk with custom toolchain without using docker.
Please let me know your comments.
Thanks, MD Sadiq
fengerhu1
commented
11 months ago @Sadiq-techie It seems that you need to use the same version of the cross-compiler as we have in the docker. Also, we'll try to update the compiler and fix any issues with it.
Thanks, Erhu Feng
Sadiq-techie
commented
11 months ago Hi Erhu Feng,
I want to use the same version of cross-compilier to build sdk, driver and opensbi.
Thanks, MD Sadiq
Sadiq-techie
commented
10 months ago Hi Erhu Feng, observed an issue while inserting penglai kernel module as follows
[root@openeuler-riscv64 ~]# insmod penglai-enclave-driver/penglai.ko
[ 2627.267942][ T1170] enclave_ioctl_init...
pmp memory overlaps with existing pmp memory!
[ 2627.270694][ T1170] [Penglai KModule]: sbi call mm_init is failed
insmod: ERROR: could not insert module penglai-enclave-driver/penglai.ko: Operation not permittedIn 1 out of 4 iterations observed this issue, driver cleanup is not occurring correctly during the module unloading process as below
[root@openeuler-riscv64 ~]# rmmod penglai
[ 2946.111615][ T1179] enclave_ioctl_exit...
[ 2946.112191][ T1179] KERNEL MODULE: free secmem:paddr:122000000, vaddr:ffffffd8a2000000, order:22
[ 2946.112597][ T1179] KERNEL MODULE: free secmem:paddr:121800000, vaddr:ffffffd8a1800000, order:22Please let me know your comments.
Thanks, MD Sadiq
Fly0307
commented
10 months ago @Sadiq-techie Can you provide more information about the execution instructions and the output before and after?
I'm not quite sure if you're executing the instructions to install and uninstall the penglai module correctly. If you uninstalled the penglai module correctly, it should output the following message:
M mode: sm_memory_reclaim: There is no mem to reclaim
After executing the rmmod penglai command, it looks like there isn't this output. Please provide more information if it's convenient and we'll fix it as soon as possible! Thanks!
Fly0307
commented
10 months ago Hi Erhu Feng, observed an issue while inserting penglai kernel module as follows
[root@openeuler-riscv64 ~]# insmod penglai-enclave-driver/penglai.ko [ 2627.267942][ T1170] enclave_ioctl_init... pmp memory overlaps with existing pmp memory! [ 2627.270694][ T1170] [Penglai KModule]: sbi call mm_init is failed insmod: ERROR: could not insert module penglai-enclave-driver/penglai.ko: Operation not permittedIn 1 out of 4 iterations observed this issue, driver cleanup is not occurring correctly during the module unloading process as below
[root@openeuler-riscv64 ~]# rmmod penglai [ 2946.111615][ T1179] enclave_ioctl_exit... [ 2946.112191][ T1179] KERNEL MODULE: free secmem:paddr:122000000, vaddr:ffffffd8a2000000, order:22 [ 2946.112597][ T1179] KERNEL MODULE: free secmem:paddr:121800000, vaddr:ffffffd8a1800000, order:22Please let me know your comments.
Thanks, MD Sadiq
We have fixed this issue in the latest version, you can try it at the following commit fix free memory
Sadiq-techie
commented
9 months ago Hi Erhu Feng,
When I was attempting to verify the TEE with the latest sources I was facing below issue
[root@openeuler-riscv64 ~]# insmod penglai.ko
[ 500.802261][ T1110] enclave_ioctl_init...
[ 500.803857][ T1110] [Penglai KModule] register_chrdev succeeded!
[root@openeuler-riscv64 ~]# ./host prime
[ 503.370486][ T1114] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:306
[ 503.370870][ T1114] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1114, name: host
[ 503.371067][ T1114] preempt_count: 1, expected: 0
[ 503.371184][ T1114] CPU: 0 PID: 1114 Comm: host Tainted: G W OE 6.4.0-g0e93d276d93e #1
[ 503.371391][ T1114] Hardware name: riscv-virtio,qemu (DT)
[ 503.371517][ T1114] Call Trace:
[ 503.371601][ T1114] [<ffffffff80006202>] dump_backtrace+0x28/0x30
[ 503.371762][ T1114] [<ffffffff80b94ac4>] show_stack+0x38/0x44
[ 503.371904][ T1114] [<ffffffff80ba853c>] dump_stack_lvl+0x44/0x5c
[ 503.372051][ T1114] [<ffffffff80ba856c>] dump_stack+0x18/0x20
[ 503.372188][ T1114] [<ffffffff8004c9f0>] __might_resched+0x112/0x120
[ 503.372336][ T1114] [<ffffffff8004ca48>] __might_sleep+0x4a/0x72
[ 503.372478][ T1114] [<ffffffff80287920>] __kmem_cache_alloc_node+0x24c/0x2f8
[ 503.372641][ T1114] [<ffffffff80216a20>] kmalloc_trace+0x32/0xc4
[ 503.372788][ T1114] [<ffffffff032ccde2>] create_enclave+0x46/0x46a [penglai]
[ 503.373088][ T1114] [<ffffffff032cd922>] penglai_enclave_create+0x10e/0x6a0 [penglai]
[ 503.373372][ T1114] [<ffffffff032cedce>] penglai_enclave_ioctl+0xa8/0x20a [penglai]
[ 503.373654][ T1114] [<ffffffff802d784e>] sys_ioctl+0x94/0xae
[ 503.373790][ T1114] [<ffffffff80ba906a>] do_trap_ecall_u+0xf0/0x104
[ 503.373939][ T1114] [<ffffffff80003e70>] ret_from_exception+0x0/0x64
host:1: enclave run
[ 503.413973][ T1114] Prime is running
[ 503.414503][ T1114] entry_point: 0000000000001000; untrusted_ptr: 0000001000000000; untrusted_size: 0000000000002000
[ 503.414692][ T1114]
[ 503.418964][ T1114] [Penglai Driver@penglai_enclave_run] run returned successfullyPlease let me know your comments.
Thanks, MD Sadiq
Sadiq-techie
commented
9 months ago Hi Erhu Feng,
After the above mentioned issue, now unable to boot the kernel and stuck with below issue
pmp size should be 2^power!
Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT mode
pmp size should be 2^power!
Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT mode
pmp size should be 2^power!
pmp size should be 2^power!
Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT mode
Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT modePlease let me know your comments.
Thanks, MD Sadiq
Fly0307
commented
9 months ago Hi Erhu Feng,
After the above mentioned issue, now unable to boot the kernel and stuck with below issue
pmp size should be 2^power! Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT mode pmp size should be 2^power! Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT mode pmp size should be 2^power! pmp size should be 2^power! Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT mode Warning: PMP region with paddr: 0x80000000 and size: 0x43000, can't be protected by pmp registers in NAPOT modePlease let me know your comments.
Thanks, MD Sadiq
You can fix this with the following patch：0001-fix-2-power-problem.patch
Sadiq-techie
commented
9 months ago Thanks Erhu Feng for the quick response, will apply this patch and verify.
Sadiq-techie
commented
9 months ago Hi Erhu Feng,
When I was attempting to verify the TEE with the latest sources I was facing below issue
[root@openeuler-riscv64 ~]# insmod penglai.ko
[ 123.154288][ T1082] penglai: loading out-of-tree module taints kernel.
[ 123.156127][ T1082] penglai: module verification failed: signature and/or required key missing - tainting kernel
[ 123.169273][ T1082] enclave_ioctl_init...
[ 123.173332][ T1082] [Penglai KModule] register_chrdev succeeded!
[root@openeuler-riscv64 ~]# ./host prime
[ 258.738913][ T1202] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:306
[ 258.739226][ T1202] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1202, name: host
[ 258.739435][ T1202] preempt_count: 1, expected: 0
[ 258.739706][ T1202] CPU: 3 PID: 1202 Comm: host Tainted: G OE 6.4.0-g0e93d276d93e #1
[ 258.739931][ T1202] Hardware name: riscv-virtio,qemu (DT)
[ 258.740133][ T1202] Call Trace:
[ 258.740323][ T1202] [<ffffffff80006202>] dump_backtrace+0x28/0x30
[ 258.740853][ T1202] [<ffffffff80b94ac4>] show_stack+0x38/0x44
[ 258.740995][ T1202] [<ffffffff80ba853c>] dump_stack_lvl+0x44/0x5c
[ 258.741141][ T1202] [<ffffffff80ba856c>] dump_stack+0x18/0x20
[ 258.741280][ T1202] [<ffffffff8004c9f0>] __might_resched+0x112/0x120
[ 258.741539][ T1202] [<ffffffff8004ca48>] __might_sleep+0x4a/0x72
[ 258.741764][ T1202] [<ffffffff80287920>] __kmem_cache_alloc_node+0x24c/0x2f8
[ 258.741952][ T1202] [<ffffffff80216a20>] kmalloc_trace+0x32/0xc4
[ 258.742117][ T1202] [<ffffffff03793de2>] create_enclave+0x46/0x46a [penglai]
[ 258.742700][ T1202] [<ffffffff03794922>] penglai_enclave_create+0x10e/0x6a0 [penglai]
[ 258.743021][ T1202] [<ffffffff03795dce>] penglai_enclave_ioctl+0xa8/0x20a [penglai]
[ 258.743339][ T1202] [<ffffffff802d784e>] sys_ioctl+0x94/0xae
[ 258.743523][ T1202] [<ffffffff80ba906a>] do_trap_ecall_u+0xf0/0x104
[ 258.743692][ T1202] [<ffffffff80003e70>] ret_from_exception+0x0/0x64
host:1: enclave attest
0xA7, 0x7A, 0xFA, 0xE8, 0x05, 0x43, 0x6C, 0x43, 0x3C, 0x59, 0x67, 0x54, 0x89, 0xA2, 0xCD, 0x5F,
0xD1, 0xB1, 0x12, 0x7F, 0x83, 0x9A, 0xD6, 0xF8, 0x9E, 0x67, 0x75, 0xA1, 0xD4, 0xB2, 0x30, 0xF1,
0x50, 0x54, 0x34, 0xE5, 0x0E, 0xC0, 0x52, 0xCB, 0x19, 0xDB, 0xA2, 0x09, 0x34, 0x73, 0xCB, 0x6C,
0x97, 0xBF, 0xE2, 0xB2, 0x77, 0x87, 0x57, 0x51, 0x46, 0x90, 0x1F, 0xEC, 0x48, 0x49, 0x5F, 0x43,
host:1: enclave run
[ 258.832961][ T1202] Prime is running
[ 258.841051][ T1202] [Penglai Driver@penglai_enclave_run] run returned successfullyPlease let me know your comments or suggestions.
Thanks, MD Sadiq
Sadiq-techie
commented
9 months ago Hi Erhu Feng, below are the observations
with the latest sources observed the kernel bug.
with this commit 9a5e2dc73 there is no kernel bug, demo prime is working as expected. In 1 out of 3 iterations facing below issue
[root@openeuler-riscv64 ~]# insmod penglai.ko
[ 1184.907648][ T1264] enclave_ioctl_init...
pmp memory overlaps with existing pmp memory!
[ 1184.909448][ T1264] [Penglai KModule]: sbi call mm_init is failed
insmod: ERROR: could not insert module penglai.ko: Operation not permittedAfter uninstalling the penglai module correctly it output the following message:
[ 1279.395467][ T1277] enclave_ioctl_exit...
`[ 1279.396234][ T1277] KERNEL MODULE: free secmem:paddr:11a400000, vaddr:ffffffd89a400000, order:22Instead of below message
M mode: sm_memory_reclaim: There is no mem to reclaim
Final output:
[root@openeuler-riscv64 ~]# insmod penglai.ko
[ 1266.564089][ T1267] enclave_ioctl_init...
[ 1266.567769][ T1267] [Penglai KModule] register_chrdev succeeded!
[root@openeuler-riscv64 ~]# ./host prime
[ 1272.364446][ T1272] [Penglai Driver@create_enclave] total_pages:512 order:9
[ 1272.374211][ T1272] [Penglai Driver@penglai_enclave_loadelf] elf_prog_addr:0x1000 elf_prog_size:0x1070, infile_addr:0x3fb03ee000
[ 1272.374287][ T1272] [Penglai Driver@penglai_enclave_loadelf] elf_prog_addr:0x3000 elf_prog_size:0x0, infile_addr:0x3fb03f0000
[ 1272.375006][ T1272] [Penglai Driver@penglai_enclave_create] untrusted_mem->addr:0xffffffd89cbe2000 untrusted_mem->size:0x2000
[ 1272.375775][ T1272] [PENGLAI Driver@acquire_big_lock] penglai_enclave_create get lock
[ 1272.376320][ T1272] [Penglai Driver@penglai_enclave_create] enclave_mem->paddr:0x11f000000, size:0x200000
[ 1272.416968][ T1272] [PENGLAI Driver@release_big_lock] penglai_enclave_create release lock
host:1: enclave attest
[ 1272.418394][ T1272] [PENGLAI Driver@acquire_big_lock] penglai_enclave_attest get lock
[ 1272.438914][ T1272] [PENGLAI Driver@release_big_lock] penglai_enclave_attest release lock
0xC9, 0xF7, 0xEF, 0xD3, 0xCE, 0x4F, 0x31, 0xF9, 0xB5, 0xB9, 0x29, 0xEA, 0x52, 0xC7, 0x5A, 0xAF,
0x76, 0xD8, 0x90, 0x08, 0x24, 0x81, 0x5D, 0xC1, 0xE2, 0x7B, 0x4B, 0x3E, 0x58, 0x88, 0xC3, 0xDC,
0xCE, 0xD7, 0xFD, 0xCD, 0x02, 0x63, 0x18, 0x38, 0xB3, 0x63, 0x39, 0x29, 0x75, 0x42, 0xBA, 0xC0,
0x7D, 0xFA, 0xBC, 0xEF, 0x6B, 0xFD, 0x72, 0x07, 0x2C, 0x6E, 0x68, 0x2B, 0x2D, 0xF0, 0xEC, 0xA9,
host:1: enclave run
[ 1272.441885][ T1272] [PENGLAI Driver@acquire_big_lock] penglai_enclave_run get lock
[ 1272.442334][ T1272] [PENGLAI Driver@release_big_lock] penglai_enclave_run release lock
[ 1272.442647][ T1272] [Penglai Driver@penglai_enclave_run] goto infinite run loop
[ 1272.443498][ T1272] Prime is running
[Penglai Monitor@exit_enclave] retval of enclave is 2002
[ 1272.450513][ T1272] [PENGLAI Driver@acquire_big_lock] penglai_enclave_run get lock
[ 1272.450975][ T1272] [Penglai Driver@penglai_enclave_run] run returned successfully
[ 1272.452802][ T1272] [PENGLAI Driver@release_big_lock] penglai_enclave_run release lock
[root@openeuler-riscv64 ~]# rmmod penglai
[ 1279.395467][ T1277] enclave_ioctl_exit...
[ 1279.396234][ T1277] KERNEL MODULE: free secmem:paddr:11a400000, vaddr:ffffffd89a400000, order:22
` Please let me know your suggestions.
Thanks, MD Sadiq
Fly0307
commented
9 months ago Hi Erhu Feng, below are the observations
with the latest sources observed the kernel bug.
with this commit 9a5e2dc there is no kernel bug, demo prime is working as expected. In 1 out of 3 iterations facing below issue
[root@openeuler-riscv64 ~]# insmod penglai.ko [ 1184.907648][ T1264] enclave_ioctl_init... pmp memory overlaps with existing pmp memory! [ 1184.909448][ T1264] [Penglai KModule]: sbi call mm_init is failed insmod: ERROR: could not insert module penglai.ko: Operation not permittedAfter uninstalling the penglai module correctly it output the following message:
[ 1279.395467][ T1277] enclave_ioctl_exit... `[ 1279.396234][ T1277] KERNEL MODULE: free secmem:paddr:11a400000, vaddr:ffffffd89a400000, order:22Instead of below message
M mode: sm_memory_reclaim: There is no mem to reclaim
You're right, when using the latest commit test and closing the debug output, it prints log like this:
[root@openeuler ~]# rmmod penglai
[ 66.969733][ T826] enclave_ioctl_exit...
M mode:Finish free and there is no mem to reclaimHi Erhu Feng,
When I was attempting to verify the TEE with the latest sources I was facing below issue
[root@openeuler-riscv64 ~]# insmod penglai.ko [ 123.154288][ T1082] penglai: loading out-of-tree module taints kernel. [ 123.156127][ T1082] penglai: module verification failed: signature and/or required key missing - tainting kernel [ 123.169273][ T1082] enclave_ioctl_init... [ 123.173332][ T1082] [Penglai KModule] register_chrdev succeeded! [root@openeuler-riscv64 ~]# ./host prime [ 258.738913][ T1202] BUG: sleeping function called from invalid context at include/linux/sched/mm.h:306 [ 258.739226][ T1202] in_atomic(): 1, irqs_disabled(): 0, non_block: 0, pid: 1202, name: host [ 258.739435][ T1202] preempt_count: 1, expected: 0 [ 258.739706][ T1202] CPU: 3 PID: 1202 Comm: host Tainted: G OE 6.4.0-g0e93d276d93e #1 [ 258.739931][ T1202] Hardware name: riscv-virtio,qemu (DT) [ 258.740133][ T1202] Call Trace: [ 258.740323][ T1202] [<ffffffff80006202>] dump_backtrace+0x28/0x30 [ 258.740853][ T1202] [<ffffffff80b94ac4>] show_stack+0x38/0x44 [ 258.740995][ T1202] [<ffffffff80ba853c>] dump_stack_lvl+0x44/0x5c [ 258.741141][ T1202] [<ffffffff80ba856c>] dump_stack+0x18/0x20 [ 258.741280][ T1202] [<ffffffff8004c9f0>] __might_resched+0x112/0x120 [ 258.741539][ T1202] [<ffffffff8004ca48>] __might_sleep+0x4a/0x72 [ 258.741764][ T1202] [<ffffffff80287920>] __kmem_cache_alloc_node+0x24c/0x2f8 [ 258.741952][ T1202] [<ffffffff80216a20>] kmalloc_trace+0x32/0xc4 [ 258.742117][ T1202] [<ffffffff03793de2>] create_enclave+0x46/0x46a [penglai] [ 258.742700][ T1202] [<ffffffff03794922>] penglai_enclave_create+0x10e/0x6a0 [penglai] [ 258.743021][ T1202] [<ffffffff03795dce>] penglai_enclave_ioctl+0xa8/0x20a [penglai] [ 258.743339][ T1202] [<ffffffff802d784e>] sys_ioctl+0x94/0xae [ 258.743523][ T1202] [<ffffffff80ba906a>] do_trap_ecall_u+0xf0/0x104 [ 258.743692][ T1202] [<ffffffff80003e70>] ret_from_exception+0x0/0x64 host:1: enclave attest 0xA7, 0x7A, 0xFA, 0xE8, 0x05, 0x43, 0x6C, 0x43, 0x3C, 0x59, 0x67, 0x54, 0x89, 0xA2, 0xCD, 0x5F, 0xD1, 0xB1, 0x12, 0x7F, 0x83, 0x9A, 0xD6, 0xF8, 0x9E, 0x67, 0x75, 0xA1, 0xD4, 0xB2, 0x30, 0xF1, 0x50, 0x54, 0x34, 0xE5, 0x0E, 0xC0, 0x52, 0xCB, 0x19, 0xDB, 0xA2, 0x09, 0x34, 0x73, 0xCB, 0x6C, 0x97, 0xBF, 0xE2, 0xB2, 0x77, 0x87, 0x57, 0x51, 0x46, 0x90, 0x1F, 0xEC, 0x48, 0x49, 0x5F, 0x43, host:1: enclave run [ 258.832961][ T1202] Prime is running [ 258.841051][ T1202] [Penglai Driver@penglai_enclave_run] run returned successfullyPlease let me know your comments or suggestions.
Thanks, MD Sadiq
This message does not affect the normal execution of the program, if necessary you can solve this problem with this patch
Hi Dong Du,
Facing kernel panic for openEuler 23.09
Workstation details: Ubuntu 22.04 RISC-V cross compilier 13.2.0 qemu version 8.1.50
Steps to reproduce: Cross compilied u-boot, opensbi-1.2 as per the steps for openEuler >23. when cross compiling sdk it was unsuccessful. used the https://ipads.se.sjtu.edu.cn:1313/d/6a464e02cd3d4c1bafb0/ sdk source and cross compilation was successful.
Booted the openEuler 23.09 with below command
copied files from host to openEuler qemu as follows:
cross compiled penglai-kernel-driver generated penglai.ko insert the enclave driver module insmod penglai.ko
prime enclave execution was successful ./host prime there is kernel call trace after enclave execution and kernel panic after some seconds
Similarly for 23.03 followed same steps with replacing openeuler 23.09 with 23.03 and was facing kernel panic as follows:
openEuler 23.03 uses 6.1.x Linux kernel and openEuler 23.09 uses 6.4.x Linux kernel.
observations: when booted with openEuler 23.03 it is observed mmu was sv57 and with openEuler 23.09 it was sv39. Any issue with OpenSBI?
Please let me know your suggestions or comments.
Thanks, MD Sadiq