search

intel / linux-sgx

Intel SGX for Linux*
https://www.intel.com/content/www/us/en/developer/tools/software-guard-extensions/linux-overview.html
Other
1.33k stars 544 forks source link

Illegal instruction #240

Open penhoi opened 6 years ago

penhoi commented 6 years ago

Error when executing: ./SampleCode/SampleEnclave

Error Message: Program received signal SIGILL, Illegal instruction. abort () at trts_pic.S:538 538 ud2 (gdb) c Continuing. [sig_handler sig_handler.cpp:93] signal handler is triggered [sig_handler sig_handler.cpp:111] exception on ERESUME [sig_handler sig_handler.cpp:149] NOT enclave signal

Latest commit: 0f45cad401e974bee57704f3fb0ff185146bbde2 Merge: a31b0b6 edb1845 Author: lzha101 lili.z.zhang@intel.com Date: Tue Mar 20 11:03:05 2018 +0800 Merge pull request #231 from lzha101/trts_red_zone Fix red zone issue in continue_execution() Signed-off-by: Zhang Lili Z lili.z.zhang@intel.com

andyzyb commented 6 years ago

Can you elaborate the detailed process how you run the sample code? Have you installed the SDK and PSW, is that in hardware or simulation mode? Does the system support SGX?

penhoi commented 6 years ago

  1. The CPU is Intel® Core™ i7-6600U, with Skylake GT2 integrated graphics card. So my system supports SGX.

  2. I have already installed SDK and PSW. The hash value of last commit is 0f45cad401e974bee57704f3fb0ff185146bbde2

  3. The app is compiled with the SGX-hardware mode. ldd ./app linux-vdso.so.1 => (0x00007ffff7ffa000) libsgx_urts.so => /usr/lib/libsgx_urts.so (0x00007ffff7b99000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007ffff797c000) libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007ffff75fa000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ffff73e4000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ffff701a000) libsgx_uae_service.so => /usr/lib/libsgx_uae_service.so (0x00007ffff6dbc000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ffff6bb8000) /lib64/ld-linux-x86-64.so.2 (0x00007ffff7dd7000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007ffff68af000) libprotobuf.so.9 => /usr/lib/x86_64-linux-gnu/libprotobuf.so.9 (0x00007ffff6591000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007ffff6377000)

  4. Linux distribution uname -a Linux sgx-pad 4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018 x86_64 x86_64 x86_64 GNU/Linu

  5. Call-stack info Program received signal SIGILL, Illegal instruction. abort () at trts_pic.S:538 538 ud2 (gdb) bt

    0 abort () at trts_pic.S:538

    1 0x00007ffff400b27d in __stack_chk_fail () at stack_chk.c:39

    2 0x00007ffff4009224 in init_stack_guard (tcs=0x7ffff47ce000) at ../trts_nsp.cpp:72

    3 0x00007ffff4009277 in enter_enclave (index=18, ms=0x7fffffffd950, tcs=0x7ffff47ce000, cssa=0) at ../trts_nsp.cpp:87

    4 0x00007ffff400af29 in enclave_entry () at trts_pic.S:147

    5 0x00007ffff7b9f94b in __morestack () at enter_enclave.S:77

    6 0x00007ffff7bb0e29 in do_ecall (fn=18, ocall_table=0x607c20 , ms=0x7fffffffd950, trust_thread=0x61cd80) at sig_handler.cpp:240

    7 0x00007ffff7ba783a in CEnclave::ecall (this=0x61f1e0, proc=18, ocall_table=0x607c20 , ms=0x7fffffffd950) at /home/sgx/project/sgx/linux-sgx/psw/urts/enclave.cpp:244

    8 0x00007ffff7babbf9 in sgx_ecall (enclave_id=2, proc=18, ocall_table=0x607c20 , ms=0x7fffffffd950) at /home/sgx/project/sgx/linux-sgx/psw/urts/routine.cpp:52

    9 0x0000000000401f2e in ecall_array_user_check (eid=2, arr=0x7fffffffd990) at App/Enclave_u.c:457

    10 0x000000000040316a in edger8r_array_attributes () at App/Edger8rSyntax/Arrays.cpp:45

    11 0x0000000000402877 in main (argc=1, argv=0x7fffffffdb08) at App/App.cpp:246

yuyuany commented 6 years ago

From the call-stack info, it looks that trts_nsp.cpp was built with -fstack-protector option, which should not be included. Can you check the compile option of trts_nsp.cpp file to see if -fno-stack-protector is included?

penhoi commented 6 years ago

g++ -c -Wnon-virtual-dtor -std=c++11 -fno-stack-protector -O2 -D_FORTIFY_SOURCE=2 -UDEBUG -DNDEBUG -ffunction-sections -fdata-sections -Wall -Wextra -Winit-self -Wpointer-arith -Wreturn-type -Waddress -Wsequence-point -Wformat-security -Wmissing-include-dirs -Wfloat-equal -Wundef -Wshadow -Wcast-align -Wconversion -Wredundant-decls -DITT_ARCH_IA64 -ffreestanding -nostdinc -fvisibility=hidden -fpie -nostdinc++ -Werror -fno-exceptions -fno-rtti -I/home/sgx/project/sgx/linux-sgx/common/inc -I/home/sgx/project/sgx/linux-sgx/common/inc/internal -I/home/sgx/project/sgx/linux-sgx/common/inc/tlibc -fPIC trts_nsp.cpp -o trts_nsp.o

penhoi commented 6 years ago

This problem also exists in v2.1, but not in v1.9 and previous sub-versions. The following is the git-difference of sdk/trts/Makefile:

git diff sgx_2.1 sgx_1.9 Makefile @@ -40,7 +40,7 @@ CXXFLAGS += $(ENCLAVE_CXXFLAGS) \ -fno-exceptions \ -fno-rtti

-TCXXFLAGS := $(patsubst -fstack-protector%,-fno-stack-protector,$(CXXFLAGS)) +TCXXFLAGS := $(filter-out -fstack-protector-strong, $(CXXFLAGS))

yuyuany commented 6 years ago

The compile option you provided looks good, -fno-stack-protector is explicitly added. It's weird that stack-protector is still enabled on trts_nsp.cpp file. What is your GCC version?

penhoi commented 6 years ago

gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper Target: x86_64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu 5.4.0-6ubuntu1~16.04.9' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-libmpx --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu Thread model: posix gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.9)

yuyuany commented 6 years ago

I don't see any problem on your environment. Could you build a debug version of SDK package again?

  1. When you build it, make sure -fno-stack-protector is explicitly added to trts_nsp.cpp compile options (just like the compile option you provided above).
  2. If the problem still exists after you install the new SDK package, run the following command: $ objdump -D $SGX_SDK/lib64/libsgx_trts.a | grep -A 10 ".text.*init_stack_guard" I wonder if %fs:0x28 can be found in your result, just like below: 
    
Disassembly of section .text._ZL16init_stack_guardPv:

0000000000000000 <_ZL16init_stack_guardPv>: 0: 55 push %rbp 1: 48 89 e5 mov %rsp,%rbp 4: 48 83 ec 30 sub $0x30,%rsp 8: 48 89 7d d8 mov %rdi,-0x28(%rbp) c: 64 48 8b 04 25 28 00 mov %fs:0x28,%rax // stack protector 13: 00 00 15: 48 89 45 f8 mov %rax,-0x8(%rbp) 19: 31 c0 xor %eax,%eax

penhoi commented 6 years ago

I reply you later.

2018-04-03 14:58 GMT+08:00 yuyuany notifications@github.com:

I don't see any problem on your environment. Could you build a debug version of SDK package again?

  1. When you build it, make sure -fno-stack-protector is explicitly added to trts_nsp.cpp compile options (just like the compile option you provided above).
  2. If the problem still exists after you install the new SDK package, run the following command: $ objdump -D $SGX_SDK/lib64/libsgx_trts.a | grep -A 10 ".text.*init_stack_guard" I wonder if %fs:0x28 can be found in your result, just like below:

Disassembly of section .text._ZL16init_stack_guardPv:

0000000000000000 <_ZL16init_stack_guardPv>: 0: 55 push %rbp 1: 48 89 e5 mov %rsp,%rbp 4: 48 83 ec 30 sub $0x30,%rsp 8: 48 89 7d d8 mov %rdi,-0x28(%rbp) c: 64 48 8b 04 25 28 00 mov %fs:0x28,%rax // stack protector 13: 00 00 15: 48 89 45 f8 mov %rax,-0x8(%rbp) 19: 31 c0 xor %eax,%eax

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/intel/linux-sgx/issues/240#issuecomment-378148072, or mute the thread https://github.com/notifications/unsubscribe-auth/AJwTsz79gOEWrmoRTTNl5oLL_N0kxCjmks5tkx2TgaJpZM4TCzl2 .

Yourens commented 6 years ago

I have the same problem today, solve it by rebuild intel sgx sdk and psw. Maybe something goes wrong in the first round.

sergefdrv commented 6 years ago

I experienced this some time ago, maybe half a year or more. Reinstalling and rebooting kinda helped...

disconnect3d commented 3 years ago

Since the problem is the example was compiled with stack cookies/canary protection flag maybe the code could check for SSP macros and error out appropriately saying what's wrong? :)

SSP macros from GCC - https://gcc.gnu.org/onlinedocs/cpp/Common-Predefined-Macros.html#Common-Predefined-Macros :

__SSP__
This macro is defined, with value 1, when -fstack-protector is in use.

__SSP_ALL__
This macro is defined, with value 2, when -fstack-protector-all is in use.

__SSP_STRONG__
This macro is defined, with value 3, when -fstack-protector-strong is in use.

__SSP_EXPLICIT__
This macro is defined, with value 4, when -fstack-protector-explicit is in use.