search

Tongsuo-Project / Tongsuo

铜锁/Tongsuo is a Modern Cryptographic Primitives and Protocols Library
https://www.tongsuo.net
Apache License 2.0
1.11k stars 180 forks source link

SMx async ssl出现coredump问题 #442

Open oyaya opened 1 year ago

oyaya commented 1 year ago

Ⅰ Issue Description 1.tengine国密开启异步模式（ssl_async on+修改tengine代码以实现)，并且使用cryptoNI指令集的方式, 加速SM2、SM3、SM4

  1. 发送国密握手请求./openssl s_client -connect 127.0.0.1:443 -cipher ECC-SM2-WITH-SM4-SM3 -enable_ntls -ntls 发送请求行GET / HTTP/1.1
  2. 产生coredump image

Ⅱ How to reproduce it (as minimally and precisely as possible)

  1. 编译openssl cd /home/xxx/qat_package/SM_package git clone https://github.com/Tongsuo-Project/Tongsuo.git cd Tongsuo ./config --prefix=/home/xxx/qat_package/SM_package/Tongsuo_install --strict-warnings --api=1.1.1 enable-ntls -d make & make install
    1. build ipp-crypto cd /home/xxx/qat_package/SM_package wget https://github.com/intel/ipp-crypto/archive/refs/tags/ippcp_2021.7.1.zip -O ipp-crypto_2021.7.1.zip unzip ipp-crypto_2021.7.1.zip cmake . -Bbuild -DCMAKE_INSTALL_PREFIX=/home/xxx/qat_package/SM_package/Tongsuo_install \ -DOPENSSL_INCLUDE_DIR=/home/xxx/qat_package/SM_package/Tongsuo_install/include \ -DOPENSSL_LIBRARIES=/home/xxx/qat_package/SM_package/Tongsuo_install/lib64 \ -DOPENSSL_ROOT_DIR=/home/xxx/qat_package/SM_package/Tongsuo_install cd build make -j64 make install
    2. intel-ipsec-mb cd /home/xxx/qat_package/SM_package wget https://github.com/intel/intel-ipsec-mb/archive/refs/tags/v1.3.zip -O intel-ipsec-mb_1.3.zip unzip intel-ipsec-mb_1.3.zip cd intel-ipsec-mb-1.3 make -j SAFE_DATA=y SAFE_PARAM=y SAFE_LOOKUP=y make install NOLDCONFIG=y PREFIX=/home/xxx/qat_package/SM_package/Tongsuo_install
    3. QAT_Engine 编译安装 cd /home/xxx/qat_package/SM_package wget https://github.com/intel/QAT_Engine/archive/refs/tags/v1.1.0.zip -O QAT_Engine_v1.1.0.zip unzip QAT_Engine_v1.1.0.zip cd QAT_Engine-1.1.0 ./autogen.sh ./configure \ --with-openssl_install_dir=/home/xxx/qat_package/SM_package/Tongsuo_install \ --enable-qat_ntls \ --enable-qat_sw \ --disable-qat_hw \ --enable-qat_sw_sm2 \ --enable-qat_sw_sm3 \ --enable-qat_sw_sm4_gcm \ --enable-qat_sw_sm4_ccm \ --enable-qat_sw_sm4_cbc \ --with-qat_sw_crypto_mb_install_dir=/home/xxx/qat_package/SM_package/Tongsuo_install \ --with-qat_sw_ipsec_mb_install_dir=/home/xxx/qat_package/SM_package/Tongsuo_install \ --with-cc-opt="-DNGX_SECURE_MEM -I/home/xxx/qat_package/SM_package/Tongsuo_install/include -Wno-error=deprecated-declarations" \ --with-ld_opt="-L/home/xxx/qat_package/SM_package/Tongsuo_install/lib64 -lcrypto -lssl" make & make install
    4. Tengine编译、安装、配置、启动 git clone https://github.com/alibaba/tengine.git 修改tengine代码使异步开关生效 ./configure --with-debug \ --prefix=/home/xxx/qat_package/SM_package/tengine_install \ --with-http_ssl_module \ --add-module=modules/ngx_tongsuo_ntls \ --with-openssl-async \ --with-cc-opt="-g -DNGX_SECURE_MEM -I/home/xxx/qat_package/SM_package/Tongsuo_install/include -Wno-error=deprecated-declarations" \ --with-ld-opt="-Wl,-rpath=/home/xxx/qat_package/SM_package/Tongsuo_install -L/home/xxx/qat_package/SM_package/Tongsuo_install/lib64" make make install 配置： image 启动Tengine
    5. 发送请求 ./openssl s_client -connect 127.0.0.1:443 -cipher ECC-SM2-WITH-SM4-SM3 -enable_ntls -ntls GET / HTTP/1.1 回车回车
    6. 现象 产生coredump: image

Tengine日志: 2023/07/03 04:51:10 [debug] 650975#0: bind() 0.0.0.0:443 2023/07/03 04:51:10 [notice] 650975#0: using the "epoll" event method 2023/07/03 04:51:10 [notice] 650975#0: nginx/1.22.1 2023/07/03 04:51:10 [notice] 650975#0: Tengine/2.4.0 2023/07/03 04:51:10 [notice] 650975#0: built by gcc 8.3.1 20191121 (Red Hat 8.3.1-5) (GCC) 2023/07/03 04:51:10 [notice] 650975#0: OS: Linux 4.18.0-240.22.1.el8_3.x86_64 2023/07/03 04:51:10 [notice] 650975#0: getrlimit(RLIMIT_NOFILE): 1024:262144 2023/07/03 04:51:10 [debug] 650975#0: write: 10, 00007FFFFFFFD450, 7, 0 2023/07/03 04:51:10 [debug] 650975#0: add cleanup: 000000000076AD98 2023/07/03 04:51:10 [debug] 650975#0: malloc: 00000000007AA7B0:8 2023/07/03 04:51:10 [debug] 650975#0: notify eventfd: 10 2023/07/03 04:51:10 [debug] 650975#0: testing the EPOLLRDHUP flag: success 2023/07/03 04:51:10 [debug] 650975#0: malloc: 0000000000752710:6144 2023/07/03 04:51:10 [debug] 650975#0: malloc: 00007FFFF7F78010:253952 2023/07/03 04:51:10 [debug] 650975#0: malloc: 00000000007B0FF0:106496 2023/07/03 04:51:10 [debug] 650975#0: malloc: 00000000007CB000:106496 2023/07/03 04:51:10 [debug] 650975#0: malloc: 00000000007E5010:106496 2023/07/03 04:51:10 [debug] 650975#0: epoll add event: fd:9 op:1 ev:00002001 2023/07/03 04:51:10 [debug] 650975#0: worker cycle 2023/07/03 04:51:10 [debug] 650975#0: epoll timer: -1 2023/07/03 04:51:14 [debug] 650975#0: epoll: fd:9 ev:0001 d:00007FFFF7F78010 2023/07/03 04:51:14 [debug] 650975#0: accept on 0.0.0.0:443, ready: 0 ... 2023/07/03 04:51:22 [debug] 650975#0: 1 http close request 2023/07/03 04:51:22 [debug] 650975#0: 1 http log handler 2023/07/03 04:51:22 [debug] 650975#0: 1 http status: request_time 232 2023/07/03 04:51:22 [debug] 650975#0: 1 free: 000000000074A610, unused: 0 2023/07/03 04:51:22 [debug] 650975#0: 1 free: 000000000074B620, unused: 2348 2023/07/03 04:51:22 [debug] 650975#0: 1 close http connection: 11 2023/07/03 04:51:22 [debug] 650975#0: 1 SSL_shutdown: -1 2023/07/03 04:51:22 [debug] 650975#0: 1 ngx_ssl_async_process_fds called 2023/07/03 04:51:22 [debug] 650975#0: 1 SSL_get_error: 9 2023/07/03 04:51:22 [debug] 650975#0: 1 ngx_ssl_async_process_fds called 2023/07/03 04:51:22 [debug] 650975#0: 1 epoll del async connection: fd:12 2023/07/03 04:51:22 [debug] 650975#0: 1 epoll del connection: fd:11 2023/07/03 04:51:22 [debug] 650975#0: 1 reusable connection: 0 2023/07/03 04:51:22 [debug] 650975#0: 1 free: 0000000000807030 2023/07/03 04:51:22 [debug] 650975#0: 1 free: 0000000000742460 2023/07/03 04:51:22 [debug] 650975#0: 1 free: 0000000000770270, unused: 24 2023/07/03 04:51:22 [debug] 650975#0: *1 free: 000000000073FC20, unused: 320 2023/07/03 04:51:22 [debug] 650975#0: timer delta: 10 2023/07/03 04:51:22 [debug] 650975#0: worker cycle 2023/07/03 04:51:22 [debug] 650975#0: epoll timer: -1 2023/07/03 04:53:17 [info] 650975#0: epoll_wait() failed (4: Interrupted system call) 2023/07/03 04:53:17 [debug] 650975#0: timer delta: 114730 2023/07/03 04:53:17 [debug] 650975#0: worker cycle 2023/07/03 04:53:17 [debug] 650975#0: epoll timer: -1

Ⅲ. Environment export PATH=/home/xxx/qat_package/SM_package/Tongsuo_install/bin/:$PATH export OPENSSL_ENGINES=/home/xxx/qat_package/SM_package/Tongsuo_install/lib64/engines-3 export OPENSSL_LIB=/home/xxx/qat_package/SM_package/Tongsuo_install/lib64 export LD_LIBRARY_PATH=/home/xxx/qat_package/SM_package/Tongsuo_install/lib64

foxriver1025 commented 1 year ago

@oyaya I meet the same issue and solved it. please see this