wrk test asynch_mode_nginx, no data

[tangds1234]

hi~，I want to use asynch_mode_nginx to test qat_sw，the following issues occurred. Can you help me see what the reason is？

install asynch_mode_nginx

./configure --prefix=$NGINX_INSTALL_DIR --with-http_ssl_module --add-dynamic-module=modules/nginx_qat_module --with-cc-opt="-DNGX_SECURE_MEM -I$OPENSSL_LIB/include -Wno-error=deprecated-declarations" --with-ld-opt="-Wl,-rpath=$OPENSSL_LIB/lib -L$OPENSSL_LIB/lib" --without-http_rewrite_module --without-http_gzip_module --with-openssl=/mnt/storage01/QAT/openssl-1.1.1j

make -j40
make install

nginx.conf

user root;
worker_processes 1;
# load_module modules/ngx_ssl_engine_qat_module.so;

events {
    use epoll;
    worker_connections 100000;
    multi_accept on;
    accept_mutex on;
}

# ssl_engine {
#   use_engine qatengine;
#   default_algorithms ALL;
#   qat_engine {
#       qat_offload_mode async;
#       qat_notify_mode poll;
#       qat_poll_mode heuristic;
#       qat_shutting_down_release off;
#   }
# }

http {
    server {
        listen 443 ssl reuseport backlog=131072 so_keepalive=off rcvbuf=65536 sndbuf=65536;
        keepalive_timeout 0s;
        ssl_verify_client off;
        ssl_session_tickets off;
        access_log off;
        ssl_asynch on;
        ssl_session_timeout 300s;
        ssl_protocols TLSv1.2;
        ssl_ciphers AES128-SHA;
        ssl_prefer_server_ciphers on;
        ssl_certificate /usr/local/nginx/cert/server.crt;
        ssl_certificate_key /usr/local/nginx/cert/server.key;
        location / {
            root html;
            index index.html index.htm;
        }
    }
}

wrk test

# nginx.conf with ssl_asynch on
./wrk -t 40 -c 10000 -d 30s  https://x.x.x.x:443
Running 30s test @ https://x.x.x.x:443
  40 threads and 10000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     0.00us    0.00us   0.00us    -nan%
    Req/Sec     0.00      0.00     0.00      -nan%
  0 requests in 30.11s, 0.00B read
  Socket errors: connect 3549362, read 0, write 0, timeout 0
Requests/sec:      0.00
Transfer/sec:       0.00B

if i turn off the "ssl_asynch"， it works.

[Yogaraj-Alamenda]

@tangds1234 Can you please enable debug in qatengine build share us the log from log/error.log and also any error message from console ? Please share engine build configure command used as well

[tangds1234]

@tangds1234 Can you please enable debug in qatengine build share us the log from log/error.log and also any error message from console ? Please share engine build configure command used as well

qat_engine configure

./configure --with-openssl_install_dir=/usr/local/ssl --enable-qat_debug --enable-qat_sw

logs/error.log

2024/01/24 17:45:24 [notice] 2215385#0: signal process started
[DEBUG][1379343.179574] PID [2215396] Thread [7ff6a1804b80][e_qat.c:491:qat_engine_finish_int()] ---- QAT Engine Finishing...

[DEBUG][1379343.179578] PID [2215396] Thread [7ff6a1804b80][qat_sw_init.c:453:multibuff_finish_int()] ---- Multibuff Finishing...

[DEBUG][1379343.179694] PID [2215397] Thread [7ff6a1804b80][e_qat.c:461:qat_engine_init()] QAT Engine initialization:
[DEBUG][1379343.179721] PID [2215397] Thread [7ff6a1804b80][qat_sw_init.c:423:multibuff_init()] QAT_SW initialization
[DEBUG][1379343.179724] PID [2215397] Thread [7ff6a1804b80][qat_sw_init.c:424:multibuff_init()] - External polling: ON
[DEBUG][1379343.179726] PID [2215397] Thread [7ff6a1804b80][qat_sw_init.c:425:multibuff_init()] - Heuristic polling: ON

console messages

[WARN][1379343.178467] PID [2215395] Thread [7ff6a1804b80][e_qat.c:858:bind_qat()] QAT Warnings enabled.
[DEBUG][1379343.178480] PID [2215395] Thread [7ff6a1804b80][e_qat.c:859:bind_qat()] QAT Debug enabled.
[WARN][1379343.178484] PID [2215395] Thread [7ff6a1804b80][e_qat.c:860:bind_qat()] qatengine - Reference implementation of QAT crypto engine(qat_sw) v0.6.9
[DEBUG][1379343.178536] PID [2215395] Thread [7ff6a1804b80][e_qat.c:947:bind_qat()] Registering QAT SW supported algorithms
[DEBUG][1379343.178556] PID [2215395] Thread [7ff6a1804b80][e_qat.c:956:bind_qat()] QAT SW RSA Supported
[DEBUG][1379343.178566] PID [2215395] Thread [7ff6a1804b80][qat_sw_rsa.c:125:multibuff_get_RSA_methods()] QAT SW RSA registration succeeded
[DEBUG][1379343.178571] PID [2215395] Thread [7ff6a1804b80][e_qat.c:418:hw_support()] Processor Support - AVX512F = 1, VAES = 1, VPCLMULQDQ = 1
[DEBUG][1379343.178647] PID [2215395] Thread [7ff6a1804b80][qat_evp.c:644:qat_get_EC_methods()] QAT SW ECDSA registration succeeded
[DEBUG][1379343.178664] PID [2215395] Thread [7ff6a1804b80][qat_evp.c:684:qat_get_EC_methods()] QAT SW ECDH registration succeeded
[DEBUG][1379343.178668] PID [2215395] Thread [7ff6a1804b80][e_qat.c:599:qat_engine_ctrl()] Enabled external polling
[DEBUG][1379343.178671] PID [2215395] Thread [7ff6a1804b80][e_qat.c:737:qat_engine_ctrl()] Enabled heuristic polling
[DEBUG][1379343.178675] PID [2215395] Thread [7ff6a1804b80][e_qat.c:461:qat_engine_init()] QAT Engine initialization:
[DEBUG][1379343.178677] PID [2215395] Thread [7ff6a1804b80][qat_sw_init.c:423:multibuff_init()] QAT_SW initialization
[DEBUG][1379343.178680] PID [2215395] Thread [7ff6a1804b80][qat_sw_init.c:424:multibuff_init()] - External polling: ON
[DEBUG][1379343.178683] PID [2215395] Thread [7ff6a1804b80][qat_sw_init.c:425:multibuff_init()] - Heuristic polling: ON
[DEBUG][1379343.179159] PID [2215395] Thread [7ff6a1804b80][e_qat.c:491:qat_engine_finish_int()] ---- QAT Engine Finishing...

[DEBUG][1379343.179163] PID [2215395] Thread [7ff6a1804b80][qat_sw_init.c:453:multibuff_finish_int()] ---- Multibuff Finishing...

[DEBUG][1379343.179285] PID [2215396] Thread [7ff6a1804b80][e_qat.c:461:qat_engine_init()] QAT Engine initialization:
[DEBUG][1379343.179312] PID [2215396] Thread [7ff6a1804b80][qat_sw_init.c:423:multibuff_init()] QAT_SW initialization
[DEBUG][1379343.179316] PID [2215396] Thread [7ff6a1804b80][qat_sw_init.c:424:multibuff_init()] - External polling: ON
[DEBUG][1379343.179319] PID [2215396] Thread [7ff6a1804b80][qat_sw_init.c:425:multibuff_init()] - Heuristic polling: ON

Even if the nginx.conf ssl_engine is not enabled, it's all fail as well.