Why aes-128-cbc-hmac-sha256 algorithm with a packet length of 4K perform better than that of 8K or even 16K?

[Wang-Robot]

When using QAT acceleration, why does the performance of the aes-128-cbc-hmac-sha256 algorithm with a packet length of 4K perform better than that of 8K or even 16K?

1. Below is my configuration file and compile commands:
   • openssl version OpenSSL 3.2.1
   • compile commands

cd QAT20.L.1.1.40-0018 ./configure make make install

cd QAT_Engine-1.6.0 ./configure --with-qat_hw_dir=/home/QAT20.L.1.1.40-0018/ --enable-qat_hw_gcm make make install

• configuration file

[root@emer` QAT_Engine-1.6.0]# cat /etc/4xxx_dev0.conf 
...
...
...
...
[GENERAL]
ServicesEnabled = sym

ConfigVersion = 2
...
...
...
[SHIM]
NumberCyInstances = 1
NumberDcInstances = 0
NumProcesses = 32
LimitDevAccess = 1
...
...

• openssl engine

  [root@emer ~]# openssl engine -t -c -v qatengine
  (qatengine) Reference implementation of QAT crypto engine(qat_hw) v1.6.0
  [RSA, AES-128-CBC-HMAC-SHA256, AES-256-CBC-HMAC-SHA256, id-aes128-GCM, id-aes192-GCM, id-aes256-GCM, id-aes128-CCM, id-aes192-CCM, id-aes256-CCM, TLS1-PRF, X25519, X448]
   [ available ]
   ENABLE_EXTERNAL_POLLING, POLL, SET_INSTANCE_FOR_THREAD, 
   GET_NUM_OP_RETRIES, SET_MAX_RETRY_COUNT, SET_INTERNAL_POLL_INTERVAL, 
   GET_EXTERNAL_POLLING_FD, ENABLE_EVENT_DRIVEN_POLLING_MODE, 
   GET_NUM_CRYPTO_INSTANCES, DISABLE_EVENT_DRIVEN_POLLING_MODE, 
   SET_EPOLL_TIMEOUT, SET_CRYPTO_SMALL_PACKET_OFFLOAD_THRESHOLD, 
   ENABLE_INLINE_POLLING, ENABLE_HEURISTIC_POLLING, 
   GET_NUM_REQUESTS_IN_FLIGHT, INIT_ENGINE, SET_CONFIGURATION_SECTION_NAME, 
   ENABLE_SW_FALLBACK, HEARTBEAT_POLL, DISABLE_QAT_OFFLOAD, HW_ALGO_BITMAP

  2.Run results (4 cores and 8 cores are bound respectively, and you can see that the performance data of 4K is higher than that of 8K) (the default data length is: 16:64:256:1024:8192:16384; the data of 4096 is run separately)

  
  [root@emer QAT_Engine-1.6.0]# taskset -c 1-4 openssl speed -elapsed -evp aes-128-cbc-hmac-sha256 -async_jobs 48 -multi 4 --engine qatengine
  ...
  ...
  ...
  ...
  Got: +H:16:64:256:1024:8192:16384 from 3
  AES-128-CBC-HMAC-SHA256    21833.30k    89299.73k   364313.06k  1407059.29k  4134684.75k  5558703.45k

[root@emer QAT_Engine-1.6.0]# taskset -c 1-4 openssl speed -elapsed -evp aes-128-cbc-hmac-sha256 -async_jobs 48 -multi 4 --engine qatengine -bytes 4096 ... ... ... ... Got: +H:4096 from 0 AES-128-CBC-HMAC-SHA256 4850803.74k

and

[root@emer QAT_Engine-1.6.0]# taskset -c 1-8 openssl speed -elapsed -evp aes-128-cbc-hmac-sha256 -async_jobs 48 -multi 8 --engine qatengine ... ... ... ... Got: +H:16:64:256:1024:8192:16384 from 7 AES-128-CBC-HMAC-SHA256 43087.70k 177992.30k 736275.11k 2915154.60k 4146301.61k 5514980.01k

[root@emer QAT_Engine-1.6.0]# taskset -c 1-8 openssl speed -elapsed -evp aes-128-cbc-hmac-sha256 -async_jobs 48 -multi 8 --engine qatengine --bytes 4096 ... ... ... ... Got: +H:4096 from 0 AES-128-CBC-HMAC-SHA256 6092023.13k

[venkatesh6911]

Thanks for raising the issue @Wang-Robot . I am looking into this and get back to you soon.

[Wang-Robot]

Thank you, looking forward to your latest progress @venkatesh6911

[venkatesh6911]

There was a discussion on this and it got the feedback that, for 8K and 16K packet sizes, the code path in the QAT firmware is not optimal and it is only generic. That is why we see the underperformance. This has nothing to do with QAT Engine though.

[Wang-Robot]

Thanks. According to our understanding, the larger the packet length, the better the performance. Do we have any special optimization for 4K packet length? In addition, what does QAT firmware mean, qatlib?