intel QAT icp_adf_userProcessToStart failed

[jaminlu]

nginx 1.10.3 +QAT_Engine-0.5.32 +QAT1.7 driver；run the command and here is the outputs: [root@194 conf]# /usr/local/ssl/bin/openssl engine -t -c -vvvv qat [error] icp_sal_userStart() - : icp_adf_userProcessToStart failed

(qat) Reference implementation of QAT crypto engine [RSA, DSA, DH, AES-128-CBC-HMAC-SHA1, AES-128-CBC-HMAC-SHA256, AES-256-CBC-HMAC-SHA1, AES-256-CBC-HMAC-SHA256, TLS1-PRF] [error] icp_sal_userStart() - : icp_adf_userProcessToStart failed

 [ unavailable ]
 ENABLE_EXTERNAL_POLLING: Enables the external polling interface to the engine.
      (input flags): NO_INPUT
 POLL: Polls the engine for any completed requests
      (input flags): NO_INPUT
 SET_INSTANCE_FOR_THREAD: Set instance to be used by this thread
      (input flags): NUMERIC
 GET_NUM_OP_RETRIES: Get number of retries
      (input flags): NO_INPUT
 SET_MAX_RETRY_COUNT: Set maximum retry count
      (input flags): NUMERIC
 SET_INTERNAL_POLL_INTERVAL: Set internal polling interval
      (input flags): NUMERIC
 GET_EXTERNAL_POLLING_FD: Returns non blocking fd for crypto engine
      (input flags): NO_INPUT
 ENABLE_EVENT_DRIVEN_POLLING_MODE: Set event driven polling mode
      (input flags): NO_INPUT
 GET_NUM_CRYPTO_INSTANCES: Get the number of crypto instances
      (input flags): NO_INPUT
 DISABLE_EVENT_DRIVEN_POLLING_MODE: Unset event driven polling mode
      (input flags): NO_INPUT
 SET_EPOLL_TIMEOUT: Set epoll_wait timeout
      (input flags): NUMERIC
 SET_CRYPTO_SMALL_PACKET_OFFLOAD_THRESHOLD: Set QAT small packet threshold
      (input flags): STRING
 ENABLE_INLINE_POLLING: Enables the inline polling mode.
      (input flags): NO_INPUT

[stevelinsell]

Hi @jaminlu,

Can you confirm the following:

Have you followed the troubleshooting section of the engines README.md? - https://github.com/01org/QAT_Engine/blob/master/README.md#troubleshooting

Are you running as a root user? If not you need to follow the instructions in the Intel® QuickAssist Technology Software for Linux* - Programmer's Guide - HW version 1.7 - Section 3.14 - Running Applications as Non-Root User. Here is a direct link to the document, but this may go out of date over time: https://01.org/sites/default/files/downloads/intelr-quickassist-technology/336210qatswprogguiderev003review.pdf

I also explored a similar issue here #35 although in that case the issue was resolved by a clean CentOs install, hopeful that is not required.

If you are still having problems with the openssl speed application initializing the QAT engine after trying the above then let me know and we can explore in more detail.

Kind Regards,

Steve.

[yavtuk]

Hi, I have got the error as same as described above. For me, It was related with openssl options. I ran with and without -multi option in order to verify it

./apps/openssl speed -engine qat -elapsed -multi 2 -evp aes-128-cbc-hmac-sha1

./apps/openssl speed -engine qat -elapsed -evp aes-128-cbc-hmac-sha1

Best regards, Alexey

[mythi]

@stevelinsell FYI, I'm seeing this too.

[error] SalCtrl_AdfServicesStartedCheck() - : Sal Ctrl failed to start in given time

[error] do_userStart() - : Failed to start services

     [ unavailable ]
140676648665280:error:8007B09D:lib(128):qat_engine_init:icp sal userstart fail:e_qat.c:351:

The devices seem OK, according to adf_ctl status.

I've checked the troubleshooting tips and they are all OK, except "lsmod icp_qa_al should be in the list". Which module is that?

[mythi]

@stevelinsell The only thing that looks suspicious here is: Cannot use PF with IOMMU enabled in dmesg. Do I need to turn IOMMU off?

[mythi]

@stevelinsell any thoughts?

[stevelinsell]

Hi @mythi,

Are you running virtualized then? If so did you setup by following: Using Intel® Virtualization Technology (Intel® VT) with Intel® QuickAssist Technology Application Note?

You won't see icp_qa_al in the list of kernel modules as you are running with the QAT 1.7 Driver and icp_qa_al is used for QAT1.5/QAT1.6. The modules you will see will depend on the accelerator you are running with but will most likely be something like:

intel_qat
qat_dh895xcc
qat_dh895xccvf
usdm_drv

In the error output you sent:

[error] SalCtrl_AdfServicesStartedCheck() - : Sal Ctrl failed to start in given time

[error] do_userStart() - : Failed to start services

     [ unavailable ]
140676648665280:error:8007B09D:lib(128):qat_engine_init:icp sal userstart fail:e_qat.c:351:

Do you happen to have the next few lines as it is missing the line with the actual error, that may give a few more hints.

Also what does lspci show when you grep it for the device id of the QuickAssist product you are using? (see the table in section 1.5 of the Virtualization Application Note linked above).

You can try disabling IOMMU if you are not using SR-IOV, the instructions on enabling and disabling it are again in the document linked to above.

Hope something above is useful,

Steve.

[lcmmhcc]

hi, i have got this error too by the non-root user. but it seems no use by following the steps of Section 3.14 - Running Applications as Non-Root User.

[work@tc-op bin]$./openssl engine -t -c -vvvv qat (qat) Reference implementation of QAT crypto engine [RSA, DSA, DH, RAND, AES-128-CBC, AES-256-CBC, RC4, DES-CBC, DES-EDE3-CBC, AES-128-CBC-HMAC-SHA1, AES-256-CBC-HMAC-SHA1, SHA1, MD5, SHA256, SHA512, PRF] [error] SalCtrl_AdfServicesStartedCheck() - : Sal Ctrl failed to start in given time [error] do_userStart() - : Failed to start services

and actually, the engine is available by root user.

[root@tc-op bin]# ./openssl engine -t -c -vvvv qat (qat) Reference implementation of QAT crypto engine [RSA, DSA, DH, RAND, AES-128-CBC, AES-256-CBC, RC4, DES-CBC, DES-EDE3-CBC, AES-128-CBC-HMAC-SHA1, AES-256-CBC-HMAC-SHA1, SHA1, MD5, SHA256, SHA512, PRF] [ available ]

my linux is centos6 kernel 2.6.32_1-12-0-0

[lcmmhcc]

hi, i have got this error too by the non-root user. but it seems no use by following the steps of Section 3.14 - Running Applications as Non-Root User.

[work@tc-op bin]$./openssl engine -t -c -vvvv qat (qat) Reference implementation of QAT crypto engine [RSA, DSA, DH, RAND, AES-128-CBC, AES-256-CBC, RC4, DES-CBC, DES-EDE3-CBC, AES-128-CBC-HMAC-SHA1, AES-256-CBC-HMAC-SHA1, SHA1, MD5, SHA256, SHA512, PRF] [error] SalCtrl_AdfServicesStartedCheck() - : Sal Ctrl failed to start in given time [error] do_userStart() - : Failed to start services

and actually, the engine is available by root user.

[root@tc-op bin]# ./openssl engine -t -c -vvvv qat (qat) Reference implementation of QAT crypto engine [RSA, DSA, DH, RAND, AES-128-CBC, AES-256-CBC, RC4, DES-CBC, DES-EDE3-CBC, AES-128-CBC-HMAC-SHA1, AES-256-CBC-HMAC-SHA1, SHA1, MD5, SHA256, SHA512, PRF] [ available ]

my linux is centos6 kernel 2.6.32_1-12-0-0

resolved , after using chgrp ssl_work /dev/icp* i find this problem by comparing the log of root user and non-root user running " strace ./openssl engine -t -c -vvvv qat"