search

intel / qatlib

Other
95 stars 34 forks source link

Does qatlib support hash-then-compress? #77

Closed neel-patel-1 closed 8 months ago

neel-patel-1 commented 8 months ago

When enabling dcc in the qat configuration file, sample hash-then-compress sample code shows that hash-then-compress operations are not supported:

Is there a way to configure use of hash-then-compress?

(base) n869p538@sapphire:qatlib$ cat /etc/sysconfig/qat
# Comment or remove next line to disable sriov
#SRIOV_ENABLE=1
POLICY=16
ServicesEnabled=dcc
(base) n869p538@sapphire:qatlib$ sudo systemctl restart qat
(base) n869p538@sapphire:qatlib$ ./cpa_sample_code runTests=128
Latency computation Disabled
qaeMemInit started
icp_sal_userStartMultiProcess("SSL") started
There are no crypto instances
*** QA version information ***
device ID               = 0
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 1
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 2
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 3
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 4
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 5
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 6
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 7
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 8
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 9
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 10
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 11
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 12
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 13
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 14
software                = 24.2.0
*** END QA version information ***
*** QA version information ***
device ID               = 15
software                = 24.2.0
*** END QA version information ***
Inst 0, Affin: 0, Dev: 0, Accel 0, EE 0, BDF F3:01:00
Inst 1, Affin: 1, Dev: 0, Accel 0, EE 0, BDF F3:01:00
Inst 2, Affin: 2, Dev: 0, Accel 0, EE 0, BDF F3:01:00
Inst 3, Affin: 3, Dev: 0, Accel 0, EE 0, BDF F3:01:00
Inst 4, Affin: 4, Dev: 0, Accel 1, EE 0, BDF F3:02:00
Inst 5, Affin: 5, Dev: 0, Accel 1, EE 0, BDF F3:02:00
Inst 6, Affin: 6, Dev: 0, Accel 1, EE 0, BDF F3:02:00
Inst 7, Affin: 7, Dev: 0, Accel 1, EE 0, BDF F3:02:00
Inst 8, Affin: 8, Dev: 0, Accel 2, EE 0, BDF F3:00:01
Inst 9, Affin: 9, Dev: 0, Accel 2, EE 0, BDF F3:00:01
Inst 10, Affin: 10, Dev: 0, Accel 2, EE 0, BDF F3:00:01
Inst 11, Affin: 11, Dev: 0, Accel 2, EE 0, BDF F3:00:01
Inst 12, Affin: 12, Dev: 0, Accel 3, EE 0, BDF F3:01:01
Inst 13, Affin: 13, Dev: 0, Accel 3, EE 0, BDF F3:01:01
Inst 14, Affin: 14, Dev: 0, Accel 3, EE 0, BDF F3:01:01
Inst 15, Affin: 15, Dev: 0, Accel 3, EE 0, BDF F3:01:01
Inst 16, Affin: 16, Dev: 0, Accel 4, EE 0, BDF F3:00:02
Inst 17, Affin: 17, Dev: 0, Accel 4, EE 0, BDF F3:00:02
Inst 18, Affin: 18, Dev: 0, Accel 4, EE 0, BDF F3:00:02
Inst 19, Affin: 19, Dev: 0, Accel 4, EE 0, BDF F3:00:02
Inst 20, Affin: 20, Dev: 0, Accel 5, EE 0, BDF F3:01:02
Inst 21, Affin: 21, Dev: 0, Accel 5, EE 0, BDF F3:01:02
Inst 22, Affin: 22, Dev: 0, Accel 5, EE 0, BDF F3:01:02
Inst 23, Affin: 23, Dev: 0, Accel 5, EE 0, BDF F3:01:02
Inst 24, Affin: 24, Dev: 0, Accel 6, EE 0, BDF F3:00:03
Inst 25, Affin: 25, Dev: 0, Accel 6, EE 0, BDF F3:00:03
Inst 26, Affin: 26, Dev: 0, Accel 6, EE 0, BDF F3:00:03
Inst 27, Affin: 27, Dev: 0, Accel 6, EE 0, BDF F3:00:03
Inst 28, Affin: 28, Dev: 0, Accel 7, EE 0, BDF F3:01:03
Inst 29, Affin: 29, Dev: 0, Accel 7, EE 0, BDF F3:01:03
Inst 30, Affin: 30, Dev: 0, Accel 7, EE 0, BDF F3:01:03
Inst 31, Affin: 31, Dev: 0, Accel 7, EE 0, BDF F3:01:03
Inst 32, Affin: 32, Dev: 0, Accel 8, EE 0, BDF F3:00:04
Inst 33, Affin: 33, Dev: 0, Accel 8, EE 0, BDF F3:00:04
Inst 34, Affin: 34, Dev: 0, Accel 8, EE 0, BDF F3:00:04
Inst 35, Affin: 35, Dev: 0, Accel 8, EE 0, BDF F3:00:04
Inst 36, Affin: 36, Dev: 0, Accel 9, EE 0, BDF F3:01:04
Inst 37, Affin: 37, Dev: 0, Accel 9, EE 0, BDF F3:01:04
Inst 38, Affin: 38, Dev: 0, Accel 9, EE 0, BDF F3:01:04
Inst 39, Affin: 39, Dev: 0, Accel 9, EE 0, BDF F3:01:04
Inst 40, Affin: 0, Dev: 0, Accel 10, EE 0, BDF F3:00:05
Inst 41, Affin: 1, Dev: 0, Accel 10, EE 0, BDF F3:00:05
Inst 42, Affin: 2, Dev: 0, Accel 10, EE 0, BDF F3:00:05
Inst 43, Affin: 3, Dev: 0, Accel 10, EE 0, BDF F3:00:05
Inst 44, Affin: 4, Dev: 0, Accel 11, EE 0, BDF F3:01:05
Inst 45, Affin: 5, Dev: 0, Accel 11, EE 0, BDF F3:01:05
Inst 46, Affin: 6, Dev: 0, Accel 11, EE 0, BDF F3:01:05
Inst 47, Affin: 7, Dev: 0, Accel 11, EE 0, BDF F3:01:05
Inst 48, Affin: 8, Dev: 0, Accel 12, EE 0, BDF F3:00:06
Inst 49, Affin: 9, Dev: 0, Accel 12, EE 0, BDF F3:00:06
Inst 50, Affin: 10, Dev: 0, Accel 12, EE 0, BDF F3:00:06
Inst 51, Affin: 11, Dev: 0, Accel 12, EE 0, BDF F3:00:06
Inst 52, Affin: 12, Dev: 0, Accel 13, EE 0, BDF F3:01:06
Inst 53, Affin: 13, Dev: 0, Accel 13, EE 0, BDF F3:01:06
Inst 54, Affin: 14, Dev: 0, Accel 13, EE 0, BDF F3:01:06
Inst 55, Affin: 15, Dev: 0, Accel 13, EE 0, BDF F3:01:06
Inst 56, Affin: 16, Dev: 0, Accel 14, EE 0, BDF F3:00:07
Inst 57, Affin: 17, Dev: 0, Accel 14, EE 0, BDF F3:00:07
Inst 58, Affin: 18, Dev: 0, Accel 14, EE 0, BDF F3:00:07
Inst 59, Affin: 19, Dev: 0, Accel 14, EE 0, BDF F3:00:07
Inst 60, Affin: 20, Dev: 0, Accel 15, EE 0, BDF F3:01:07
Inst 61, Affin: 21, Dev: 0, Accel 15, EE 0, BDF F3:01:07
Inst 62, Affin: 22, Dev: 0, Accel 15, EE 0, BDF F3:01:07
Inst 63, Affin: 23, Dev: 0, Accel 15, EE 0, BDF F3:01:07
Hash + compress chained operation is not supported on logical instance 2
Hash + compress chained operation is not supported on logical instance 12
Hash + compress chained operation is not supported on logical instance 50
Hash + compress chained operation is not supported on logical instance 6
Hash + compress chained operation is not supported on logical instance 9
Hash + compress chained operation is not supported on logical instance 17
Hash + compress chained operation is not supported on logical instance 60
Hash + compress chained operation is not supported on logical instance 30
Hash + compress chained operation is not supported on logical instance 27
Hash + compress chained operation is not supported on logical instance 35
Hash + compress chained operation is not supported on logical instance 23
Hash + compress chained operation is not supported on logical instance 39
Hash + compress chained operation is not supported on logical instance 55
Hash + compress chained operation is not supported on logical instance 36
Hash + compress chained operation is not supported on logical instance 62
Hash + compress chained operation is not supported on logical instance 48
Hash + compress chained operation is not supported on logical instance 41
Hash + compress chained operation is not supported on logical instance 54
Hash + compress chained operation is not supported on logical instance 47
Hash + compress chained operation is not supported on logical instance 56
Hash + compress chained operation is not supported on logical instance 37
Hash + compress chained operation is not supported on logical instance 52
Hash + compress chained operation is not supported on logical instance 46
Hash + compress chained operation is not supported on logical instance 49
Hash + compress chained operation is not supported on logical instance 53
Hash + compress chained operation is not supported on logical instance 57
Hash + compress chained operation is not supported on logical instance 63
Hash + compress chained operation is not supported on logical instance 0
Hash + compress chained operation is not supported on logical instance 1
Hash + compress chained operation is not supported on logical instance 51
Hash + compress chained operation is not supported on logical instance 10
Hash + compress chained operation is not supported on logical instance 5
Hash + compress chained operation is not supported on logical instance 20
Hash + compress chained operation is not supported on logical instance 7
Hash + compress chained operation is not supported on logical instance 14
Hash + compress chained operation is not supported on logical instance 15
Hash + compress chained operation is not supported on logical instance 59
Hash + compress chained operation is not supported on logical instance 21
Hash + compress chained operation is not supported on logical instance 18
Hash + compress chained operation is not supported on logical instance 61
Hash + compress chained operation is not supported on logical instance 29
Hash + compress chained operation is not supported on logical instance 28
Hash + compress chained operation is not supported on logical instance 24
Hash + compress chained operation is not supported on logical instance 58
Hash + compress chained operation is not supported on logical instance 11
Hash + compress chained operation is not supported on logical instance 32
Hash + compress chained operation is not supported on logical instance 26
Hash + compress chained operation is not supported on logical instance 34
Hash + compress chained operation is not supported on logical instance 40
Hash + compress chained operation is not supported on logical instance 43
Hash + compress chained operation is not supported on logical instance 33
Hash + compress chained operation is not supported on logical instance 13
Hash + compress chained operation is not supported on logical instance 8
Hash + compress chained operation is not supported on logical instance 16
Hash + compress chained operation is not supported on logical instance 19
Hash + compress chained operation is not supported on logical instance 22
Hash + compress chained operation is not supported on logical instance 38
Hash + compress chained operation is not supported on logical instance 31
Hash + compress chained operation is not supported on logical instance 45
Hash + compress chained operation is not supported on logical instance 25
Hash + compress chained operation is not supported on logical instance 42
Hash + compress chained operation is not supported on logical instance 3
Hash + compress chained operation is not supported on logical instance 4
Hash + compress chained operation is not supported on logical instance 44
---------------------------------------
neel-patel-1 commented 8 months ago

Our system is running Linux 6.5.7

Is a newer version required for this feature?

gcabiddu commented 8 months ago

Yes, Kernel 6.7.

neel-patel-1 commented 8 months ago

Thank you

neel-patel-1 commented 8 months ago

Having difficulties enabling chaining on kernel 6.7:

(base) n869p538@sapphire:qatlib$ cat /etc/sysconfig/qat
# Comment or remove next line to disable sriov
#SRIOV_ENABLE=1
POLICY=1
ServicesEnabled=dcc
(base) n869p538@sapphire:qatlib$ uname -r
6.7.0-060700-generic
(base) n869p538@sapphire:interrupt_qat_dc$ sudo systemctl status qat.service
× qat.service - QAT service
     Loaded: loaded (/lib/systemd/system/qat.service; enabled; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2024-03-15 15:46:34 CDT; 38s ago
TriggeredBy: ● qat.timer
    Process: 164468 ExecStartPre=/bin/sh -c test $(getent group qat) (code=exited, status=0/SUCCESS)
    Process: 164475 ExecStartPre=/usr/local/sbin/qat_init.sh (code=exited, status=0/SUCCESS)
    Process: 164763 ExecStart=/usr/local/sbin/qatmgr --policy=${POLICY} (code=exited, status=1/FAILURE)
        CPU: 1.020s

Mar 15 15:46:33 sapphire.ittc.ku.edu qat_init.sh[164475]: /usr/local/sbin/qat_init.sh, device 0000:76:00.0 configured with services: dcc
Mar 15 15:46:33 sapphire.ittc.ku.edu qat_init.sh[164475]: /usr/local/sbin/qat_init.sh, device 0000:f3:00.0 configured with services: dcc
Mar 15 15:46:33 sapphire.ittc.ku.edu qat_init.sh[164475]: /usr/local/sbin/qat_init.sh: 364: echo: echo: I/O error
Mar 15 15:46:34 sapphire.ittc.ku.edu qat_init.sh[164475]: /usr/local/sbin/qat_init.sh: 364: echo: echo: I/O error
Mar 15 15:46:34 sapphire.ittc.ku.edu qatmgr[164765]: No devices found
Mar 15 15:46:34 sapphire.ittc.ku.edu qatmgr[164763]: No QAT device found
Mar 15 15:46:34 sapphire.ittc.ku.edu systemd[1]: qat.service: Control process exited, code=exited, status=1/FAILURE
Mar 15 15:46:34 sapphire.ittc.ku.edu systemd[1]: qat.service: Failed with result 'exit-code'.
Mar 15 15:46:34 sapphire.ittc.ku.edu systemd[1]: Failed to start QAT service.
Mar 15 15:46:34 sapphire.ittc.ku.edu systemd[1]: qat.service: Consumed 1.020s CPU time.

Starting with ServicesEnabled=dc

● qat.service - QAT service
     Loaded: loaded (/lib/systemd/system/qat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-03-15 15:48:03 CDT; 3s ago
TriggeredBy: ● qat.timer
    Process: 165277 ExecStartPre=/bin/sh -c test $(getent group qat) (code=exited, status=0/SUCCESS)
    Process: 165279 ExecStartPre=/usr/local/sbin/qat_init.sh (code=exited, status=0/SUCCESS)
    Process: 165931 ExecStart=/usr/local/sbin/qatmgr --policy=${POLICY} (code=exited, status=0/SUCCESS)
   Main PID: 165933 (qatmgr)
      Tasks: 1 (limit: 308979)
     Memory: 2.3M
        CPU: 1.748s
     CGroup: /system.slice/qat.service
             └─165933 /usr/local/sbin/qatmgr --policy=1

Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: Detected DC configuration
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: section 8, BDF 7602
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: Detected DC configuration
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: section 9, BDF F302
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: Detected DC configuration
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: section 10, BDF 760A
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: Detected DC configuration
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: section 11, BDF F30A
Mar 15 15:48:03 sapphire.ittc.ku.edu systemd[1]: Started QAT service.
Mar 15 15:48:03 sapphire.ittc.ku.edu qatmgr[165933]: Detected DC configuration

Do you know if there is a configuration option I am missing?

gcabiddu commented 8 months ago

Hi @neel-patel-1, can you confirm you have the latest firmware image from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/plain/qat_4xxx.bin

If not, can you copy it to /lib/firmware, reload the qat driver, restart the qat service and optionally rebuild the initramfs?

sudo mv qat_4xxx.bin /lib/firmware
sudo rmmod qat_4xxx intel_qat
sudo modprobe qat_4xxx
systemctl restart qat
dracut -f # if you are using fedora
neel-patel-1 commented 8 months ago 
(base) n869p538@sapphire:interrupt_qat_dc$ sudo mv qat_4xxx.bin /lib/firmware
(base) n869p538@sapphire:interrupt_qat_dc$ sudo rmmod qat_4xxx intel_qat
(base) n869p538@sapphire:interrupt_qat_dc$ sudo modprobe qat_4xxx
(base) n869p538@sapphire:interrupt_qat_dc$ systemctl restart qat

Yes these steps worked:

(base) n869p538@sapphire:interrupt_qat_dc$ sudo systemctl status qat.service
● qat.service - QAT service
     Loaded: loaded (/lib/systemd/system/qat.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2024-03-15 15:57:26 CDT; 39s ago
TriggeredBy: ● qat.timer
    Process: 167094 ExecStartPre=/bin/sh -c test $(getent group qat) (code=exited, status=0/SUCCESS)
    Process: 167096 ExecStartPre=/usr/local/sbin/qat_init.sh (code=exited, status=0/SUCCESS)
    Process: 167806 ExecStart=/usr/local/sbin/qatmgr --policy=${POLICY} (code=exited, status=0/SUCCESS)
   Main PID: 167808 (qatmgr)
      Tasks: 1 (limit: 308979)
     Memory: 1.8M
        CPU: 1.964s
     CGroup: /system.slice/qat.service
             └─167808 /usr/local/sbin/qatmgr --policy=1

Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: Detected DC configuration
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: section 8, BDF 7602
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: Detected DC configuration
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: section 9, BDF F302
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: Detected DC configuration
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: section 10, BDF 760A
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: Detected DC configuration
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: section 11, BDF F30A
Mar 15 15:57:26 sapphire.ittc.ku.edu systemd[1]: Started QAT service.
Mar 15 15:57:26 sapphire.ittc.ku.edu qatmgr[167808]: Detected DC configuration

Thank you for your help