icastats is still empty

[guikarai]

Bug description: icastats not updated Distro release: RHEL 7.4 openssl-ibmca package version [root@ghrhel74crypt ~]# rpm -qa openssl-ibmca openssl-ibmca-1.3.0-2.el7.s390 openssl-ibmca-1.3.0-2.el7.s390x

libica package version [root@ghrhel74crypt ~]# rpm -qa libica libica-3.0.2-2.el7.s390x libica-3.0.2-2.el7.s390

steps to reproduce the bug

1. Install RHEL7.4 on IBM Z14
2. yum install libica libica-utils openssl openssl-ibmca
3. icastats #empty
4. modprobe aes_s390
5. install ibmca engine in openssl
6. openssl speed -evp aes-128-cbc -engine ibmca 7 icastats # still empty (but performance is OK to consider CPACF in use 4,5 GB/S in encryption and 13 GB/s in decryption.

[p-steuer]

Performance will be about the same for openssl and openssl-ibmca-libica stack, since openssl has integrated CPACF support for aes-cbc (KMC instruction).

Loading the aes_s390 kernel module is not needed for openssl/ibmca/libica hw support.

A few questions regarding the icastats problem:

• Was the engine configured correctly? You can use 'openssl engine -c ibmca' to check this. The output should look like:

  (ibmca) Ibmca hardware engine support [ list of engine capabilities ]

• Did you run the commands as root user?

• Do you use the FIPS-build of libica? Is FIPS mode active? You can use 'icainfo' to check this. The last line says something like:

  Built-in FIPS support: FIPS mode inactive.

• Is the prng kernel module loaded? You can check this with 'lsmod | grep prng'.

• Do you have CEX C crypto adapters online? Use 'lszcrypt' to check.

[p-steuer]

Okay, i had a look at a z14 rhel7.4 installation. The libica package has built-in FIPS support.

So my guess is that libica initialization fails because it cannot access the approved entropy sources i.e., /dev/prandom or /dev/hwrng. The former needs the prng kernel module to be loaded, the latter needs online CEX C adapters and the ap kernel module to be loaded.

If my guess is correct, 'icainfo' should say:

Built-in FIPS support: FIPS mode inactive. FIPS SELF-TEST FAILURE. CHECK THE SYSLOG.

And the syslog ('journalctl') will tell you:

Libica DRBG-SHA-512 entropy source failed.

And 'openssl engine -c ibmca' will output:

(ibmca) Ibmca hardware engine support [ list of engine capabilities is empty/not present ]

(if the engine was configured correctly).

The solution is to make an approved entropy source available, as described above.

[guikarai]

Hello dear crypto guys,

You can find below according to your recommandation, output of some commands.

First, 'icainfo' display the following

[root@ghrhel74crypt ~]# icainfo
      Cryptographic algorithm support      
-------------------------------------------
 function      |  hardware  |  software  
---------------+------------+------------
         SHA-1 |    yes     |     yes
       SHA-224 |    yes     |     yes
       SHA-256 |    yes     |     yes
       SHA-384 |    yes     |     yes
       SHA-512 |    yes     |     yes
      SHA3-224 |    yes     |      no
      SHA3-256 |    yes     |      no
      SHA3-384 |    yes     |      no
      SHA3-512 |    yes     |      no
     SHAKE-128 |    yes     |      no
     SHAKE-256 |    yes     |      no
         GHASH |    yes     |      no
         P_RNG |    yes     |     yes
  DRBG-SHA-512 |    yes     |     yes
          ECDH |     no     |     yes
    ECDSA Sign |     no     |     yes
  ECDSA Verify |     no     |     yes
        ECKGEN |     no     |     yes
        RSA ME |    yes     |     yes
       RSA CRT |    yes     |     yes
       DES ECB |    yes     |     yes
       DES CBC |    yes     |     yes
       DES OFB |    yes     |      no
       DES CFB |    yes     |      no
       DES CTR |    yes     |      no
      DES CMAC |    yes     |      no
      3DES ECB |    yes     |     yes
      3DES CBC |    yes     |     yes
      3DES OFB |    yes     |      no
      3DES CFB |    yes     |      no
      3DES CTR |    yes     |      no
     3DES CMAC |    yes     |      no
       AES ECB |    yes     |     yes
       AES CBC |    yes     |     yes
       AES OFB |    yes     |      no
       AES CFB |    yes     |      no
       AES CTR |    yes     |      no
      AES CMAC |    yes     |      no
       AES XTS |    yes     |      no
       AES GCM |    yes     |      no
-------------------------------------------
No built-in FIPS support.

You intuited well with the journalctl:

Feb 28 09:34:47 ghrhel74crypt.mop.fr.ibm.com sshd[54752]: Libica DRBG-SHA-512 entropy source failed.

Regarding openssl and its engine ibmca, i have no error, but there is no presence of a list of capabilities as it is the case in my other implementation:

[root@ghrhel74crypt ~]# openssl engine -c
(dynamic) Dynamic engine loading support
(ibmca) Ibmca hardware engine support

My openssl.cnf looks like the following:

# OpenSSL example configuration file.
# This is mostly being used for generation of certificate requests.
#

# This definition stops the following lines choking if HOME isn't
# defined.
HOME            = .
RANDFILE        = $ENV::HOME/.rnd
openssl_conf        = openssl_def

[TRUNCATED]

#
#openssl_conf = openssl_def

[openssl_def]
engines = engine_section

[engine_section]
ibmca = ibmca_section

[ibmca_section]

# The openssl engine path for libibmca.so.
# Set the dynamic_path to where the libibmca.so engine
# resides on the system.
dynamic_path = /usr/lib64/openssl/engines/libibmca.so
#dynamic_path = /usr/lib/ibmca.so
engine_id = ibmca
init = 1

#
# The following ibmca algorithms will be enabled by these parameters
# to the default_algorithms line. Any combination of these is valid,
# with "ALL" denoting the same as all of them in a comma separated
# list.
#
# RSA
# - RSA encrypt, decrypt, sign and verify, key lengths 512-4096
#
# RAND
# - Hardware random number generation
#
# CIPHERS
# - DES-ECB, DES-CBC, DES-CFB, DES-OFB, DES-EDE3, DES-EDE3-CBC, DES-EDE3-CFB,
#   DES-EDE3-OFB, AES-128-ECB, AES-128-CBC, AES-128-CFB, AES-128-OFB,
#   AES-192-ECB, AES-192-CBC, AES-192-CFB, AES-192-OFB, AES-256-ECB,
#   AES-256-CBC, AES-256-CFB, AES-256-OFB symmetric crypto
#
# DIGESTS
# - SHA1, SHA256, SHA512 digests
#
default_algorithms = ALL
#default_algorithms = RAND,RSA,CIPHERS,DIGESTS

I tested if calls to Crypto Express card 01 configured as accelerators works. openssl speed rsa2048 don't increate counters at all.

[root@ghrhel74crypt ~]# lszcrypt -VVVV
card01: CEX5A       online  hwtype=11 depth=8 request_count=0 pendingq_count=0 requestq_count=0 functions=0x68800000

[p-steuer]

Okay thanks.

Your initial bug report indicates a libica version 3.0.2, but since the icainfo output shows EC algorithms i assume you build and installed a second libica from source (libica master branch).

However, the syslog and the missing capability list indicate, that my first guess is correct.

Did you try the solution i described? I.e, just do 'modprobe prng' and everything should work.

[guikarai]

You know what? Reading once again your post, I did the following and it works.

First I assess that nothing works one more time...: [FAIL]

[root@ghrhel74crypt openssl-ibmca]# icastats
 function     |           hardware       |            software
--------------+--------------------------+-------------------------
              |      ENC    CRYPT   DEC  |      ENC    CRYPT   DEC 
--------------+--------------------------+-------------------------
        SHA-1 |               0          |                0
      SHA-224 |               0          |                0
      SHA-256 |               0          |                0
      SHA-384 |               0          |                0
      SHA-512 |               0          |                0
     SHA3-224 |               0          |                0
     SHA3-256 |               0          |                0
     SHA3-384 |               0          |                0
     SHA3-512 |               0          |                0
    SHAKE-128 |               0          |                0
    SHAKE-256 |               0          |                0
        GHASH |               0          |                0
        P_RNG |               0          |                0
 DRBG-SHA-512 |            2688          |                0
         ECDH |               0          |                0
   ECDSA Sign |               0          |                0
 ECDSA Verify |               0          |                0
       ECKGEN |               0          |                0
       RSA-ME |               0          |                0
      RSA-CRT |               0          |                0
      DES ECB |         0              0 |         0             0
      DES CBC |         0              0 |         0             0
      DES OFB |         0              0 |         0             0
      DES CFB |         0              0 |         0             0
      DES CTR |         0              0 |         0             0
     DES CMAC |         0              0 |         0             0
     3DES ECB |         0              0 |         0             0
     3DES CBC |         0              0 |         0             0
     3DES OFB |         0              0 |         0             0
     3DES CFB |         0              0 |         0             0
     3DES CTR |         0              0 |         0             0
    3DES CMAC |         0              0 |         0             0
      AES ECB |         0              0 |         0             0
      AES CBC |         0              0 |         0             0
      AES OFB |         0              0 |         0             0
      AES CFB |         0              0 |         0             0
      AES CTR |         0              0 |         0             0
     AES CMAC |         0              0 |         0             0
      AES XTS |         0              0 |         0             0
      AES GCM |         0              0 |         0             0

I checked also one more time my ibmca engine for openssl: [FAIL]

[root@ghrhel74crypt openssl-ibmca]# openssl engine -c
(dynamic) Dynamic engine loading support
(ibmca) Ibmca hardware engine support

Then, I checked if rng mod was loaded or not: [FAIL]

[root@ghrhel74crypt openssl-ibmca]# lsmod | grep prng
ansi_cprng             13669  0 

Because not, I started if manually: [SUCESSFUL]

[root@ghrhel74crypt openssl-ibmca]# modprobe prng
[root@ghrhel74crypt openssl-ibmca]# lsmod | grep prng
prng                   15562  0 
ansi_cprng             13669  0 

I issued same commands again: [SUCESSFUL]

[root@ghrhel74crypt openssl-ibmca]# openssl engine -c
(dynamic) Dynamic engine loading support
(ibmca) Ibmca hardware engine support
 [RSA, DSA, DH, RAND, DES-ECB, DES-CBC, DES-OFB, DES-CFB, DES-EDE3, DES-EDE3-CBC, DES-EDE3-OFB, DES-EDE3-CFB, AES-128-ECB, AES-192-ECB, AES-256-ECB, AES-128-CBC, AES-192-CBC, AES-256-CBC, AES-128-OFB, AES-192-OFB, AES-256-OFB, AES-128-CFB, AES-192-CFB, AES-256-CFB, SHA1, SHA256, SHA512]
[root@ghrhel74crypt openssl-ibmca]# 
[root@ghrhel74crypt openssl-ibmca]# icastats
 function     |           hardware       |            software
--------------+--------------------------+-------------------------
              |      ENC    CRYPT   DEC  |      ENC    CRYPT   DEC 
--------------+--------------------------+-------------------------
        SHA-1 |               1          |                0
      SHA-224 |               1          |                0
      SHA-256 |               1          |                0
      SHA-384 |               1          |                0
      SHA-512 |               1          |                0
     SHA3-224 |             247          |                0
     SHA3-256 |               0          |                0
     SHA3-384 |             169          |                0
     SHA3-512 |               1          |                0
    SHAKE-128 |               1          |                0
    SHAKE-256 |               0          |                0
        GHASH |               0          |                0
        P_RNG |               0          |                0
 DRBG-SHA-512 |            2856          |                0
         ECDH |               0          |                0
   ECDSA Sign |               0          |                0
 ECDSA Verify |               2          |                0
       ECKGEN |              11          |                0
       RSA-ME |               2          |                0
      RSA-CRT |               2          |                0
      DES ECB |         1              1 |         0             0
      DES CBC |        14              5 |         0             0
      DES OFB |        49              3 |         0             0
      DES CFB |        12             18 |         0             0
      DES CTR |         3              3 |         0             0
     DES CMAC |         6              6 |         0             0
     3DES ECB |       156              3 |         0             0
     3DES CBC |        96             15 |         0             0
     3DES OFB |         2              2 |         0             0
     3DES CFB |         0              0 |         0             0
     3DES CTR |         0              0 |         0             0
    3DES CMAC |         0              0 |         0             0
      AES ECB |         0              0 |         0             0
      AES CBC |         0              0 |         0             0
      AES OFB |         0              0 |         0             0
      AES CFB |         0              0 |         0             0
      AES CTR |         0              0 |         0             0
     AES CMAC |         0              0 |         0             0
      AES XTS |         0              0 |         0             0
      AES GCM |         0              0 |         0             0

Then, I tried to check if it works also with my crypto cards: [SUCESSFUL]

[root@ghrhel74crypt openssl-ibmca]# openssl speed rsa2048 -engine ibmca
engine "ibmca" set.
Doing 2048 bit private rsa's for 10s: 4611 2048 bit private RSA's in 0.03s
Doing 2048 bit public rsa's for 10s: 6526 2048 bit public RSA's in 0.04s
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,char) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) 
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DB_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -march=z196 -mtune=zEC12 -Wa,--noexecstack -DPURIFY -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DAES_CTR_ASM -DAES_XTS_ASM -DGHASH_ASM
                  sign    verify    sign/s verify/s
rsa 2048 bits 0.000007s 0.000006s 153700.0 163150.0
[root@ghrhel74crypt openssl-ibmca]#
[root@ghrhel74crypt openssl-ibmca]# lszcrypt -VVVV
card01: CEX5A       online  hwtype=11 depth=8 request_count=11564 pendingq_count=0 requestq_count=0 functions=0x68800000

To conclude, I tried to test with an openssl test with aes [SUCESSFUL]**

[root@ghrhel74crypt openssl-ibmca]# openssl speed -evp aes-128-cbc -engine ibmca
engine "ibmca" set.
Doing aes-128-cbc for 3s on 16 size blocks: 22239541 aes-128-cbc's in 2.98s
Doing aes-128-cbc for 3s on 64 size blocks: 20390047 aes-128-cbc's in 2.98s
Doing aes-128-cbc for 3s on 256 size blocks: 16219095 aes-128-cbc's in 2.99s
Doing aes-128-cbc for 3s on 1024 size blocks: 9229856 aes-128-cbc's in 2.98s
Doing aes-128-cbc for 3s on 8192 size blocks: 1751055 aes-128-cbc's in 2.99s
OpenSSL 1.0.2k-fips  26 Jan 2017
built on: reproducible build, date unspecified
options:bn(64,64) md2(int) rc4(8x,char) des(idx,cisc,16,int) aes(partial) idea(int) blowfish(idx) 
compiler: gcc -I. -I.. -I../include  -fPIC -DOPENSSL_PIC -DZLIB -DOPENSSL_THREADS -D_REENTRANT -DDSO_DLFCN -DHAVE_DLFCN_H -DKRB5_MIT -m64 -DB_ENDIAN -Wall -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches   -m64 -march=z196 -mtune=zEC12 -Wa,--noexecstack -DPURIFY -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_GF2m -DRC4_ASM -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DAES_ASM -DAES_CTR_ASM -DAES_XTS_ASM -DGHASH_ASM
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes
aes-128-cbc     119406.93k   437907.05k  1388658.30k  3171601.52k  4797539.32k
[root@ghrhel74crypt openssl-ibmca]# sudo icastats
 function     |           hardware       |            software
--------------+--------------------------+-------------------------
              |      ENC    CRYPT   DEC  |      ENC    CRYPT   DEC 
--------------+--------------------------+-------------------------
        SHA-1 |             364          |                0
      SHA-224 |               4          |                0
      SHA-256 |               4          |                0
      SHA-384 |               4          |                0
      SHA-512 |               4          |                0
     SHA3-224 |             988          |                0
     SHA3-256 |               0          |                0
     SHA3-384 |             676          |                0
     SHA3-512 |           13185          |                0
    SHAKE-128 |            9520          |                0
    SHAKE-256 |               0          |                0
        GHASH |               0          |                0
        P_RNG |               0          |                0
 DRBG-SHA-512 |            2856          |                0
         ECDH |               0          |                0
   ECDSA Sign |               0          |                0
 ECDSA Verify |               8          |                0
       ECKGEN |              44          |                0
       RSA-ME |               8          |                0
      RSA-CRT |               8          |                0
      DES ECB |         4              4 |         0             0
      DES CBC |        56             20 |         0             0
      DES OFB |       196             12 |         0             0
      DES CFB |  69829642             72 |         0             0
      DES CTR |        12             12 |         0             0
     DES CMAC |        24             24 |         0             0
     3DES ECB |       624             12 |         0             0
     3DES CBC |       384             60 |         0             0
     3DES OFB |         8              8 |         0             0
     3DES CFB |         0              0 |         0             0
     3DES CTR |         0              0 |         0             0
    3DES CMAC |         0              0 |         0             0
      AES ECB |         0              0 |         0             0
      AES CBC |         0              0 |         0             0
      AES OFB |         0              0 |         0             0
      AES CFB |         0              0 |         0             0
      AES CTR |         0              0 |         0             0
     AES CMAC |         0              0 |         0             0
      AES XTS |         0              0 |         0             0
      AES GCM |         0              0 |         0             0

I see that, there is a shift in the way data is reported. My aes-cbc test is reported in des-cfb row.

Any idea?

[p-steuer]

Check my reply above in case you missed it.

I assumed a 'modprobe prng' would solve the problem.

Yes, looks like everything is okay now.

The problem with icastats is, as i said, that you have 2 libica installations: The distro package in /usr and the version you build/installed from source in /usr/local. This confuses the icastats counters which are located in shared memory: You can see, that your aes-cbc operations are counted as des-cfb (offset -10) and your rsa operations are counted as sha3-512/shake-128 (offset -10).

To solve this you need to remove one of the libica installations and delete the shared memory segment. After these steps, the counters should work correctly.

[guikarai]

Thank you, I will do that.

[p-steuer]

As everything is working as expected, i would close this issue.

Note that libica versions starting from yesterday's 3.2.1 release will fall back to /dev/urandom for seeding (which is always present) in FIPS-build so that the dependency on kernel modules no longer exists.

[p-steuer]

To further elaborate on the icastats -10 offset:

With 2 libica installations, you have 2 icastats tools: /usr/bin/icastats (distro) /usr/local/bin/icastats (custom)

and two libraries: /usr/lib/libica.so (distro) /usr/local/lib/libica.so (custom)

The older icastats distro version does not have the counters for sha3/shake and EC i.e., the newer custom icastats version has 10 counter fields more, but both tools use the same shared memory segment.

So what happened is that the distros library is used and counts the operations in shared memory according to the distros icastats layout. When you now invoke the custom icastats, all counter fields after the sha3/shake and EC fields have an offset of -10.

That is, if you dont want to remove the distro default installation you can set the corresponding environment variables (LD_LIBRARY_PATH, LD_RUN_PATH ...) accordingly, such that the custom library and the custom icastats tool is used.

[guikarai]

I provisionned another RHEL 7.4.

1. I first assess the issue with libica.

   [root@ghrhel74crypt ~]# icainfo
     Cryptographic algorithm support      
   -------------------------------------------
   function      |  hardware  |  software  
   ---------------+------------+------------
        SHA-1 |  blocked   |   blocked
      SHA-224 |  blocked   |   blocked
      SHA-256 |  blocked   |   blocked
      SHA-384 |  blocked   |   blocked
      SHA-512 |  blocked   |   blocked
        GHASH |  blocked   |   blocked
        P_RNG |  blocked   |   blocked
   DRBG-SHA-512 |  blocked   |   blocked
       RSA ME |  blocked   |   blocked
      RSA CRT |  blocked   |   blocked
      DES ECB |  blocked   |   blocked
      DES CBC |  blocked   |   blocked
      DES OFB |  blocked   |   blocked
      DES CFB |  blocked   |   blocked
      DES CTR |  blocked   |   blocked
     DES CMAC |  blocked   |   blocked
     3DES ECB |  blocked   |   blocked
     3DES CBC |  blocked   |   blocked
     3DES OFB |  blocked   |   blocked
     3DES CFB |  blocked   |   blocked
     3DES CTR |  blocked   |   blocked
    3DES CMAC |  blocked   |   blocked
      AES ECB |  blocked   |   blocked
      AES CBC |  blocked   |   blocked
      AES OFB |  blocked   |   blocked
      AES CFB |  blocked   |   blocked
      AES CTR |  blocked   |   blocked
     AES CMAC |  blocked   |   blocked
      AES XTS |  blocked   |   blocked
   -------------------------------------------
   Built-in FIPS support: FIPS mode inactive.
   FIPS SELF-TEST FAILURE. CHECK THE SYSLOG.

2. Second, I load the prng module.

   [root@ghrhel74crypt ~]# modprobe prng

3. I assess change in the behavior.

   [root@ghrhel74crypt ~]# icainfo
     Cryptographic algorithm support      
   -------------------------------------------
   function      |  hardware  |  software  
   ---------------+------------+------------
        SHA-1 |    yes     |     yes
      SHA-224 |    yes     |     yes
      SHA-256 |    yes     |     yes
      SHA-384 |    yes     |     yes
      SHA-512 |    yes     |     yes
        GHASH |    yes     |      no
        P_RNG |    yes     |     yes
   DRBG-SHA-512 |    yes     |     yes
       RSA ME |     no     |     yes
      RSA CRT |     no     |     yes
      DES ECB |    yes     |     yes
      DES CBC |    yes     |     yes
      DES OFB |    yes     |      no
      DES CFB |    yes     |      no
      DES CTR |    yes     |      no
     DES CMAC |    yes     |      no
     3DES ECB |    yes     |     yes
     3DES CBC |    yes     |     yes
     3DES OFB |    yes     |      no
     3DES CFB |    yes     |      no
     3DES CTR |    yes     |      no
    3DES CMAC |    yes     |      no
      AES ECB |    yes     |     yes
      AES CBC |    yes     |     yes
      AES OFB |    yes     |      no
      AES CFB |    yes     |      no
      AES CTR |    yes     |      no
     AES CMAC |    yes     |      no
      AES XTS |    yes     |      no
   -------------------------------------------
   Built-in FIPS support: FIPS mode inactive.

[p-steuer]

ok, this is the expected behavior with the distro's libica package (no ec/sha3/shake algorithms).