Questions about cSHAKE256 MCT

[tpj]

I've received this MCT test vector for cSHAKE256: (I set increment=8 in the algorithm form since our cSHAKE256 only handles bytes.)

Test vector:

{
    "vsId": 573821,
    "algorithm": "CSHAKE-256",
    "revision": "1.0",
    "isSample": true,
    "testGroups": [
    ...
        {
            "tgId": 2,
            "testType": "MCT",
            "hexCustomization": false,
            "minOutLen": 16,
            "maxOutLen": 65536,
            "tests": [
                {
                    "tcId": 101,
                    "msg": "E6487934DBA8D81C604F3638832E8BF43796AD8DE34D7683A1EBFFF680C6F6FF",
                    "len": 256,
                    "functionName": "",
                    "customization": ""
                }
            ]
        }
    ]
}

Expected result:

{
    "tgId": 2,
    "tests": [
        {
            "tcId": 101,
            "resultsArray": [
                {
                    "md": "F392EE3BFC744AE1E7C0B2BAD...",
                    "outLen": 24712
                },
                {
                    "md": "9C1C59EFD5E7561EAA840E151...",
                    "outLen": 8016
                },
                {
                    "md": "37E3032E012206AF5145F2784...",
                    "outLen": 15696
                }
            ]
        }
    ]
}

Questions:

• Acccording to the cSHAKE256 MCT (https://pages.nist.gov/ACVP/draft-celi-acvp-xof.html#section-6.2.1) there should be 100 md's in resultArray. Why are there only three in the expected result?

• The MCT specifies Range = (MaxOutLen – MinOutLen + 1) which gives Range = 65536 - 16 + 1 = 65521. But 65521 is not a multiple of 8. So when we in the inner loop compute

  OutputLen = MinOutLen + (8 * Rightmost_Output_bits % Range);

  we end up with a new OutputLen that is not a whole number of bytes. Is this the intention?

• In the inner loop we compute

  Rightmost_output_bits = Right(Output[i], 16); OutputLen = MinOutLen + (8 * Rightmost_Output_bits % Range);

  In the last line the 16-bit Rightmost_output_bits is interpreted as a number. Is this little or big endian?

[livebe01]

Hi @tpj,

I just took a look at your questions related to cSHAKE256 MCT testing. For the first bullet, we only compute the first 3 md's when isSample is set to true in the registration for cSHAKE. isSample set to true is only allowed on demo. Only calculating the first 3 md's is a shortcut we've taken to allow us to generate vector sets more quickly on demo. If you want to perform a test using all 100 rounds, you should be able to accomplish this by setting isSample to false.

Looking at the 2nd bullet, I believe that OutputLen should always evaluate to a whole number of bytes. (8 * Rightmost_Output_bits % Range) will always evaluate to a multiple of 8 and minOutLen is equal to 16 bits. Their sum should always be a multiple of 8.

For #3, 'Rightmost_output_bits' is little endian.

Sorry for not getting back to you before now.

[tpj]

Thanks a lot for the response. I'm still confused regarding the 2nd bullet. Here's the intermediate values I get when running with the MCT vector above:

Initially I compute:

range         = 65521  [ maxOutLen - minOutLen + 1 ]
outputLen     = 65536  [ floor(maxOutLen/8) * 8 ]
customization =        [ empty string ]
functionName  =        [ empty string ]

The first inner iterations (j=0, i=1,2,3):

i=1:
  output[i-1]    = e6487934dba8d81c604f3638832e8bf43796ad8de34d7683a1ebfff680c6f6ff
  innerMsg       = e6487934dba8d81c604f3638832e8bf4
  output[i]      = cSHAKE256(innerMsg, outputLen, functionName, customization)
                 = 012515112f4cc9ff...12ffabb07875c316 [len=65536]
  rightmost_output_bits (bits)     = c316
  rightmost_output_bits (number)   = 5827  [ little endian ]
  outputLen                        = 46632 [ MinOutLen + (8 * Rightmost_Output_bits % Range) ]
  customization                    = WURALMICSBCEBUJKNW
i=2:
  output[i-1]    = 012515112f4cc9ff...12ffabb07875c316 [len=65536]
  innerMsg       = 012515112f4cc9fff8bcc7211dd1cc3a
  output[i]      = cSHAKE256(innerMsg, outputLen, functionName, customization)
                 = fc01ec3d889ef93e...e8028bb452075130 [len=46632]
  rightmost_output_bits (bits)     = 5130
  rightmost_output_bits (number)   = 12369  [ little endian ]
  outputLen                        = 33447 [ MinOutLen + (8 * Rightmost_Output_bits % Range) ]
  customization                    = BLVRVYTVOGRHDBWGDW
i=3:
  output[i-1]    = fc01ec3d889ef93e...e8028bb452075130 [len=46632]
  innerMsg       = fc01ec3d889ef93eed831b8a1f5fc3d6

But for i=3 I'm now stuck, since I have to call CSHAKE with outputLen=33447 which is not a whole number of bytes.

My first thought was that the expression for range is off by one. When I remove the +1 from the expression, i.e., when I just set range = maxOutLen - minOutLen = 65520, then the outputLen is always a full number of bytes, and I can run all 1001 iterations. But doing so, I don't get the right result:

range         = 65520  [ maxOutLen - minOutLen ]
outputLen     = 65536  [ floor(maxOutLen/8) * 8 ]
customization = 
i=1:
  output[i-1]    = e6487934dba8d81c604f3638832e8bf43796ad8de34d7683a1ebfff680c6f6ff
  innerMsg       = e6487934dba8d81c604f3638832e8bf4
  output[i]      = cSHAKE256(innerMsg, outputLen, functionName, customization)
                 = 012515112f4cc9ff...12ffabb07875c316 [len=65536]
  rightmost_output_bits (bits)     = c316
  rightmost_output_bits (number)   = 5827  [ little endian]
  outputLen                        = 46632 [ MinOutLen + (8 * Rightmost_Output_bits % Range) ]
  customization                    = WURALMICSBCEBUJKNW

...
...
...

i=1000:
  output[i-1]    = acf1ef6012c5ca51...aa756ec4be8eb43f [len=64184]
  innerMsg       = acf1ef6012c5ca515ed25121975d94ae
  output[i]      = cSHAKE256(innerMsg, outputLen, functionName, customization)
                 = 35e4a44e286ded19...8b3ba708cce4b04b [len=64960]

The final output[1000] doesn't match the expected output result for j=0 (see above), which says:

"md": "F392EE3BFC744AE1E7C0B2BAD...",
"outLen": 24712

I get the same results using both Go's CSHAKE256 implementation (golang.org/x/crypto/sha3) and a Python CSHAKE implementation (https://github.com/Hemoth/cSHAKE/blob/master/cSHAKEPython/cSHAKE.py), so I think this must be an issue with the way I do the test rather than with the CSHAKE implementation.

What am I doing wrong here?

Could you perhaps point me to some sample CSHAKE256 MCT test vector along with correct intermediate values for just the first few iteration (i.e., correct values of Range, Output, Rightmost_output_bits, Customization). If so, that would make it easier to pinpoint where the test goes wrong.

[livebe01]

Ah, sorry @tpj. You're right about the 2nd bullet... I was goofing the math. Our current cSHAKE MCT implementation (ParallelHash and TupleHash also, I believe) is not honoring increments provided by the user for outputLen and that is incorrect. We'll work on getting this fixed.

[tpj]

Thanks, @livebe01. When you have a fix ready, I'll try it out :thumbsup:

[Kritner]

This change is on demo in release v1.1.0.22

[livebe01]

@tpj see also the updates to the MCT pseudocode here

[livebe01]

This change is on prod in release v1.1.0.22