search

usnistgov / ACVP-Server

A repository tracking releases of NIST's ACVP server. See www.github.com/usnistgov/ACVP for the protocol.
51 stars 18 forks source link

Generated Test vectors for RSA PSS with MGF1=SHAKE128 uses 16 byte output length for MGF1. #338

Open slontis opened 4 months ago

slontis commented 4 months ago

There are test vectors in https://github.com/usnistgov/ACVP-Server/blob/master/gen-val/json-files/RSA-SigGen-FIPS186-5/internalProjection.json For the combination of hashAlg=SHAKE128 MGF=MGF1

The signature was only verified when the output size for SHAKE 128 is 32 for the HashAlg and 16 during the MGF. I was expecting the same output length of 32 for both operations? (16 bytes gives only half of the security strength)

Is this mismatch deliberate?, or has this been fixed and new test vectors should be generated?

jbrock24 commented 4 months ago

Hi @slontis, I'll look into this for you, ty.

slontis commented 4 months ago

I am starting to think that using SHAKE inside MGF1 should not be allowed.

In FIPS 186-5 5.4 (b) Specifically states that if "Hash" is SHAKE then the output size MUST BE 32/64 bytes for SHAKE128/256 5.4.1 Mask Generation Functions in RSASSA-PSS allows SHAKE as alternative MGF's. It does not however mention using shake inside MGF1, or what the default size should be.

Using an XOF as a MGF makes sense to me, but using it as a general hash seems much less useful if a fixed output length is required (such as inside MGF1).