Fix the conversion between our sentinel salt length variables and the OpenSSL versions in SignRSAPSS and VerifyRSAPSS . We previously set -1 (hash length equals salt length) when 0 was passed when we should've been setting -2 or -3.
This buggy behavior was inherited from BoringSSL bindings since the very beginning, but it hasn't triggered any bug yet because Go rsa.SignPSS and rsa.VerifyPSS resolved the salt length special case before passing it to boring. There is an upstream CL that will remove this handling from the Go side and fix the Boring side.
We should fix the OpenSSL bindings before that CL lands, else the sync process will fail in the test phase.
Fix the conversion between our sentinel salt length variables and the OpenSSL versions in SignRSAPSS and VerifyRSAPSS . We previously set -1 (hash length equals salt length) when 0 was passed when we should've been setting -2 or -3.
This buggy behavior was inherited from BoringSSL bindings since the very beginning, but it hasn't triggered any bug yet because Go
rsa.SignPSSandrsa.VerifyPSSresolved the salt length special case before passing it to boring. There is an upstream CL that will remove this handling from the Go side and fix the Boring side.We should fix the OpenSSL bindings before that CL lands, else the sync process will fail in the test phase.