Invalid encodings in test vectors

[bytemare]

Hello,

I tried testing my implementation in Go with the vectors found here: https://github.com/cfrg/draft-irtf-cfrg-voprf/blob/8ec42c73db97a862827b8eec959a4df3d031adec/poc/vectors/allVectors.json

I encounter errors decoding private key values for Ristretto255. Stripping the 0x, the value is of length 63, which is odd, and Go's hex.DecodeString() returns the error "encoding/hex: odd length hex string". I suppose something went wrong with generating these values.

Maybe testing the values generated by https://github.com/cfrg/draft-irtf-cfrg-voprf/blob/8ec42c73db97a862827b8eec959a4df3d031adec/poc/test_oprf.sage would detect this kind of issues earlier.

[armfazh]

I get into the same issue, am preparing a fix for this.

Encoding numbers must be always padded with zeros.

[bytemare]

Padding seems to solve some of the issues, indeed. Thanks! Decoding from hex now works for me, but some values seem not to be correct for internal representations.

E.g. For Ristretto255Sha512, the value "skSm": "0x038ecf12e5465e4f1362d237521104338cde6717e26a25a5770da7ad85c704c6" fails with the error "invalid scalar encoding" using https://github.com/gtank/ristretto255/blob/9e5f56cbf4b15d3a98c048363d8d52fe11e0e2d0/internal/scalar/scalar.go#L76, in Base mode. But in Verifiable mode, the value "skSm": "0x091a688c1b83ecbc9dac5da9042f60bbda9f332fd6cdf828252920741dbd9c01" decodes correctly to a Ristretto scalar using the same function.

Also: thanks for adding in the cipher suite ID alongside the name in #210 :)

[armfazh]

Here the question is whether the go implementation is compatible with the sage code. @claucece could you please investigate which code is correct?

[chris-wood]

@armfazh I suspect it's an endianness issue. hex dumps the integer in big-endian format, whereas ristretto255 and decaf448 expect scalar encodings to be little-endian. We need EncodeScalar and DecodeScalar-like functions to take care of these group-specific things. Can you (or @claucece) please add that to your change in #210?

[claucece]

ristretto and decaf as @chris-wood said are little endian. The padding with zeros does not ONLY happen with ristretto and decaf although in this case it might have. It happened to me with p384, as well, with some test vectors. So it is NOT about the encoding and decoding of ristretto and decaf, but rather the fact that hex does not include leading zeros. See: https://stackoverflow.com/questions/15884677/python-printing-hex-removes-first-0

[claucece]

Both codes are correct in Golang and sage with the test vectors I added at that time @armfazh .. not sure what you mean by the question. The Golang CIRCL implementation does not include ristretto nor decaf. If you need to include the leading zeros with the python hex func, you can use: str.zfill(): hex(15)[2:].zfill(2).

[armfazh]

The padding with zeros is already solved.

I suspect it's an endianness issue. hex dumps the integer in big-endian format, Can you please add that to your change in #210?

I can change endiannes for these groups.

[claucece]

Cool! Thank you @armfazh ! If you want to see the full testing with the test vectors of the draft of ristretto255 and decaf448 @bytemare , you can find it here: https://github.com/claucece/sage-ristretto255-decaf448

[bytemare]

Awesome, thanks! 👍