Compatibility with other na.cl libraries

[bitlogik]

I initially opened this issue in pynacl, but this is directly linked to the behavior of libsodium.

The XSalsa20Engine of Java BouncyCastle library, and the XSalsa20 Crypto++ library, use tests vectors from the original na.cl and libsodium doesn't output the same encrypted text for crypto_secretbox_xsalsa20poly1305.

People on this repo can help to solve this mystery on why libsodium SecretBox (through pynacl) is providing different result from na.cl, crypto++ and BouncyCastle?

Here's a Python script pointing out the matter : https://gist.github.com/bitlogik/b9368c82aa71aa10299263d65d9486be

[jedisct1]

I don't know how these test vectors were generated, but they don't make sense for SalsaPoly.

The ciphertext and the message have the same size, which, unless the MAC is provided separately, is impossible for an authenticated cipher.

With any secretbox implementation, the ciphertext is 16 bytes longer than the plaintext, which is very likely to be the case with these Python bindings as well.

[bitlogik]

Yes, I corrected the underlying function involved in the Python gist comments. This is about crypto_stream_xsalsa20. Still, the difference is about the encrypted text. And xsalsa20poly1305 is just stream_xsalsa20 with an added MAC.

[bitlogik]

To be more accurate : pynacl.bindings.crypto_secretbox(plaintext, nonce, key)[24:-16] (the encrypted text part, which uses libsodium crypto_secretbox_xsalsa20poly1305 and then crypto_stream_xsalsa20_xor) gives a different result than the crypto_stream_xsalsa20 naclcrypto-20090308 test vector (on which Crypto++ and BouncyCastle xsalsa20 implementations are based).

[jedisct1]

Looks like PyNaCl doesn't support unauthenticated encryption.

So, if you want to do cross-implementation testing, I'd recommend using XSalsaPoly (secret.SecretBox in PyNaCl, crypto_secretbox_easy in libsodium; I'm not familiar with other libraries) for all implementations rather than mixing constructions.

Libsodium's XSalsaPoly functions behave exactly like NaCl and other implementations.

Maybe start with an empty message to better understand how these different libraries work and check that the ciphertext is indeed 16 bytes larger than the message.

Good luck!

[jedisct1]

Here is a secretbox implementation for BouncyCastle. Untested, but it looks pretty straightforward.

[bitlogik]

Thanks. Can you tell how the MAC impact the encrypted message part ? To me crypto_secretbox(m, n, k)[24:-16] should be the same as crypto_stream_xsalsa20_xor(m, n, k).

[jedisct1]

pynacl.bindings.crypto_secretbox(plaintext, nonce, key)[24:-16]

In a secretbox, the tag is at the beginning, the keying material the message is encrypted with starts after the second block, and I don't think PyNaCl prepends a nonce if a nonce was already specified.

So, once again, I'd recommend comparing secretbox with secretbox.

[bitlogik]

OK, so the difference comes from the fact the secretbox use the xsalsa20 in its stream pipeline before encrypting the clear text, the tex is prepended with a tag, so the encrypted text is 100% different. And the test vectors are for xsalsa20, the "raw" function. In this raw test vector the key is directly used. Thanks for you help. The question is now, where can we find "external" test vectors for this secretbox ?

[jedisct1]

You can use these: https://github.com/jedisct1/libsodium/blob/stable/test/default/secretbox_easy.c https://github.com/jedisct1/libsodium/blob/stable/test/default/secretbox_easy.exp