Keccak256 and secp256k1 Support

[databeforedishonor]

Is it possible to add support for keccak256 and secp256k1? I haven't found good viable options for these 2 within R.

Thanks

[jeroen]

I think the keccak version in openssl is sha3. If your bindings are built against libcrypto 3.0 or higher this should work:

openssl:::rawstringhash('foo', 'sha3-256', NULL)

There is currently no R wrapper, but you can easily add one similar to sha2:

https://github.com/jeroen/openssl/blob/9eb04f6231bc98de31f17b344979f757128ec5d1/R/hash.R#L82-L86

[rmsams]

openssl:::rawstringhash('foo', 'sha3-256', NULL)
[1] "76d3bc41c9f588f7fcd0d5bf4718f8f84b1c41b20882703100b9eb9413807c01"

but keccak256 is:

echo -n "foo" | keccak256   # using https://lib.rs/crates/keccak256-cli
0x41b1a0649752af1b28b3dc29a1556eee781e4a4c3a1f7f53f90fa834de098c4d

I believe keccak support was included in openssl here https://github.com/openssl/openssl/pull/16594

[MrFlick]

When calling rawstringhash, I noticed that EVP_get_digestbyname is being using to find the algorithm. https://github.com/jeroen/openssl/blob/master/src/hash.c#L16C22-L16C42. When I was doing some local testing I noticed that

const char* algo = "KECCAK-256";
const EVP_MD *md = EVP_get_digestbyname(algo)

would return NULL but using EVP_MD_fetch to get the reference would work

OSSL_LIB_CTX *libctx = OSSL_LIB_CTX_new();
 const char* algo = "KECCAK-256";
 const EVP_MD *md = EVP_MD_fetch(libctx, algo, NULL);

So despite the fact that something like this works

echo -n "foo" | openssl dgst -keccak-256 
# KECCAK-256(stdin)= 41b1a0649752af1b28b3dc29a1556eee781e4a4c3a1f7f53f90fa834de098c4d

it seems that particular algorithm still can't be used inside R because sha3-256 does return different values than keccak-256 and openssl:::rawstringhash('foo', 'keccak-256', NULL) reports "Unknown cryptographic algorithm keccak-256"

tested with openssl_2.1.2 where openssl::openssl_config()$version returns OpenSSL 3.1.1 30 May 2023

[rmsams]

@jeroen is there a fix to make openssl:::rawstringhash('foo', 'keccak-256', NULL) work? Keccak is widely used and implemented in openssl.

Some background:

keccak-256 is sha3-256 in the sense that the keccak team won the NIST hash function competition in 2012 and published FIPS 202, which became the official SHA-3 standard in 2015. FIPS 202 contained some tweaks not in the team's original implementation, so we're left with keccak-256 and (standard) sha3-256, and both are widely used.

[MrFlick]

@rmsams Apparently the secretbase package supports the algorithm if you need a solution now.

[rmsams]

@rmsams Apparently the secretbase package supports the algorithm if you need a solution now.

I wasn't aware, thank's for the reference @MrFlick

[jeroen]

Thanks, I've tried to add it here to based on https://github.com/jeroen/openssl/issues/117#issuecomment-2072978413

[jeroen]

Hmm it looks like keccak is only available on openssl 3.2 and up...

[jeroen]

OK I have released this on CRAN. So you can use keccak() on Windows and Mac at least, but on Linux it requires a distro with at least libssl-dev 3.2 or newer.

[rmsams]

That was fast, thanks @jeroen !

but on Linux it requires a distro with at least libssl-dev 3.2 or newer.

Yes, that's a shame b/c it will take some time before libssl-dev >=3.2 makes it into the LTS releases of most linux distros. I suppose you could not export keccak() so that all the exposed user interfaces can be expected to work in a platform-independent way; those of us on linux who want the functionality will still have it, and be aware of the library dependency.

[jeroen]

This is why I am generally a bit reluctant to add this. Also this algorithm is not available on systems with alternative openssl implementations such as libressl. So you can use it locally on Windows and Mac, but if you want to write portable software that may be used by other people, I would recommend to stick with some of the more standard algorithms.

[rmsams]

Sure, I get it. But keccak is a primitive in most blockchain protocols so as far as sha3 implementation are concerned, it's here to stay and will propagate into most openssl implementations over time. Until that day arrives, maybe you can just keep keccak() as a non-exported, undocumented function if you want portability to be assumed for all the package's userland functions.