Allow shake128/256 to produce outputs of unlimited length

[tianxiadys]

What is the problem this feature will solve?

SHAKE128 is essentially a hash algorithm that can output an infinite-length stream. Rather than being just a hash algorithm, it's more like a stream cipher such as RC4. Support for SHAKE128/256 is currently available, but the implementation requires specifying an outputSize in advance, which is fixed and limited. This doesn't meet my needs, so I have to rely on third-party libraries.

What is the feature you are proposing to solve the problem?

Support for SHAKE was first introduced in 2019, but I believe this support is incomplete.

https://github.com/nodejs/node/issues/28757 https://github.com/nodejs/node/pull/28805

What alternatives have you considered?

No response

[tianxiadys]

Allowing the digest method to be called multiple times in certain situations could be an option, such as adding a {streamMode: true} option. This approach seems clunky, but it could solve the problem.

[tianxiadys]

Alternatively, a new method like digestAsStream(length: number) could be added. This method would represent a streaming hash, with the length parameter specifying the desired output length. It could be called an unlimited number of times.

[RedYetiDev]

Hi! Could you provide some example code of what it is that you are looking for?

[tniessen]

I don't think there's much we can do here until EVP_DigestSqueeze() becomes available in the OpenSSL version that we are linking against, which might take a while.

[tianxiadys]

I'm actually writing a Node.js version of the Xray-core library, which is an encryption proxy tool. Living in mainland China, it's an important tool for me. The library is poorly written; it implements a proxy protocol called VMess. In one part, it uses SHAKE128 to generate an infinite byte stream and XORs this stream with plaintext data to transmit over the network. This is definitely a misuse, as it incorrectly applies a hash algorithm as a stream cipher. However, I'm just replicating the existing protocol and need to find a function that supports this usage.

This is a quote from the original code.

https://github.com/XTLS/Xray-core/blob/main/proxy/vmess/encoding/auth.go

type ShakeSizeParser struct {
    shake  sha3.ShakeHash
    buffer [2]byte
}

func NewShakeSizeParser(nonce []byte) *ShakeSizeParser {
    shake := sha3.NewShake128()
    common.Must2(shake.Write(nonce))
    return &ShakeSizeParser{
        shake: shake,
    }
}

func (*ShakeSizeParser) SizeBytes() int32 {
    return 2
}

func (s *ShakeSizeParser) next() uint16 {
    common.Must2(s.shake.Read(s.buffer[:]))
    return binary.BigEndian.Uint16(s.buffer[:])
}

func (s *ShakeSizeParser) Decode(b []byte) (uint16, error) {
    mask := s.next()
    size := binary.BigEndian.Uint16(b)
    return mask ^ size, nil
}

func (s *ShakeSizeParser) Encode(size uint16, b []byte) []byte {
    mask := s.next()
    binary.BigEndian.PutUint16(b, mask^size)
    return b[:2]
}

func (s *ShakeSizeParser) NextPaddingLen() uint16 {
    return s.next() % 64
}

func (s *ShakeSizeParser) MaxPaddingLen() uint16 {
    return 64
}

[tniessen]

This is definitely a misuse, as it incorrectly applies a hash algorithm as a stream cipher.

Side note: it's not an approved usage of an XOF, but it's not known to be vulnerable in Keccak's sponge construction either. It even was suggested in the 2011 paper on sponge function but has not been approved so far. In many ways, SHAKE is not a hash function but rather an XOF.