Inaccuracy in the step #12 description

[dmhts]

Hi, and thanks for this titanic work! It seems there is some inaccuracy in the description of the step #12:

Encode a message containing 32 null bytes with the shared secret using HMAC SHA256. Take this value and extend it to 80 bytes using HKDF. Call this value sharedSecretExpanded. This is done with HKDF(HmacSha256("\0"*32, sharedSecret), 80).

According to the code 32 null bytes here is used as a key and not as a message. I'm implementing the described algorithm for iOS and still cannot pass the HMAC validation, just trying to find where I could be wrong.

[sigalor]

Using 32 null bytes as a key is correct here. I don't know why the WhatsApp developers chose to do it like this, but apparently they did. Regarding your problem, I also tried to reimplement the encryption stuff, but in JavaScript (using SJCL) You can test it by running node index_jsdemo.js in the root directory and then just opening client/login-via-js-demo.html as a regular file:// in a web browser. In there, I can successfully get the encoding and mac keys. The line you are stuck with semantically is this one. To solve it, please make sure that the HKDF and HmacSha256 algorithm used by iOS is exactly the same as the one used here and in the Python script (it's implementation could differ from the standard and you may need to pass additional parameters to enforce the desired behavior). You could test this by hashing simple strings first and test the equality of the results.

So, as I said, getting the encoding and mac key is no problem, at least in this script. What I am stuck with is that the HMAC validation of actual messages (this line) keeps failing, although algorithmically nothing is different from the Python script. But first, let's solve your current problem.

[dmhts]

Thanks for the quick response!

Using 32 null bytes as a key is correct here.

By the inaccuracy, I rather meant that the phrase:

Encode a message containing 32 null bytes with the shared secret using HMAC SHA256.

is not equal to: HmacSha256("\0"*32, sharedSecret), where "\0"*32 is a key, and sharedSecret is a message. So the first time I made a mistake just reading the phrase without looking at the code first.

To solve it, please make sure that the HKDF and HmacSha256 algorithm used by iOS is exactly the same as the one used here and in the Python script

Thanks for the idea, indeed, I will try to compare the outputs of the functions between the platforms, because everything else I have checked twice. I will get back soon.

[sigalor]

Ah, thanks so much for pointing that out. I guess I read the entire instructions so many times that I didn't even notice this; just changed the readme. I am looking forward to hear your results! :)

[dmhts]

So the output of the HKDF algorithm implemented on iOS (by CryptoSwift) is different, and according to RFC 5869 it is a correct one (it passes all the test cases at the end of the document).

I'm not a deep expert in cryptography but I have compared the implementation with a python version and found the difference. The iOS version implements both stages of the HKDF algorithm - extract-then-expand (with a pseudo-random key), the Python version implements only the second stage - expand i.e. it skips the extract stage.

Anyway, the second version passes HMAC validation and it means that WhatsApp engineers also prefer it and it's enough to move further.

Just in case, the Swift version of the "only-expand" algorithm:

func HKDF(key: Array<UInt8>, length: Int) throws -> Array<UInt8> {
        var keyStream = [UInt8](), keyBlock  = [UInt8]()
        var blockIndex: UInt8 = 1;
        while keyStream.count < length {
            keyBlock.append(blockIndex)
            keyBlock = try HMAC(key: key, variant: .sha256).authenticate(keyBlock)
            blockIndex += 1
            keyStream += keyBlock
        }

        return Array(keyStream[0..<length])
}

Many thanks for sending in the right direction.

[sigalor]

Awesome, that's good to know! I already suspected that WhatsApp doesn't fully comply with the standard, but now we have the facts. :) By the way, would you mind adding your Swift code to the repository later? I think it would be nice to have some reference implementations of the entire encryption algorithm in different programming languages (e.g. in a new directory called "reference-implementations").

[dmhts]

Sure, it should be a worthy tribute to your work!