Help regarding decrypting GenericAEADCipher.

[faihan6]

This is a continuation of this issue from StackOverflow. Continuing here as stated in the comments.

I tried a new DTLS session and will attach the parameters below. I am still facing the same error when decrypting.

ClientRandom 14668a92fc430b58092fcce747f499b13b0c56574b54377990c26c628f575b88
ServerRandom 835c25d160012c38fe33c39f8fa4b2c105e258754a7abe0e0ee5e72ec01b42da
serverPrivateKey 1870b8712bf8fb52135b60f6da502e71ef27905b9d02a77d2c609e5b347bb4c5
serverPublicKey 044dbb1a56d472daf4a04dd4ef25a1a13fc44ce2373fc9388d0d2115c030fd43e26c1bfbce8ce10681807e20637fa3b217a1ed9e9cd028a87732ebd04578d95e37
clientPublicKey 04ff8af815092f288a204fc85419d9fcb3422337f6b4868eaed24d6c9179990ec7fd5449804e48b00b3610642e2267299749a6ebc39eaa0ebec729a688a436a4aa
encryptedHandshakeMessage d5fa7c4d573a28d56f956a759f15de8945f4081b19ba2f3eebe15ae564cbde1d170866427cdb9db9a745701625754f62

Calculated values:
preMasterSecret 3259315425730a7e39e97eb733f30c0a5cdb121e09750d78271618d9a5f05b54
masterSecret f3e09249ecd0b258d889e4b792f6b3524f8a7b83158cd0cd075a8c8cf99b2fc5120e6f988e3d045e131e23b4a39b6bb1
Key block: 312e7c5c32e6801720a123a91f539fa54eeeb91406c287c0c4047613a09dc88d9164f6ad964ab3ff
symmetricKeys {
  clientWriteKey: <Buffer 31 2e 7c 5c 32 e6 80 17 20 a1 23 a9 1f 53 9f a5>,
  serverWriteKey: <Buffer 4e ee b9 14 06 c2 87 c0 c4 04 76 13 a0 9d c8 8d>,
  clientWriteIV: <Buffer 91 64 f6 ad>,
  serverWriteIV: <Buffer 96 4a b3 ff>
}

Key: 312e7c5c32e6801720a123a91f539fa5
Implicit Nonce: 9164f6ad
Payload: d5fa7c4d573a28d56f956a759f15de8945f4081b19ba2f3eebe15ae564cbde1d170866427cdb9db9a745701625754f62
   Explicit Nonce: d5fa7c4d573a28d5
   CipherText: 6f956a759f15de8945f4081b19ba2f3eebe15ae564cbde1d170866427cdb9db9a745701625754f62
      Encrypted: 6f956a759f15de8945f4081b19ba2f3eebe15ae564cbde1d
      Tag: 170866427cdb9db9a745701625754f62
Additional data: 000100000000000016fefd0030

Please find the Wireshark dump here

Thanks in advance.

[boaks]

ECDHE master secret F3E09249ECD0B258D889E4B792F6B3524F8A7B83158CD0CD075A8C8CF99B2FC5120E6F988E3D045E131E23B4A39B6BB1
Keys Seed 835C25D160012C38FE33C39F8FA4B2C105E258754A7ABE0E0EE5E72EC01B42DA14668A92FC430B58092FCCE747F499B13B0C56574B54377990C26C628F575B88
Request Bytes 40
Client Write Key @0,312E7C5C32E6801720A123A91F539FA5
Server Write Key @16,4EEEB91406C287C0C4047613A09DC88D
Client Write IV @32,9164F6AD
Server Write IV @36,964AB3FF

OK, the keys are well. I will check decrypting the encrypted handshake message, but it will take some time.

[boaks]

Client Write Key @0,312E7C5C32E6801720A123A91F539FA5
Server Write Key @16,4EEEB91406C287C0C4047613A09DC88D
Client Write IV @32,9164F6AD
Server Write IV @36,964AB3FF
AAD 000100000000000016FEFD0018
NOUNCE 9164F6ADD5FA7C4D573A28D5
payload 1400000C000300000000000C4FF7BF3EBBEFC50BBFE0E982

Your AAD:

000100000000000016fefd0030

You use the record length, but it's the fragment length. In case of decrypt, that's the record.length - record.header.length(13) - record.explicitIV.length(8) - record.mac.length(16) .

For record

16fefd00010000000000000030d5fa7c4d573a28d56f956a759f15de8945f4081b19ba2f3eebe15ae564cbde1d170866427cdb9db9a745701625754f62

61-13-8-16 => 24 = 0x18

[boaks]

RFC 5246 6.2.2 Record Compression and Decompression

The compression algorithm translates a TLSPlaintext structure into a TLSCompressed structure.

Though mainly CompressionMethod.null is used,

struct {
          ContentType type;       /* same as TLSPlaintext.type */
          ProtocolVersion version;/* same as TLSPlaintext.version */
          uint16 length;
          opaque fragment[TLSCompressed.length];
      } TLSCompressed;

will contain the plain-payload (unencrypted), and TLSCompressed.length is the length of fragment (here, before encryption, the same as payload).

RFC 5246 6.2.3.3 AEAD Ciphers

additional_data = seq_num + TLSCompressed.type +
                        TLSCompressed.version + TLSCompressed.length;

The aead_output consists of the ciphertext output by the AEAD
   encryption operation.  The length will generally be larger than
   TLSCompressed.length, 

So, on decrypt, it's required to calculate the length of the plain payload, using the record.length will be to large.

[faihan6]

This is it. Using the length of just encrypted output instead of the whole record length in AAD solves the problem.

Thanks you very much. It means a lot. Will consolidate all the issues and post it as an answer to the original StackOverflow question.

[boaks]

You're welcome. I did that "step-by-step" so many times, for each extension. But it's the only way it works, because small differences will make it impossible to trace the error back, that's what good encryption is.