TLS in Scapy should have more doc

[BlobbyBob]

Brief description

When connecting to a TLS server using TLS1.3 that negotiates TLS_CHACHA20_POLY1305, using tlsSessions fails with ValueError: not enough values to unpack (expected 3, got 2)

Scapy version

2.5.0

Python version

3.12.3

Operating system

Linux Kernel 6.8.8

Additional environment information

No response

How to reproduce

Send a client record such as

client_hello = TLSClientHello(
    ciphers=[
        tls.TLS_CHACHA20_POLY1305_SHA256,
    ],
    ext=[
        tls.TLS_Ext_ServerName(servernames=[tls.ServerName(servername=b"localhost")]),
        tls.TLS_Ext_ExtendedMasterSecret(),
        tls.TLS_Ext_EncryptThenMAC(),
        tls.TLS_Ext_SupportedGroups(groups=["x25519", "secp256r1", "x448"]),
        tls.TLS_Ext_SupportedVersion_CH(versions=["TLS 1.3"]),
        tls.TLS_Ext_SignatureAlgorithms(sig_algs=["ed25519", "sha256+rsaepss", "sha256+ecdsa"]),
        tls.TLS_Ext_KeyShare_CH(client_shares=[tls.KeyShareEntry(group="x25519")]),
    ],
)
ch_record = TLS(type=22, version=0x0303, len=len(client_hello.build())) / client_hello

to a server like openssl s_server and try to decrypt the response through the tlsSession.

Actual result

No response

Expected result

No response

Related resources

The error occurs at

File .venv/lib/python3.12/site-packages/scapy/layers/tls/record.py:537, in TLS.pre_dissect(self, s)
    535         cfrag, mac = self._tls_auth_decrypt(hdr, efrag)
    536     else:
--> 537         iv, cfrag, mac = self._tls_auth_decrypt(hdr, efrag)
    538     decryption_success = True       # see XXX above
    540 frag = self._tls_decompress(cfrag)

ValueError: not enough values to unpack (expected 3, got 2)

Quick debugging showed that

type(tls_session.rcs.cipher) = scapy.layers.tls.crypto.cipher_aead.Cipher_CHACHA20_POLY1305_TLS13
isinstance(tls_session.rcs.cipher, tls.Cipher_CHACHA20_POLY1305) = False

as Cipher_CHACHA20_POLY1305 is a subclass of Cipher_CHACHA20_POLY1305_TLS13 and not reverse.

Thus, the check in scapy/layers/tls/record.py does not cover this case.

[guedou]

Thanks for your report.Can your share a complete example to reproduce the issue and ideally a target ?Sent from my iPhoneOn 3 May 2024, at 12:46, BlobbyBob @.***> wrote: Brief description When connecting to a TLS server using TLS1.3 that negotiates TLS_CHACHA20_POLY1305, using tlsSessions fails with ValueError: not enough values to unpack (expected 3, got 2) Scapy version 2.5.0 Python version 3.12.3 Operating system Linux Kernel 6.8.8 Additional environment information No response How to reproduce Send a client record such as client_hello = TLSClientHello( ciphers=[ tls.TLS_CHACHA20_POLY1305_SHA256, ], ext=[ tls.TLS_Ext_ServerName(servernames=[tls.ServerName(servername=b"localhost")]), tls.TLS_Ext_ExtendedMasterSecret(), tls.TLS_Ext_EncryptThenMAC(), tls.TLS_Ext_SupportedGroups(groups=["x25519", "secp256r1", "x448"]), tls.TLS_Ext_SupportedVersion_CH(versions=["TLS 1.3"]), tls.TLS_Ext_SignatureAlgorithms(sig_algs=["ed25519", "sha256+rsaepss", "sha256+ecdsa"]), tls.TLS_Ext_KeyShare_CH(client_shares=[tls.KeyShareEntry(group="x25519")]), ], ) ch_record = TLS(type=22, version=0x0303, len=len(client_hello.build())) / client_hello

to a server like openssl s_server and try to decrypt the response through the tlsSession. Actual result No response Expected result No response Related resources The error occurs at File .venv/lib/python3.12/site-packages/scapy/layers/tls/record.py:537, in TLS.pre_dissect(self, s) 535 cfrag, mac = self._tls_auth_decrypt(hdr, efrag) 536 else: --> 537 iv, cfrag, mac = self._tls_auth_decrypt(hdr, efrag) 538 decryption_success = True # see XXX above 540 frag = self._tls_decompress(cfrag)

ValueError: not enough values to unpack (expected 3, got 2)

Quick debugging showed that type(tls_session.rcs.cipher) = scapy.layers.tls.crypto.cipher_aead.Cipher_CHACHA20_POLY1305_TLS13 isinstance(tls_session.rcs.cipher, tls.Cipher_CHACHA20_POLY1305) = False

as Cipher_CHACHA20_POLY1305 is a subclass of Cipher_CHACHA20_POLY1305_TLS13 and not reverse. Thus, the check in scapy/layers/tls/record.py does not cover this case.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

[BlobbyBob]

Of course, here you go:

Script test.py:

import socket

import scapy.layers.tls.all as tls

client_hello = tls.TLSClientHello(
    ciphers=[
        tls.TLS_CHACHA20_POLY1305_SHA256
    ],
    ext=[
        tls.TLS_Ext_ServerName(servernames=[tls.ServerName(servername=b"localhost")]),
        tls.TLS_Ext_ExtendedMasterSecret(),
        tls.TLS_Ext_EncryptThenMAC(),
        tls.TLS_Ext_SupportedGroups(groups=["x25519"]),
        tls.TLS_Ext_SupportedVersion_CH(versions=["TLS 1.3"]),
        tls.TLS_Ext_SignatureAlgorithms(sig_algs=["sha256+rsaepss"]),
        tls.TLS_Ext_KeyShare_CH(client_shares=[tls.KeyShareEntry(group="x25519")]),
    ],
)
chrecord = tls.TLS(type=22, version=0x0303, len=len(client_hello.build())) / client_hello
chrecord.tls_session.tls13_client_privshares["x25519"] = client_hello[tls.KeyShareEntry].privkey
payload = chrecord.build()

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    sock.connect(("localhost", 4433))
    sock.sendall(payload)
    resp_bytes = sock.recv(2**12)

    # Iterate through records
    session = chrecord.tls_session.mirror()
    i = 0
    while i < len(resp_bytes):
        server_record = tls.TLS(resp_bytes[i:], tls_session=session)
        i += server_record.len + 5
        server_record.show()
finally:
    sock.close()

Openssl Server (version 3.3.0):

openssl s_server -cert localhost.crt -key localhost.key  # with some self-signed cert + associated privkey

Scapy Commands:

$ docker run --rm -it --net=host python bash
# pip install scapy cryptography\<42
# python test.py

[BlobbyBob]

I also just discovered that the issue persists when announcing TLS_AES_128_GCM_SHA256 as only cipher suite. Initially, I thought it was only a issue for Chacha due to the mentioned branching instruction in record.py, but now I suspect the underlying issue might be a bit different.

[evverx]

ValueError: not enough values to unpack (expected 3, got 2)

Something like that was reported and fixed in https://github.com/secdev/scapy/pull/4082. If it isn't reproducible with the master branch or https://github.com/secdev/scapy/releases/tag/v2.6.0rc1 it was probably the same issue.

Edit: Looks like it's reproducible with the master branch.

[gpotter2]

There are two issues with your code:

1. You need to build the TLS packet using the msg field (and not appending the client hello). Doing that, you also don't need the 'hack' where you're injecting tls13_client_privshares:

   chrecord = tls.TLS(type=22, version=0x0303, len=len(client_hello.build()), msg=[client_hello])
   # instead of
   # chrecord = tls.TLS(type=22, version=0x0303, len=len(client_hello.build())) / client_hello

2. TLS() is greedy, and will automatically dissect all TLS() payloads following the current one. So you don't need the loop at the end of your code.

All in all, this gives the following, which works:

import socket

import scapy.layers.tls.all as tls

client_hello = tls.TLSClientHello(
    ciphers=[
        tls.TLS_CHACHA20_POLY1305_SHA256,
    ],
    ext=[
        tls.TLS_Ext_ServerName(servernames=[tls.ServerName(servername=b"localhost")]),
        tls.TLS_Ext_ExtendedMasterSecret(),
        tls.TLS_Ext_EncryptThenMAC(),
        tls.TLS_Ext_SupportedGroups(groups=["x25519"]),
        tls.TLS_Ext_SupportedVersion_CH(versions=["TLS 1.3"]),
        tls.TLS_Ext_SignatureAlgorithms(sig_algs=["sha256+rsaepss"]),
        tls.TLS_Ext_KeyShare_CH(client_shares=[tls.KeyShareEntry(group="x25519")]),
    ],
)
chrecord = tls.TLS(type=22, version=0x0303, len=len(client_hello.build()), msg=[client_hello])
# chrecord.tls_session.tls13_client_privshares["x25519"] = client_hello[tls.KeyShareEntry].privkey
payload = chrecord.build()

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    sock.connect(("localhost", 4433))
    sock.sendall(payload)
    resp_bytes = sock.recv(2**12)

    # Iterate through records
    session = chrecord.tls_session.mirror()
    server_record = tls.TLS(resp_bytes, tls_session=session)
    server_record.show()
finally:
    sock.close()

[gpotter2]

That being said.

We might want to have a more graceful failure for this case. The error

ValueError: not enough values to unpack (expected 3, got 2)

generally happens when Scapy tries to dissect TLS 1.3 as TLS 1.2, or the opposite. This usually means that the session is invalid (in this case: the session is invalid because msg=[] wasn't used, but our computing of the session happens when building the field that handles msg).

It might also be a good idea to trigger a warning when someone tries to append a TLS(13)ClientHello field to TLS (instead of using msg). Without looking at some examples, that's pretty hard to guess.

[gpotter2]

That being also said,

We really need to have examples on https://scapy.readthedocs.io regarding various common TLS usages :(

[BlobbyBob]

Thanks for your clarifications. I was aware of the second aspect but not of the first one.

We really need to have examples on https://scapy.readthedocs.io/ regarding various common TLS usages :(

Yeah, I can definitely stress that. It is pretty tough to find usage information and I often ended up reading the source files directly to find out what data types belong to what fields and how the sessions really work.
The only example-style hint is included in the docs of scapy.layers.tls.session.tlsSession.mirror(), but fully reverse engineering the intended usage from that is rather difficult and apparently error-prone.