Fixes unknown TLSExtensions messing up the extension list.

[tintinweb]

Added LengthFieldPacket() type Packet which only provides up to .length bytes to the dissector of the next layer (its payload). This fixes dissection of unknown tlsextensions.

The LengthFieldPacket() packet type can be used whenever we want to only provide up to .length bytes to the next layer. I guess we could also use this to dissect the initial TLSRecords and get rid of the PacketNoPadding type layer.

Current dissector:

Note the Raw() layer is eating up all the bytes at we do not have a layer for the (deprecated) NPN extension. Also see the "status_request" extension.

    ###[ SSL/TLS ]###
               \records\
                |###[ TLS Record ]###
                |  content_type= handshake
                |  version= TLS_1_0
                |  length= 0xd4
                |###[ TLS Handshake ]###
                |     type= client_hello
                |     length= 0xd0
                |###[ TLS Client Hello ]###
                |        version= TLS_1_2
                |        gmt_unix_time= 3242904930L
                |        random_bytes= 'x"W\xe6\xfd\x97\xb7\xaf \xda\x12c\x8c\x07 o\xe3\th\xc3\xc1\xe0\xe3C\xe4\x00\xc6\xc7'
                |        session_id_length= 0x0
                |        session_id= ''
                |        cipher_suites_length= 0x28
                |        cipher_suites= [49195, 49199, 158, 52244, 52243, 49162, 49161, 49171, 49172, 49159, 49169, 51, 50, 57, 156, 47, 53, 10, 5, 4]
                |        compression_methods_length= 0x1
                |        compression_methods= [0]
                |        extensions_length= 0x7f
                |        \extensions\
                |         |###[ TLS Extension ]###
                |         |  type= server_name
                |         |  length= 0x17
                |         |###[ TLS Extension Servername Indication ]###
                |         |     length= 0x15
                |         |     \server_names\
                |         |      |###[ TLS Servername ]###
                |         |      |  type= host
                |         |      |  length= 0x12
                |         |      |  data= 'ad.doubleclick.net'
                |         |###[ TLS Extension ]###
                |         |  type= renegotiation_info
                |         |  length= 0x1
                |         |###[ TLS Extension Renegotiation Info ]###
                |         |     length= 0x0
                |         |     data= ''
                |         |###[ TLS Extension ]###
                |         |  type= supported_groups
                |         |  length= 0x8
                |         |###[ TLS Extension Elliptic Curves ]###
                |         |     length= 0x6
                |         |     elliptic_curves= [23, 24, 25]
                |         |###[ TLS Extension ]###
                |         |  type= ec_point_formats
                |         |  length= 0x2
                |         |###[ TLS Extension EC Points Format ]###
                |         |     length= 0x1
                |         |     ec_point_formats= [0]
                |         |###[ TLS Extension ]###
                |         |  type= SessionTicket TLS
                |         |  length= 0x0
                |         |###[ TLS Extension SessionTicket TLS ]###
                |         |     data= ''
                |         |###[ TLS Extension ]###
                |         |  type= next_protocol_negotiation
                |         |  length= 0x0
                |         |###[ Raw ]###
                |         |     load= '\x00\x10\x00\x1a\x00\x18\x08spdy/3.1\x05h2-14\x08http/1.1uP\x00\x00\x00\x05\x00\x05\x01\x00\x00\x00\x00\x00\x12\x00\x00\x00\r\x00\x12\x00\x10\x04\x01\x05\x01\x02\x01\x04\x03\x05\x03\x02\x03\x04\x02\x02\x02'

Fixed dissector:

Note that the NPN extension is now dissected as TLSExtension()/Raw() with the sublayer having the size of TLSExtension.length.

    ###[ SSL/TLS ]###
               \records\
                |###[ TLS Record ]###
                |  content_type= handshake
                |  version= TLS_1_0
                |  length= 0xd4
                |###[ TLS Handshake ]###
                |     type= client_hello
                |     length= 0xd0
                |###[ TLS Client Hello ]###
                |        version= TLS_1_2
                |        gmt_unix_time= 3242904930L
                |        random_bytes= 'x"W\xe6\xfd\x97\xb7\xaf \xda\x12c\x8c\x07 o\xe3\th\xc3\xc1\xe0\xe3C\xe4\x00\xc6\xc7'
                |        session_id_length= 0x0
                |        session_id= ''
                |        cipher_suites_length= 0x28
                |        cipher_suites= [49195, 49199, 158, 52244, 52243, 49162, 49161, 49171, 49172, 49159, 49169, 51, 50, 57, 156, 47, 53, 10, 5, 4]
                |        compression_methods_length= 0x1
                |        compression_methods= [0]
                |        extensions_length= 0x7f
                |        \extensions\
                |         |###[ TLS Extension ]###
                |         |  type= server_name
                |         |  length= 0x17
                |         |###[ TLS Extension Servername Indication ]###
                |         |     length= 0x15
                |         |     \server_names\
                |         |      |###[ TLS Servername ]###
                |         |      |  type= host
                |         |      |  length= 0x12
                |         |      |  data= 'ad.doubleclick.net'
                |         |###[ TLS Extension ]###
                |         |  type= renegotiation_info
                |         |  length= 0x1
                |         |###[ TLS Extension Renegotiation Info ]###
                |         |     length= 0x0
                |         |     data= ''
                |         |###[ TLS Extension ]###
                |         |  type= supported_groups
                |         |  length= 0x8
                |         |###[ TLS Extension Elliptic Curves ]###
                |         |     length= 0x6
                |         |     elliptic_curves= [23, 24, 25]
                |         |###[ TLS Extension ]###
                |         |  type= ec_point_formats
                |         |  length= 0x2
                |         |###[ TLS Extension EC Points Format ]###
                |         |     length= 0x1
                |         |     ec_point_formats= [0]
                |         |###[ TLS Extension ]###
                |         |  type= SessionTicket TLS
                |         |  length= 0x0
                |         |###[ TLS Extension ]###
                |         |  type= next_protocol_negotiation
                |         |  length= 0x0
                |         |###[ TLS Extension ]###
                |         |  type= application_layer_protocol_negotiation
                |         |  length= 0x1a
                |         |###[ TLS Extension Application-Layer Protocol Negotiation ]###
                |         |     length= 0x18
                |         |     \protocol_name_list\
                |         |      |###[ TLS ALPN Protocol ]###
                |         |      |  length= 0x8
                |         |      |  data= 'spdy/3.1'
                |         |      |###[ TLS ALPN Protocol ]###
                |         |      |  length= 0x5
                |         |      |  data= 'h2-14'
                |         |      |###[ TLS ALPN Protocol ]###
                |         |      |  length= 0x8
                |         |      |  data= 'http/1.1'
                |         |###[ TLS Extension ]###
                |         |  type= 0x7550
                |         |  length= 0x0
                |         |###[ TLS Extension ]###
                |         |  type= status_request
                |         |  length= 0x5
                |         |###[ Raw ]###
                |         |     load= '\x01\x00\x00\x00\x00'
                |         |###[ TLS Extension ]###
                |         |  type= signed_certificate_timestamp
                |         |  length= 0x0
                |         |###[ TLS Extension ]###
                |         |  type= signature_algorithms
                |         |  length= 0x12
                |         |###[ TLS Extension Signature And Hash Algorithm ]###
                |         |     length= 0x10
                |         |     \algorithms\
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha256
                |         |      |  signature_algorithm= rsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha384
                |         |      |  signature_algorithm= rsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha1
                |         |      |  signature_algorithm= rsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha256
                |         |      |  signature_algorithm= ecdsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha384
                |         |      |  signature_algorithm= ecdsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha1
                |         |      |  signature_algorithm= ecdsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha256
                |         |      |  signature_algorithm= dsa
                |         |      |###[ TLS Signature Hash Algorithm Pair ]###
                |         |      |  hash_algorithm= sha1
                |         |      |  signature_algorithm= dsa

[alexmgr]

Looks good to me. +1 on removing PacketNoPadding, since this should be an in place replacement for it. Nice clean up!

[buildhive]

tintin » scapy-ssl_tls #154 SUCCESS This pull request looks good (what's this?)

[buildhive]

tintin » scapy-ssl_tls #163 SUCCESS This pull request looks good (what's this?)

[tintinweb]

Ok, it is not that easy :/ We kind of need both Packet types, here's why:

1. PacketNoPayload for leaf-packets which are not expected to have a sub-layer/payload. The Packet will terminate when all fields are set. Further bytes won't be dissected as payload to that packet but will be dissected in the overall bytestream dissection i.e. eventually dissecting another sublayer of another subtree. For example variable lists of TLSCertificates followed by a variable list of TLSProtoALPN.

   class TLSCertificate(PacketNoPayload):
      name = "TLS Certificate"
      fields_desc = [ XBLenField("length", None, length_of="data", fmt="!I", numbytes=3),
                      PacketLenField("data", None, x509.X509Cert, length_from=lambda x:x.length),]
   
   class TLSCertificateList(PacketNoPayload):
      name = "TLS Certificate List"
      fields_desc = [
                     XBLenField("length", None, length_of="certificates", fmt="!I", numbytes=3),
                     PacketListField("certificates", None, TLSCertificate, length_from=lambda x:x.length),
                    ]

   Note that both packets are PacketNoPayled as both the TLSCertificateList and the TLSCertificate are not expected to have a payload. TLSCertificate's are fully contained within TLSCertificateList in the TLSCertificateList.certificates PacketListField.

2. PacketLengthPayload for packets like TLSExtension where the last field is the length (in bytes) of the next payload/sub-layer. This packet type will only provide up to .length bytes to the dissector of the next layer. If no .length is available it'll act like an ordinary Packet()

   class TLSExtension(PacketLengthFieldPayload):
      name = "TLS Extension"
      fields_desc = [XShortEnumField("type", TLSExtensionType.SERVER_NAME, TLS_EXTENSION_TYPES),
                     XLenField("length", None, fmt="!H"),
                    ]
   

   Note that TLSDecryptablePacket is of type PacketLengthPayload in order to work for TLSHandshake.

Also had a look at generic stacking for PacketLengthPayload type packets but it only works for packets in PacketListFields (auto.dissects Padding layers as PacketListField prototype layer) atm. Will draft a layer that allows generic stacking and only returns Padding when the bytestream cannot be dissected.

[buildhive]

tintin » scapy-ssl_tls #165 SUCCESS This pull request looks good (what's this?)

[alexmgr]

Thanks for the detailed explanation. Both use cases make sense.

[buildhive]

tintin » scapy-ssl_tls #172 SUCCESS This pull request looks good (what's this?)