search

KimiNewt / pyshark

Python wrapper for tshark, allowing python packet parsing using wireshark dissectors
MIT License
2.23k stars 422 forks source link

pyshark fails to parse the tls.handshake_certificates into actual certificate data.（pyshark无法解析tls.handshake_certificates为实际证书数据） #699

Open zhaoyangaiwushu opened 3 months ago

zhaoyangaiwushu commented 3 months ago

我在使用pyshark解析TLS握手消息时，由于存在多个证书 发现tls.handshake_certificates字段总是返回一个描述性字符串('Certificates (1082 bytes)')，而不是实际的证书数据。我期望能够直接从字段获取多个 DER编码的证书链，以便进一步处理和分析。

When using pyshark to parse TLS handshake messages, especially when there are multiple certificates involved, I've noticed that the tls.handshake_certificates field consistently returns a descriptive string ('Certificates (1082 bytes)'), rather than the actual certificate data. My expectation is to be able to directly obtain the DER-encoded certificate chain from this field, to facilitate further processing and analysis. This translation maintains the original meaning and intent of your statement in Chinese, describing the issue you're encountering with pyshark and specifying your desired outcome.

if 'handshake_certificates' in tls_field_names: print(str(pkt.tls)) handshake_certificates = pkt.tls.handshake_certificates print(handshake_certificates) x509af_serialnumber = pkt.tls.x509af_serialnumber print(x509af_serialnumber) handshake_certificate = pkt.tls.handshake_certificate print(handshake_certificate)

print Layer TLS : TLCP Record Layer: Handshake Protocol: Server Hello Content Type: Handshake (22) Version: TLCP (0x0101) Length: 74 Handshake Protocol: Server Hello Handshake Type: Server Hello (2) Length: 70 Version: TLCP (0x0101) Random: 662f0819bc56fd52d5068212b5829d53810ce053ec08ade20b0a2bcbd9cddfeb GMT Unix Time: Apr 29, 2024 10:38:17.000000000 中国标准时间 Random Bytes: bc56fd52d5068212b5829d53810ce053ec08ade20b0a2bcbd9cddfeb Session ID Length: 32 Session ID: 57e73d25feabe79444f7549d3751e9e1ecaa54beccbd1d1b9cc0551c7de34565 Cipher Suite: ECC_SM4_CBC_SM3 (0xe013) Compression Method: null (0) JA3S Fullstring: 257,57363, JA3S: 29679b312f26ae62539fc44a9673b5ca Certificates Length: 840 Certificates (840 bytes) Certificate Length: 417 Certificate: 3082019d30820143a003020102020134300a06082a811ccf55018375304f310b30090603… (id-at-commonName=GateWay,id-at-organizationalUnitName=Venus VPN,id-at-organizationName=Venus,id-at-countryName=CN) signedCertificate version: v3 (2) serialNumber: 0x34 signature (iso.2.156.10197.1.501) Algorithm Id: 1.2.156.10197.1.501 (iso.2.156.10197.1.501) issuer: rdnSequence (0) rdnSequence: 4 items (id-at-commonName=SMxCA for Venus VPN,id-at-organizationalUnitName=Venus VPN,id-at-organizationName=Venus,id-at-countryName=CN) RDNSequence item: 1 item (id-at-countryName=CN) RelativeDistinguishedName item (id-at-countryName=CN) Object Id: 2.5.4.6 (id-at-countryName) CountryName: CN DirectoryString: uTF8String (4) uTF8String: Venus validity notBefore: utcTime (0) utcTime: 2017-06-15 02:46:55 (UTC) notAfter: generalizedTime (1) generalizedTime: Dec 31, 2099 10:46:55.000000000 中国标准时间 subject: rdnSequence (0) rdnSequence: 4 items (id-at-commonName=GateWay,id-at-organizationalUnitName=Venus VPN,id-at-organizationName=Venus,id-at-countryName=CN) subjectPublicKeyInfo algorithm (id-ecPublicKey) ECParameters: namedCurve (1) namedCurve: 1.2.156.10197.1.301 (iso.2.156.10197.1.301) Padding: 0 subjectPublicKey: 04e21b585319df3a09b966310de58c0b4720c573f9b7737baf8b2affab9b9ff5e9ca7627… extensions: 2 items Extension (id-ce-basicConstraints) Extension Id: 2.5.29.19 (id-ce-basicConstraints) BasicConstraintsSyntax [0 length] KeyUsage: c0 1... .... = digitalSignature: True .1.. .... = contentCommitment: True ..0. .... = keyEncipherment: False ...0 .... = dataEncipherment: False .... 0... = keyAgreement: False .... .0.. = keyCertSign: False .... ..0. = cRLSign: False .... ...0 = encipherOnly: False 0... .... = decipherOnly: False algorithmIdentifier (iso.2.156.10197.1.501) encrypted: 30450220376dead8240c99c1da4b41435171dad16625b518813d52e0ea7a2ed11b192458… TLCP Record Layer: Handshake Protocol: Certificate TLCP Record Layer: Handshake Protocol: Server Key Exchange TLCP Record Layer: Handshake Protocol: Server Hello Done Content Type: Handshake (22) Content Type: Handshake (22) Content Type: Handshake (22) Version: TLCP (0x0101) Version: TLCP (0x0101) Version: TLCP (0x0101) Length: 847 Length: 77 Length: 4 Handshake Protocol: Certificate Handshake Protocol: Server Key Exchange Handshake Protocol: Server Hello Done Handshake Type: Certificate (11) Handshake Type: Server Key Exchange (12) Handshake Type: Server Hello Done (14) Length: 843 Length: 73 Length: 0 Certificate Length: 417 Certificate: 3082019d30820143a003020102020133300a06082a811ccf55018375304f310b30090603… (id-at-commonName=GateWay,id-at-organizationalUnitName=Venus VPN,id-at-organizationName=Venus,id-at-countryName=CN) signedCertificate version: v3 (2) serialNumber: 0x33 signature (iso.2.156.10197.1.501) Algorithm Id: 1.2.840.10045.2.1 (id-ecPublicKey) Algorithm Id: 1.2.156.10197.1.501 (iso.2.156.10197.1.501) Algorithm Id: 1.2.156.10197.1.501 (iso.2.156.10197.1.501) Algorithm Id: 1.2.840.10045.2.1 (id-ecPublicKey) Algorithm Id: 1.2.156.10197.1.501 (iso.2.156.10197.1.501) issuer: rdnSequence (0) rdnSequence: 4 items (id-at-commonName=SMxCA for Venus VPN,id-at-organizationalUnitName=Venus VPN,id-at-organizationName=Venus,id-at-countryName=CN) RDNSequence item: 1 item (id-at-organizationName=Venus) RDNSequence item: 1 item (id-at-organizationalUnitName=Venus VPN) RDNSequence item: 1 item (id-at-commonName=SMxCA for Venus VPN) RDNSequence item: 1 item (id-at-countryName=CN) RDNSequence item: 1 item (id-at-organizationName=Venus) RDNSequence item: 1 item (id-at-organizationalUnitName=Venus VPN) RDNSequence item: 1 item (id-at-commonName=GateWay) RDNSequence item: 1 item (id-at-countryName=CN) RDNSequence item: 1 item (id-at-organizationName=Venus) RDNSequence item: 1 item (id-at-organizationalUnitName=Venus VPN) RDNSequence item: 1 item (id-at-commonName=SMxCA for Venus VPN) RDNSequence item: 1 item (id-at-countryName=CN) RDNSequence item: 1 item (id-at-organizationName=Venus) RDNSequence item: 1 item (id-at-organizationalUnitName=Venus VPN) RDNSequence item: 1 item (id-at-commonName=GateWay) RelativeDistinguishedName item (id-at-organizationName=Venus) RelativeDistinguishedName item (id-at-organizationalUnitName=Venus VPN) RelativeDistinguishedName item (id-at-commonName=SMxCA for Venus VPN) RelativeDistinguishedName item (id-at-countryName=CN) RelativeDistinguishedName item (id-at-organizationName=Venus) RelativeDistinguishedName item (id-at-organizationalUnitName=Venus VPN) RelativeDistinguishedName item (id-at-commonName=GateWay) RelativeDistinguishedName item (id-at-countryName=CN) RelativeDistinguishedName item (id-at-organizationName=Venus) RelativeDistinguishedName item (id-at-organizationalUnitName=Venus VPN) RelativeDistinguishedName item (id-at-commonName=SMxCA for Venus VPN) RelativeDistinguishedName item (id-at-countryName=CN) RelativeDistinguishedName item (id-at-organizationName=Venus) RelativeDistinguishedName item (id-at-organizationalUnitName=Venus VPN) RelativeDistinguishedName item (id-at-commonName=GateWay) Object Id: 2.5.4.10 (id-at-organizationName) Object Id: 2.5.4.11 (id-at-organizationalUnitName) Object Id: 2.5.4.3 (id-at-commonName) Object Id: 2.5.4.6 (id-at-countryName) Object Id: 2.5.4.10 (id-at-organizationName) Object Id: 2.5.4.11 (id-at-organizationalUnitName) Object Id: 2.5.4.3 (id-at-commonName) Object Id: 2.5.4.6 (id-at-countryName) Object Id: 2.5.4.10 (id-at-organizationName) Object Id: 2.5.4.11 (id-at-organizationalUnitName) Object Id: 2.5.4.3 (id-at-commonName) Object Id: 2.5.4.6 (id-at-countryName) Object Id: 2.5.4.10 (id-at-organizationName) Object Id: 2.5.4.11 (id-at-organizationalUnitName) Object Id: 2.5.4.3 (id-at-commonName) CountryName: CN CountryName: CN CountryName: CN DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) DirectoryString: uTF8String (4) uTF8String: Venus VPN uTF8String: SMxCA for Venus VPN uTF8String: Venus uTF8String: Venus VPN uTF8String: GateWay uTF8String: Venus uTF8String: Venus VPN uTF8String: SMxCA for Venus VPN uTF8String: Venus uTF8String: Venus VPN uTF8String: GateWay validity notBefore: utcTime (0) utcTime: 2017-06-15 02:46:55 (UTC) notAfter: generalizedTime (1) generalizedTime: Dec 31, 2099 10:46:55.000000000 中国标准时间 subject: rdnSequence (0) rdnSequence: 4 items (id-at-commonName=GateWay,id-at-organizationalUnitName=Venus VPN,id-at-organizationName=Venus,id-at-countryName=CN) subjectPublicKeyInfo algorithm (id-ecPublicKey) ECParameters: namedCurve (1) namedCurve: 1.2.156.10197.1.301 (iso.2.156.10197.1.301) Padding: 6 Padding: 0 Padding: 0 Padding: 3 Padding: 0 subjectPublicKey: 04e21b585319df3a09b966310de58c0b4720c573f9b7737baf8b2affab9b9ff5e9ca7627… extensions: 2 items Extension (id-ce-keyUsage) Extension (id-ce-basicConstraints) Extension (id-ce-keyUsage) Extension Id: 2.5.29.15 (id-ce-keyUsage) Extension Id: 2.5.29.19 (id-ce-basicConstraints) Extension Id: 2.5.29.15 (id-ce-keyUsage) BasicConstraintsSyntax [0 length] KeyUsage: 38 0... .... = digitalSignature: False .0.. .... = contentCommitment: False ..1. .... = keyEncipherment: True ...1 .... = dataEncipherment: True .... 1... = keyAgreement: True .... .0.. = keyCertSign: False .... ..0. = cRLSign: False .... ...0 = encipherOnly: False 0... .... = decipherOnly: False algorithmIdentifier (iso.2.156.10197.1.501) encrypted: 304502201b18de102c857e5e823edfb68140c03dd59c824670b16483d3291b59bb9885cf…

johnbumgarner commented 2 months ago

Here is some usage documentation that I developed and actively maintain for PyShark.

I'm not sure what specific data you're seeking from the tls layer. You mentioned that you want the "actual certificate data" from tls.handshake

First here is an article on Dissecting TLS Using Wireshark. It discusses in detail the handshake process.

Here is my code, where I extracted the data related to tls.handshake.

import pyshark

network_interface = 'en0'
capture = pyshark.LiveCapture(interface=network_interface,
                              display_filter='tls')

try:
    for packet in capture:
        if hasattr(packet, 'tls'):
            # obtain all the field names within the TLS packets
            field_names = packet.tls._all_fields

            # obtain all the field values
            field_values = packet.tls._all_fields.values()

            # enumerate the field names and field values
            for field_name, field_value in zip(field_names, field_values):
                if 'tls.handshake' in field_name:
                    print(f'{field_name}:  {field_value}')
except AttributeError as error:
    print('here')
    pass

Below is part of the extraction related to a certificate. As you see tls.handshake.certificate contains data, which can be used to follow a session. In my short capture I'm obtained 4 unique tls.handshake.certificates.

Is this what you were looking for or was it something else? 

tls.handshake:  Handshake Protocol: Certificate
tls.handshake.type:  11
tls.handshake.length:  2146
tls.handshake.certificates_length:  2143
tls.handshake.certificates:  Certificates (2143 bytes)
tls.handshake.certificate_length:  1173
tls.handshake.certificate:  30:82:04:91:30:82:04:37:a0:03:02:01:02:02:10:0d:8e:45:e3:b9:f5:16:04:06:36:b2:53:f6:59:89:c5:30:0a:06:08:2a:86:48:ce:3d:04:03:02:30:5e:31:0b:30:09:06:03:55:04:06:13:02:55:53:31:15:30:13:06:03:55:04:0a:13:0c:44:69:67:69:43:65:72:74:20:49:6e:63:31:19:30:17:06:03:55:04:0b:13:10:77:77:77:2e:64:69:67:69:63:65:72:74:2e:63:6f:6d:31:1d:30:1b:06:03:55:04:03:13:14:47:65:6f:54:72:75:73:74:20:45:43:43:20:43:41:20:32:30:31:38:30:1e:17:0d:32:34:30:32:31:34:30:30:30:30:30:30:5a:17:0d:32:35:30:33:31:36:32:33:35:39:35:39:5a:30:16:31:14:30:12:06:03:55:04:03:0c:0b:2a:2e:61:64:6e:78:73:2e:63:6f:6d:30:59:30:13:06:07:2a:86:48:ce:3d:02:01:06:08:2a:86:48:ce:3d:03:01:07:03:42:00:04:5f:76:f9:9d:21:5b:ba:db:33:a6:85:58:e2:a5:0d:9a:82:f1:b5:7c:7a:8f:af:27:5e:99:d2:39:d8:ac:5c:6e:90:c1:56:f3:18:6b:9b:b5:3d:61:0c:50:e4:fe:ba:9a:1d:50:85:01:12:3d:d4:4c:33:79:fe:eb:0c:6a:42:0a:a3:82:03:1d:30:82:03:19:30:1f:06:03:55:1d:23:04:18:30:16:80:14:ee:9a:2e:46:f0:c2:da:3c:5c:c7:8c:d6:a4:75:98:de:a8:19:0a:65:30:1d:06:03:55:1d:0e:04:16:04:14:7b:b0:0c:81:44:af:fa:5b:09:39:f9:26:f2:01:43:67:24:46:c9:27:30:21:06:03:55:1d:11:04:1a:30:18:82:0b:2a:2e:61:64:6e:78:73:2e:63:6f:6d:82:09:61:64:6e:78:73:2e:63:6f:6d:30:3e:06:03:55:1d:20:04:37:30:35:30:33:06:06:67:81:0c:01:02:01:30:29:30:27:06:08:2b:06:01:05:05:07:02:01:16:1b:68:74:74:70:3a:2f:2f:77:77:77:2e:64:69:67:69:63:65:72:74:2e:63:6f:6d:2f:43:50:53:30:0e:06:03:55:1d:0f:01:01:ff:04:04:03:02:03:88:30:1d:06:03:55:1d:25:04:16:30:14:06:08:2b:06:01:05:05:07:03:01:06:08:2b:06:01:05:05:07:03:02:30:3e:06:03:55:1d:1f:04:37:30:35:30:33:a0:31:a0:2f:86:2d:68:74:74:70:3a:2f:2f:63:64:70:2e:67:65:6f:74:72:75:73:74:2e:63:6f:6d:2f:47:65:6f:54:72:75:73:74:45:43:43:43:41:32:30:31:38:2e:63:72:6c:30:75:06:08:2b:06:01:05:05:07:01:01:04:69:30:67:30:26:06:08:2b:06:01:05:05:07:30:01:86:1a:68:74:74:70:3a:2f:2f:73:74:61:74:75:73:2e:67:65:6f:74:72:75:73:74:2e:63:6f:6d:30:3d:06:08:2b:06:01:05:05:07:30:02:86:31:68:74:74:70:3a:2f:2f:63:61:63:65:72:74:73:2e:67:65:6f:74:72:75:73:74:2e:63:6f:6d:2f:47:65:6f:54:72:75:73:74:45:43:43:43:41:32:30:31:38:2e:63:72:74:30:0c:06:03:55:1d:13:01:01:ff:04:02:30:00:30:82:01:7e:06:0a:2b:06:01:04:01:d6:79:02:04:02:04:82:01:6e:04:82:01:6a:01:68:00:77:00:4e:75:a3:27:5c:9a:10:c3:38:5b:6c:d4:df:3f:52:eb:1d:f0:e0:8e:1b:8d:69:c0:b1:fa:64:b1:62:9a:39:df:00:00:01:8d:a9:d5:08:52:00:00:04:03:00:48:30:46:02:21:00:c9:e3:de:3e:77:28:4f:e5:5c:a2:95:ba:e6:cb:95:40:88:2f:6d:cd:c5:7e:22:cb:d2:57:ec:7a:6d:c9:41:fa:02:21:00:ae:ae:b6:38:fd:b6:2c:fa:14:0f:0e:60:1b:a3:ee:30:a0:47:46:02:4f:88:5f:fa:4f:de:5b:34:f8:9f:4d:19:00:76:00:7d:59:1e:12:e1:78:2a:7b:1c:61:67:7c:5e:fd:f8:d0:87:5c:14:a0:4e:95:9e:b9:03:2f:d9:0e:8c:2e:79:b8:00:00:01:8d:a9:d5:07:f0:00:00:04:03:00:47:30:45:02:21:00:dc:7b:6c:1c:cc:37:40:4e:3d:6c:0c:d5:5b:a0:6b:83:49:6e:f7:0a:c0:db:b9:8b:2d:ec:d0:de:fc:ec:db:41:02:20:51:ca:d1:fb:c1:bb:74:5b:f4:bd:6e:cd:85:3d:8b:05:ac:66:56:41:36:e5:44:3a:74:5b:4b:c4:9d:ef:b7:c6:00:75:00:e6:d2:31:63:40:77:8c:c1:10:41:06:d7:71:b9:ce:c1:d2:40:f6:96:84:86:fb:ba:87:32:1d:fd:1e:37:8e:50:00:00:01:8d:a9:d5:08:09:00:00:04:03:00:46:30:44:02:20:46:73:ae:66:24:6c:86:49:cb:13:9a:ec:b9:87:5e:f4:c3:e0:ff:73:71:b4:bc:77:06:b2:0a:5f:42:3f:82:de:02:20:65:bf:90:06:fc:8e:94:d2:5b:9c:0b:86:e8:e3:03:04:36:fb:a3:b7:c9:86:e3:0b:66:88:85:a0:17:c0:1a:b8:30:0a:06:08:2a:86:48:ce:3d:04:03:02:03:48:00:30:45:02:20:16:a5:cf:5a:1c:3d:6b:dc:6c:95:6a:74:9c:7d:9e:e4:bb:50:36:62:8f:d6:a8:22:23:d2:bf:0b:93:27:f9:f1:02:21:00:bf:27:21:97:37:80:e8:f2:c6:27:7b:b1:f1:b3:e6:de:c9:6a:c6:cd:53:39:a5:a2:10:83:e8:0b:78:d0:72:6c