search

moooofly / MarkSomethingDownLLS

本仓库用于记录自 2017年10月16日起，在英语流利说任职期间的各方面知识汇总（以下内容不足以体现全部，一些敏感内容已移除）～
MIT License
72 stars 37 forks source link

Wireshark 解析 HTTPS #49

Open moooofly opened 5 years ago

moooofly commented 5 years ago

todo:

moooofly commented 5 years ago

HTTP/HTTPS Analysis Using Wireshark

小结

基于 private key 进行 HTTPS 解码是一种比较“老”的方式，只能针对使用 SSL 加密的 HTTPS ，如下图所示

image

SSLv3 是 SSL 的最高版本

抓包示例文件：

与 HTTPS 解析有关的内容

Then if we click on any application data that data is unreadable to us it’s all gibberish but with wireshark we can decrypt that data only thing we need is the Private Key of the server.

Once again go to Preference → Protocol → SSL

Add these value

实验步骤：

image

image

image

点击确定后，HTTP 报文就能够解析出来了

image

moooofly commented 5 years ago

Wireshark/SSL

Secure Socket Layer (SSL)

Transport Layer Security (TLS) provides security in the communication between two hosts. It provides integrity, authentication and confidentiality. It is used most commonly in web browsers, but can be used with any protocol that uses TCP as the transport layer.

TLS 为安全通信提供了如下特性：

SSL is the former version of the TLS protocol. These names are often used interchangeably which can lead to some confusion:

  • A configuration that uses the SSL protocol (SSLv2/SSLv3) is insecure. The TLS protocol should be used instead.
  • X.509 certificates for authentication are sometimes also called SSL Certificates.
  • Some applications (such as email) use a single port for both unencrypted and encrypted sessions. To change from unencrypted to encrypted, (START)TLS is used. When a single port directly uses the TLS protocol, it is often referred to as SSL.
  • For historical reasons, software (Wireshark included) refer to SSL or SSL/TLS while it actually means the TLS protocol since that is nowadays what everyone uses.

SSL 是 TLS 的老版本； SSLv2 和 SSLv3 已经被认为是不安全的协议了，应该使用 TLS 进行替代； X.509 证书有时也被称作 SSL 证书； 有一些应用会使用单独一个 port 同时用于未加密和加密会话； 由于历史原因，一些软件会使用 SSL 或 SSL/TLS 来指代 TLS ；

SSL dissection in Wireshark

The SSL/TLS dissector is fully functional and even supports advanced features such as decryption of TLS if appropriate secrets are provided. TLS decryption requires Wireshark to be built with Libgcrypt (mandatory since Wireshark 2.4). Support for RSA private key files requires GnuTLS. The official Wireshark binaries have both dependencies.

解密 TLS 要求 Wireshark 使用了 Libgcrypt ； 解密 SSL 要求 Wireshark 使用了 GnuTLS ；

Preference Settings

Go to Wireshark -> Preferences. Open the Protocols tree and select SSL. Alternatively, select a SSL/TLS packet in the packet list, right-click on the SSL layer in the packet details view and open the Protocol preferences menu.

image

RSA keys list

The RSA keys list button opens a new dialog with the following fields:

Item Meaning
IP address The IP address of the SSL server in IPv4 or IPv6 format, or the following special values: any, anyipv4, anyipv6, 0.0.0.0 (this field is ignored since Wireshark 2.0)
Port The port number, the special value start_tls or 0. Ignores since Wireshark 2.0.
Protocol A protocol name for the decrypted network data. Popular choices are http or data. If you enter an invalid protocol name an error message will show you the valid values.
Key File path to the RSA private key.
Password Leave it empty for PEM-encoded private key files or provide it for encrypted PKCS#12 key files.

The RSA key file can either be a PEM format private key or a PKCS#12 keystore. If the file is a PKCS#12 keystore (typically a file with a .pfx or .p12 extension), the password for the keystore must be specified in the Password field.

RSA key 文件可以有两种形式：

Starting with Wireshark 2.0, the RSA key file is automatically matched against the public key as found in the Certificate handshake message. Before Wireshark 2.0, it relied on the user to enter a valid Address and Port value. Note that only RSA key exchanges can be decrypted using this RSA private key, Diffie-Hellman key exchanges cannot be decrypted using a RSA key file! (See "SSLKEYLOGFILE" if you have such a capture.)

Key File format conversion

略

Example capture file

dump.pcapng TLSv1.2 capture with 73 cipher suites, you need this premaster.txt file for decrypting the traffic.

dump.pcapng 就是一个基于 TLSv1.2 加密的 HTTPS 通信抓包，在设置好 premaster.txt 文件后，可以看到如下图所示内容

image

image

Using the (Pre)-Master-Secret

Decoding an SSL connection requires either knowledge of the (asymmetric) secret server key and a handshake that does not use DH or the (base of) the symmetric keys used to run the actual encryption. Support was added to Wireshark with SVN revision 37401 to do this, so it became available with Wireshark 1.6. For instructions look at this question on ask.wireshark.org

解密 SSL connection 前提条件：

Since SVN revision 36876, it is also possible to decrypt traffic when you do not possess the server key but have access to the pre-master secret. For more details, see this security.stackexchange.com answer or this step-by-step walkthrough. That answer also contains some suggestions on finding out why SSL/TLS sessions do not get decrypted. In short, it should be possible to log the pre-master secret to a file with a current version of Firefox, Chromium or Chrome by setting an environment variable (SSLKEYLOGFILE=</path/to/private/directory/with/logfile>).

moooofly commented 5 years ago

一个要命的失误

image

在进行实验的时候，尝试点击了红框中的 Disable SSL... ，之后 SSL 相关的内容都看不到了；

image

更加尴尬的是，这个问题在网络上根本搜索不到解决办法（至少我搜到的内容都是教你如何解析 SSL 的）；于是我尝试在 Mac 上重新安装 Wireshark ，但重装后依旧无法显示 SSL 相关内容；

最后，还是在 Wireshark 图形界面中搜到了，如下图所示

image

image

image

moooofly commented 5 years ago

Decrypting TLS in Wireshark when using DHE_RSA ciphersuites

Some background: Wireshark supports decryption of SSL sessions when the master secret can be calculated (which can be derived from a pre-master secret). For cipher suites using the RSA key exchange, the private RSA key can be used to decrypt the encrypted pre-master secret.

解说：

For ephemeral Diffie-Hellman (DHE) cipher suites, the RSA private key is only used for signing the DH parameters (and not for encryption). These parameters are used in a DH key exchange, resulting in a shared secret (effectively the pre-master secret which is of course not visible on the wire).

解说：

Wireshark supports various methods to decrypt SSL:

  • By decrypting the pre-master secret using a private RSA key. Works for RSA key exchanges and subject to the above limitation.
  • Using a SSL keylog file which maps identifiers to master secrets. The available identifiers are:
    • The first 8 bytes (16 hex-encoded chars) of an encrypted pre-master secret (as transmitted over the wire in the ClientKeyExchange handshake message). (RSA XXX YYY, since Wireshark 1.6.0)
    • The 32 bytes (64 bytes hex-encoded chars) within the Random field of a Client Hello handshake message. (CLIENT_RANDOM XXX YYY, since Wireshark 1.8.0)
      • A variant that maps the Client Random to a pre-master secret (rather than master-secret) also exists. (PMS_CLIENT_RANDOM XXX ZZZ, since Wireshark 2.0)
      • Another variant exists to support TLS 1.3 and maps the Client Random to respective secrets. Instead of CLIENT_RANDOM, the key is one of CLIENT_EARLY_TRAFFIC_SECRET, CLIENT_HANDSHAKE_TRAFFIC_SECRET,SERVER_HANDSHAKE_TRAFFIC_SECRET, CLIENT_TRAFFIC_SECRET_0 or SERVER_TRAFFIC_SECRET_0. Since Wireshark 2.4.
    • The Session ID field of a Server Hello handshake message. (RSA Session-ID:XXX Master-Key:YYY, since Wireshark 1.6.0)
    • The Session Ticket in a Client Hello TLS extension or Session Ticket handshake message. (RSA Session-ID:XXX Master-Key:YYY, since Wireshark 1.11.3)

Wireshark 基于多种防范支持 SSL 解密：

To generate such a SSL key log file for a session, set the SSLKEYLOGFILE environment variable to a file before starting the NSS application. Example shell commands for Linux:

export SSLKEYLOGFILE=$PWD/premaster.txt
firefox

The SSL key log file can be configured for Wireshark at Edit -> Preferences, Protocols -> SSL, field (Pre)-Master-Secret log filename (or pass the -o ssl.keylog_file:path/to/keys.log to wireshark or tshark).

After doing this, you can decrypt SSL sessions for previous and live captures. Should you encounter a situation where you still cannot decrypt traffic, check:

  • whether the key log file path is correct (use absolute paths in case the program changes the working directory).
  • whether the key log file actually contains key material for your program.
  • whether Wireshark was compiled with GnuTLS (I have tested Wireshark 1.10.1 with GnuTLS 3.2.4 and libgcrypt 1.5.3)
  • whether other sessions can be decrypted. For instance, I tried https://lekensteyn.nl/ which works, but a site using a Camellia cipher suite failed.

解析：

To start debugging, save your capture and start wireshark with SSL logging enabled:

wireshark -o ssl.debug_file:debug.txt savedcapture.pcapng
moooofly commented 5 years ago

Wireshark/HTTPS

moooofly commented 5 years ago

Walkthrough: Decrypt SSL/TLS traffic (HTTPS and HTTP/2) in Wireshark

Disable the Diffie-Hellman cipher

We can confirm an SSL session is using a Diffie-Hellman cipher if the Cipher Suite value of the Server Hello message contains "ECDHE" or "DHE".

确认 SSL session 使用了 Diffie-Hellman 的办法：如果 "Server Hello" 消息中 "Cipher Suite" 的值包含 "ECDHE" or "DHE" 就代表使用了 Diffie-Hellman ；

image

The session key is transferred encrypted with a dynamically generated key pair (instead of encrypted with the public key from the certificate) if the SSL session is using a Diffie-Hellman cipher. The solution is to disable Diffie-Hellman from the client or the server. I would recommend to disable it from the client so that we can keep the server secure.

如果 SSL session 使用了 Diffie-Hellman ，那么 session key 则是基于 dynamically generated key pair 加密的（而不是基于 certificate 中的 public key 加密的）；

为了方便解密，可以在 client 侧或 server 避免 DH 加密；

Using SSLKEYLOGFILE

The second method to decrypt SSL/TLS packets is setting an environment variable called SSLKEYLOGFILE that points to a writable text file. Chrome and Firefox will look for the variable when they start up. If it exists, the browser will write the values used to generate TLS session keys out to that file. We can configure Wireshark to read this file to decrypt SSL/TLS packets.

基于 SSLKEYLOGFILE 和 Chrome 与 Firefox ，配置 wireshark 后就能解密 SSL/TLS 包了；

Summary

This article introduces two methods to decrypt SSL/TLS trace in Wireshark, you can evaluate the pros and cons of them to choose the best method for you.

The private key of the server certificate

Pros:

Cons:

SSLKEYLOGFILE

Pros:

Cons:

moooofly commented 5 years ago

Decrypting TLS Browser Traffic With Wireshark – The Easy Way!

Intro

One of the problems with the way Wireshark works is that it can’t easily analyze encrypted traffic, like TLS. It used to be if you had the private key(s) you could feed them into Wireshark and it would decrypt the traffic on the fly, but it only worked when using RSA for the key exchange mechanism. As people have started to embrace forward secrecy this broke, as having the private key is no longer enough derive the actual session key used to decrypt the data. The other problem with this is that a private key should not or can not leave the client, server, or HSM it is in. This lead me to coming up with very contrived ways of man-in-the-middling myself to decrypt the traffic(e.g. sslstrip or mitmproxy).

Session Key Logging to the Rescue!

Well my friends I’m here to tell you that there is an easier way! It turns out that Firefox and Chrome both support logging the symmetric session key used to encrypt TLS traffic to a file. You can then point Wireshark at said file and presto! decrypted TLS traffic. Read on to learn how to set this up.

Edit: If you are having trouble getting it to work on OS X take a look at the comments below. It seems that Apple has changed how environmental variables work in recent versions of OS X. Try launching firefox and wireshark within the same terminal window with,

# export SSLKEYLOGFILE=/Users/username/sslkeylogs/output.log
# open -a firefox
# wireshark