search

DanieleDeSensi / peafowl

High performance Deep Packet Inspection (DPI) framework to identify L7 protocols and extract and process data and metadata from network traffic.
MIT License
114 stars 41 forks source link

Inspect HTTPs packet #38

Closed prafulbagai closed 3 years ago

prafulbagai commented 3 years ago

Is it possible to inspect the HTTPs packet (its data/headers/complete URL)?

DanieleDeSensi commented 3 years ago

HTTPS packets are encrypted, so it is not possible to inspect the packets unless you have the key.

prafulbagai commented 3 years ago

Not even via the DPI?

On the other note, How do ISPs get to know my browsing history? Or the other free VPN providers, that make business out of our browsing data?

prafulbagai commented 3 years ago

I'm doing a POC on a product where I need to set up a VPN server and manipulate the request (basically the URL parameters, not the headers/form data). Wanted to understand whether that;s possible via DPI?

DanieleDeSensi commented 3 years ago

Not even via DPI. If the packet is encrypted you can't read the content unless you have the key. You can infer the server address from the TLS Hello packet during the handshake. I suggest you to read this StackOverflow thread that explains how it works https://stackoverflow.com/questions/499591/are-https-urls-encrypted However, I don't think you can modify the HTTPS URL without having the key.

prafulbagai commented 3 years ago

Thanks @DanieleDeSensi for the quick responses. The linked SO question was helpful. On a general note, wanted a suggestion (expert opinion) from you. When VPN orgs market themselves by saying ISPs peeks into your browsing history, how is it possible for ISPs then? Your thoughts? Is it only the DNS that they are able to view or much more than that?

prafulbagai commented 3 years ago

Also, after TLS 1.3, DNS is also not visible. That would leave VPNs out of business. Your thoughts?

prafulbagai commented 3 years ago

Also, FYI (a couple of years before) HTTPS packets were actually visible using some proxies(MITM/Squid). However, after the introduction of certificate pinning that also went away.

DanieleDeSensi commented 3 years ago

I am not 100% sure but I think this could either be done by checking the SNI (https://en.wikipedia.org/wiki/Server_Name_Indication), the JA3 fingerprint (https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967), or the DNS request