google / nogotofail

An on-path blackbox network traffic security testing tool
Apache License 2.0
2.94k stars 418 forks source link

Handlers in MiTM server for detection of PII in HTTP and HTTPS traffic #100

Open mkenne11 opened 8 years ago

mkenne11 commented 8 years ago

Handlers httppii and httpspii added to detect PII in HTTP and HTTPS traffic.

mkenne11 commented 8 years ago

This is 1 of 3 PRs to add Android app PII detection functionality to nogotofail.

The code added to the MiTM server, Android client and Android test harness components were separated across 3 PRs to hopefully simplify code review. Note. All 3 PRs need to be merged to ensure functionality is in sync.

mkenne11 commented 8 years ago

I generated timing metrics for the key PII handler methods. The timings were performed on a GCE server type "g1-small". During testing 10 Android apps were "manually" used over a 5 minute period and the server attempted to detect 10 PII data items (plus base-64 and URL encoded variants).

The times shown are the maximum time taken for each method, however the average time for each method was an order of magnitude faster.

"httppii" handler (class HttpPiiDetection):

"httpspii" handler (class HttpsPiiDetection):

I had trouble determing the best way to structure the HTTP and HTTPS PII handlers. "httppii" is a data handler and "httpspii" is a connection handler - there is a bit of code duplication between the two handler methods. I am keen to hear any suggestions you have on how to structure these :)

mkenne11 commented 8 years ago

Removed some unnecessary code.