Plugin to allow probe request capture

[maximcherny]

Hi,

The original version of Snoopy (circa 2012) facilitated the ability to store the attributes of probe requests. The ng version only seems to capture aggregates via num_probes.

Is this a deliberate design decision?

Cheers

[glennzw]

The new version collects probe requests too. They're stored in a table called wifi_client_ssids, or explorable via the Maltego transform 'fetchClientSSIDs'.

[maximcherny]

Cool, thanks for clarifying!

[maximcherny]

Given the structures of:

_wifi_clientssids

• mac
• ssid
• sunc
• run_id

_wifi_clientobs

• mac
• first_obs l* ast_obs
• num_probes
• sunc
• run_id

I am unable to figure out where the actual probe requests are stored? I understand that _wifi_clientssids contains data on ssids being probed for. Does the plugin facilitate the storage of ALL probes sent by a device?

[glennzw]

Individual probe requests aren't stored. The wifi_client_obs table represents a grouping of observed probe requests - a "session" almost. If a device hasn't emitted a probe for a defined period of time (5 mins by default I think) then the session is closed, and subsequent probes will start a new observation / session.

[maximcherny]

Okay, would you be willing to accept a pull request to add a wifi plugin option that can set whether to also store individual probes plus the needed logic to perform so?

The default would be false of course.

There can be several benefits such as being able to perform subsequent pattern identification - i.e. a particular device appears only at certain times, days of week, days etc. Timing could be used to do stats for additional fingerprinting etc (if milliseconds are also captured).

[sensepost]

Of course! Pull request are not only welcome, but encouraged.

On Sat, Jul 12, 2014 at 11:35:28PM -0700, maximcherny wrote:

Okay, would you be willing to accept a pull request to add a wifi plugin option that can set whether to also store individual probes plus the needed logic to perform so?

The default would be false of course.

There can be several benefits such as being able to perform subsequent pattern identification - i.e. a particular device appears only at certain times, days of week, days etc. Timing could be used to do stats for additional fingerprinting etc (if milliseconds are also captured).

---

Reply to this email directly or view it on GitHub: https://github.com/sensepost/snoopy-ng/issues/4#issuecomment-48833231

|| Dominic White | SensePost | +27 82 305 3801 | PGP sensepost.com/pgp/dominic.txt ||

[glennzw]

I originally had each probe request saved in a separate table, but for the sake of saving space I decided to group probes into observed sessions (once my data got to >2million probes). Date/time identification is possible as it stands - we can see the first and last observation of a device, and the total number of probe requests emitted.

If I understand correctly, you want to know the exact arrival time of each probe request during an observation as this could be a method for identifying a device?

[maximcherny]

Yes, different manufacturers / OS are said to have different probe request timings.

I realise that the data storage and processing requirements would be significant and the data would not be intended for real-time analysis in most cases. But, if the option existed, it could be useful.

[glennzw]

Ah yes - I read a paper discussing this:

http://hal.inria.fr/docs/00/92/34/67/PDF/Wi-Fi_Stalking.pdf

See: "Frequency of the unique identifier transmission"

I'll add the functionality.

[maximcherny]

Cool, cheers. Also, the channel could be captured with every probe to facilitate identification based on "channel dwell time", since the probing strategies are not prescribed by the protocol and are implemented differently between vendors.

[jimbonet]

I have tried your headway version that captures and logs the probes individually which is a cool feature. I have then had problems when I sync the database with the back-end server(also running headway code). Although most tables are synced it appears that bluetooth_details hasn't transferred its data. Any ideas?

[maximcherny]

Glad you are interested in my fork. By no means it's a production-ready version though, mainly for me to try various bits and pieces. Do you know what error may be causing the sync to fail on the server end?

[jimbonet]

Not really sure - the sync appears to work correctly as both the pi(runs the wifi and bluetooth plugins) and the server(just runs as server) report they have transferred table data. when I look at the server's snoopy.db file in sqlite database browser I find all the new data in the tables has updated apart from the bluetooth_details table. Here is the process i use:

1. Pi called DRONEA runs wifi and bluetooth plugins as a standalone (not connected to network). I have fitted a ds1307 realtime clock (Adafruit) to keep time as no ntp.
2. I have a Desktop that runs just the server plugin with an account/key created called BACKUP.
3. After a period of time I sftp the snoopy.db file off the pi onto a laptop into its snoopy-ng-headway folder.
4. I then connect the laptop into my home network and run snoopy without plugins -d BACKUP -k(mykey) -s (my server ip). This now syncs all the table data from the pi (DRONEA)and creates a session entry called BACKUP in the server's snoppy.db.

It all works apart from that bluetooth_details table is empty (the orginal table from the pi has data in it from my test devices!its just not transferred).

Was anything changed in your bluetooth.py from the original sensepost code that may explain it?

[jimbonet]

Just to clarify - im not syncing the pi directly with the server. I ftp the .db file off the pi onto a laptop and then sync the laptop to the server - hope that makes sense.

[maximcherny]

I have not made any changes to the bluetooth plugin and to be honest I haven't really explored it yet. You might have some luck with increasing the verbosity level on the server side to see what went wrong.

Having done multiple tests with data syncing, I'm starting to lean towards an approach that would potentially rely on established native techniques such as database replication for this sort of thing. Nothing concrete at this stage though.

[JosephGregg]

Concerning data syncing: I was thinking that a light-weight approach might be to find a way to perform replication within SQLite itself. Maybe by using triggers. I plan on exploring options for data syncing possibilities in the coming weeks.

[glennzw]

Regarding data replication; I wrote the code because I couldn't find any standard ways to handle it, but it isn't the most elegant solution. I'm hoping to have a ZeroMQ version built at some point.