search

EFForg / badger-sett

Automated training for Privacy Badger. Badger Sett automates browsers to visit websites to produce fresh Privacy Badger tracker data.
https://www.eff.org/badger-pretraining
MIT License
120 stars 15 forks source link

click around option during crawl #57

Closed ablanathtanalba closed 3 years ago

ablanathtanalba commented 4 years ago

Fixes #25 by identifying clickable links on each domain, then iterating through and clicking on each.

ablanathtanalba commented 4 years ago

Most recent commit now filters all clickable options as first party to the given domain. Question: is it best to limit the amount of links that can be clicked on?

ghostwords commented 4 years ago

I suggest adding print statements (just for development debugging) to see stuff like which links get selected for clicking, and also using the --no-xvfb option (I haven't tested it with Docker though, sorry) to actually see what the browser ends up doing.

ablanathtanalba commented 4 years ago

I suggest adding print statements (just for development debugging) to see stuff like which links get selected for clicking, and also using the --no-xvfb option (I haven't tested it with Docker though, sorry) to actually see what the browser ends up doing.

I've added some print statements. Testing with --no-xvfb actually causes the scan to crash when running it with Docker -- I didn't realize it until testing it with that suggestion you made. That error isn't special to this branch, I tested on master and it has the same problem with that build as well.

ghostwords commented 3 years ago

Implemented by #60.