search

blacklanternsecurity / bbot

A recursive internet scanner for hackers.
https://www.blacklanternsecurity.com/bbot/
GNU General Public License v3.0
4.19k stars 381 forks source link

Blacklist by Keyword #1310

Open TheTechromancer opened 3 months ago

TheTechromancer commented 3 months ago

Discussed in https://github.com/blacklanternsecurity/bbot/discussions/1243

Originally posted by **amiremami** April 9, 2024 In some programs, we need to blacklist specific path such as https://www.example.com/blog/ However, it seems this is not possible with bbot, I wanted to suggest if it's possible add blacklist based on keyword. So, if I add blog , then it won't scan or crawl any links that have blog in it. Thanks 🙏 Update: I was also thinking about a way to limit crawling of similar links. For example, a site can have 100k products. I want to crawl only one of them, because the others are similar to this. one. Or for example a site can have 50k posts, but I want to crawl one of them. That would be great if it's possible to implement this.