search

blacklanternsecurity / bbot

A recursive internet scanner for hackers.
https://www.blacklanternsecurity.com/bbot/
GNU General Public License v3.0
4.45k stars 397 forks source link

CRT Module #1579

Open Sh4d0wHunt3rX opened 1 month ago

Sh4d0wHunt3rX commented 1 month ago

I was checking https://crt.sh/?q=dell.com and I got this:

image

It seems because of big results, it asks to use this instead https://crt.sh/?q=dell.com&exclude=expired&group=none

So, none of these return complete list of all certificates including expired. I wanted to know if bbot has a mechanism (or possible to implement?) that can get the complete list?

Thanks 🙏

Sh4d0wHunt3rX commented 1 week ago

The better way here instead of sending http request to the crt website is querying the crt database directly, which is much faster and will return results up to 10k.

By using

psql -t -h crt.sh -p 5432 -U guest certwatch

For example, I'm using this method for getting root domains.

image

The sql query can be shown on the website if you enable it.

I noticed subfinder is also using this method: https://github.com/projectdiscovery/subfinder/blob/9b16512d56f45534f61319c6a042cc28042d8d58/v2/pkg/subscraping/sources/crtsh/crtsh.go

TheTechromancer commented 1 week ago

dude! That's legit!