search

blacklanternsecurity / bbot

A recursive internet scanner for hackers.
https://www.blacklanternsecurity.com/bbot/
GNU General Public License v3.0
4.58k stars 414 forks source link

Utilise DNS CAA records, extract authorised CAA's as affiliates, extract emails and URL's from any IODEF reporting destinations #1400

Closed colin-stubbs closed 3 months ago

colin-stubbs commented 4 months ago

Description

Process DNS CAA records, extract anything useful we find.

Usually low value, but sometimes includes interesting email addresses or URL's.

Approved public CA's that have web portals may also be of interest.

https://en.wikipedia.org/wiki/DNS_Certification_Authority_Authorization

apple.com provides a useful example, as from their CAA records we would currently expect to get the following additional information:

  1. affiliate "entrust.net", one of their approved public CA's, which they presumably have an account that one or more staff login to via Entrust Certificate Services ( https://login.entrust.net/ ) :-)
  2. email address "contact_pki@apple.com"

Given "pki.apple.com" is a sub-domain that does not have any A/AAAA/CNAME records ( though it does have TXT's... more on that later :-) ), bbot should at present filter it out from any final results.

Example,

(bbot-py3.10) user@bbot:~/bbot$ dig CAA apple.com

; <<>> DiG 9.18.18-0ubuntu0.22.04.2-Ubuntu <<>> CAA apple.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 24298
;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 65494
;; QUESTION SECTION:
;apple.com.                     IN      CAA

;; ANSWER SECTION:
apple.com.              26      IN      CAA     0 issue "entrust.net"
apple.com.              26      IN      CAA     0 iodef "mailto:contact_pki@apple.com"
apple.com.              26      IN      CAA     0 issue "pki.apple.com"

;; Query time: 0 msec
;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
;; WHEN: Thu May 23 16:47:43 AEST 2024
;; MSG SIZE  rcvd: 147

(bbot-py3.10) user@bbot:~/bbot$ bbot -m affiliates,dnscaa -t apple.com -y
[WARN] BBOT performs better with multiple DNS servers. Your system currently only has one.
[INFO] 
[INFO] ### MODULES ###
[INFO] 
[INFO] +------------+--------+-----------------+--------------------------------+------------------------------+-------------------+-------------------+
[INFO] | Module     | Type   | Needs API Key   | Description                    | Flags                        | Consumed Events   | Produced Events   |
[INFO] +============+========+=================+================================+==============================+===================+===================+
[INFO] | affiliates | scan   | No              | Summarize affiliate domains at | affiliates, passive, report, | *                 |                   |
[INFO] |            |        |                 | the end of a scan              | safe                         |                   |                   |
[INFO] +------------+--------+-----------------+--------------------------------+------------------------------+-------------------+-------------------+
[INFO] | dnscaa     | scan   | No              | Check for CAA records          | email-enum, passive, safe,   | DNS_NAME          | EMAIL_ADDRESS,    |
[INFO] |            |        |                 |                                | subdomain-enum               |                   | URL_UNVERIFIED    |
[INFO] +------------+--------+-----------------+--------------------------------+------------------------------+-------------------+-------------------+
[INFO] This is a passive scan. No connections will be made to target
[INFO] Scan with 2 modules seeded with 1 targets
[INFO] Loaded 2/2 scan modules (affiliates,dnscaa)
[INFO] Loaded 3/3 internal modules (aggregate,excavate,speculate)
[INFO] Loaded 3/3 output modules, (csv,human,json)
[INFO] Setting up modules...
[INFO] internal.speculate: No portscanner enabled. Assuming open ports: 80, 443
[SUCC] Setup succeeded for 8/8 modules.
[SUCC] Starting scan considerate_rachel
[SCAN]                  considerate_rachel (SCAN:984002ac41577b9bf02c9de2c2aad32d97472e6e)      TARGET  (in-scope)
[INFO] Encountered domain with wildcard DNS (MX,TXT): apple.com
[DNS_NAME]              apple.com       TARGET  (a-record, aaaa-record, caa-record, domain, in-scope, mx-record, mx-wildcard-domain, ns-record, resolved, soa-record, target, txt-record, txt-wildcard-domain, wildcard-domain)
[EMAIL_ADDRESS]         contact_pki@apple.com   dnscaa  (caa-record, in-scope, mx-wildcard-domain, txt-wildcard-domain, wildcard-domain)
[ORG_STUB]              apple   speculate       (distance-1)
[DNS_NAME]              entrust.net     CAA     (a-record, affiliate, caa-record, distance-1, domain, ns-record, resolved, soa-record, txt-record)
[DNS_NAME]              asia.apple.com  PTR     (a-record, in-scope, mx-record, mx-wildcard, ns-record, resolved, soa-record, subdomain, txt-record, txt-wildcard, wildcard)
[DNS_NAME]              _wildcard.apple.com     CAA     (in-scope, mx-record, mx-wildcard, resolved, subdomain, txt-record, txt-wildcard, wildcard)
[DNS_NAME]              mx-in-vib.apple.com     MX      (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              mx-in-hfd.apple.com     MX      (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              _spf.apple.com  TXT     (in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              firewire.apple.com      PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              _spf-txn.apple.com      TXT     (in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              iphone.apple.com        PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              mx-in-mdn.apple.com     MX      (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              applescript.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              mx-in-rno.apple.com     MX      (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              shake.apple.com PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              iworktrialbuy.apple.com PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              guide.apple.com PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              applejava.apple.com     PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              advertising.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              livepage.apple.com      PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              seminars.apple.com      PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              podcast.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              aperturetrialbuy.apple.com      PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp24.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-mx-lapp01.apple.com PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              rn-mailsvcp-ppex-lapp45.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              itunespartner.apple.com PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp34.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp44.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              ma1-aaemail-dr-lapp02.apple.com PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              ma-mailsvcp-mx-lapp01.apple.com PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              rn-mailsvcp-ppex-lapp14.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp35.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              hfd-mx01.apple.com      PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              rn-mailsvcp-ppex-lapp25.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              ma-mailsvcp-mx-lapp03.apple.com PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              vib-mx01.apple.com      PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              ma1-aaemail-dr-lapp01.apple.com PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              rn-mailsvcp-ppex-lapp15.apple.com       PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              ma1-aaemail-dr-lapp03.apple.com PTR     (a-record, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              crk-mailsvcp-mx-lapp01.euro.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              crk-mailsvcp-mx-lapp02.euro.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              c.ns.apple.com  NS      (a-record, aaaa-record, in-scope, resolved, subdomain)
[DNS_NAME]              b.ns.apple.com  NS      (a-record, aaaa-record, in-scope, resolved, subdomain)
[DNS_NAME]              a.ns.apple.com  NS      (a-record, aaaa-record, in-scope, resolved, subdomain)
[DNS_NAME]              d.ns.apple.com  NS      (a-record, aaaa-record, in-scope, resolved, subdomain)
[DNS_NAME]              euro.apple.com  speculate       (a-record, in-scope, mx-record, mx-wildcard, ns-record, resolved, soa-record, subdomain, txt-record, txt-wildcard, wildcard)
[DNS_NAME]              rn-mailsvcp-mx-lapp03.apple.com PTR     (a-record, caa-error, in-scope, resolved, subdomain, txt-record)
[DNS_NAME]              rn-mailsvcp-ppex-lapp24.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp14.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp34.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp44.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp45.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp25.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp35.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              rn-mailsvcp-ppex-lapp15.rno.apple.com   PTR     (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              mx-in.g.apple.com       MX      (a-record, in-scope, resolved, subdomain)
[DNS_NAME]              a.gslb.aaplimg.com      NS      (a-record, aaaa-record, affiliate, distance-1, resolved, subdomain)
[DNS_NAME]              ns1.g.aaplimg.com       NS      (a-record, aaaa-record, affiliate, distance-1, resolved, subdomain)
[DNS_NAME]              ns3.g.aaplimg.com       NS      (a-record, affiliate, distance-1, resolved, subdomain)
[DNS_NAME]              b.gslb.aaplimg.com      NS      (a-record, aaaa-record, affiliate, distance-1, resolved, subdomain)
[DNS_NAME]              ns2.g.aaplimg.com       NS      (a-record, aaaa-record, affiliate, distance-1, resolved, subdomain)
[DNS_NAME]              g.apple.com     speculate       (in-scope, mx-wildcard-domain, ns-record, resolved, soa-record, subdomain, txt-wildcard-domain, wildcard-domain)
[DNS_NAME]              ns4.g.aaplimg.com       NS      (a-record, affiliate, distance-1, resolved, subdomain)
[DNS_NAME]              usmsc2-extxfr-001.dns.apple.com SOA     (a-record, in-scope, resolved, subdomain)
[INFO] Finishing scan
[INFO] affiliates: +-------------------------------+---------+---------+
[INFO] affiliates: | Affiliate                     | Score   | Count   |
[INFO] affiliates: +===============================+=========+=========+
[INFO] affiliates: | aaplimg.com                   | 17.50   | 12      |
[INFO] affiliates: +-------------------------------+---------+---------+
[INFO] affiliates: | entrust.net                   | 2.00    | 1       |
[INFO] affiliates: +-------------------------------+---------+---------+
[INFO] affiliates: | webexdomainverification.8c462 | 2.00    | 1       |
[INFO] affiliates: +-------------------------------+---------+---------+
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | Module    | Produced                       | Consumed                         |
[INFO] aggregate: +===========+================================+==================================+
[INFO] aggregate: | PTR       | 56 (56 DNS_NAME)               | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | TXT       | 26 (7 DNS_NAME, 19 IP_ADDRESS) | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | NS        | 26 (26 DNS_NAME)               | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | MX        | 20 (20 DNS_NAME)               | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | A         | 10 (10 IP_ADDRESS)             | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | SOA       | 6 (6 DNS_NAME)                 | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | speculate | 5 (4 DNS_NAME, 1 ORG_STUB)     | 108 (66 DNS_NAME, 42 IP_ADDRESS) |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | AAAA      | 5 (5 IP_ADDRESS)               | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | CAA       | 4 (4 DNS_NAME)                 | 0                                |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] aggregate: | dnscaa    | 1 (1 EMAIL_ADDRESS)            | 6 (6 DNS_NAME)                   |
[INFO] aggregate: +-----------+--------------------------------+----------------------------------+
[INFO] output.csv: Saved CSV output to /home/user/.bbot/scans/considerate_rachel/output.csv
[INFO] output.human: Saved TXT output to /home/user/.bbot/scans/considerate_rachel/output.txt
[INFO] output.json: Saved JSON output to /home/user/.bbot/scans/considerate_rachel/output.ndjson
[SUCC] Scan considerate_rachel completed in 12 seconds with status FINISHED
[INFO] Saved word cloud (145 words) to /home/user/.bbot/scans/considerate_rachel/wordcloud.tsv
(bbot-py3.10) user@bbot:~/bbot$
colin-stubbs commented 4 months ago

@TheTechromancer with this one... I have a branch ready here: https://github.com/colin-stubbs/bbot/tree/dnscaa

Any suggestions on changes before I open a PR?

TheTechromancer commented 4 months ago

@colin-stubbs go ahead and open a draft, it's easier to see that way.

TheTechromancer commented 4 months ago

I took a look at your branch and I see you dug into the core code and took the time to write module tests. That's awesome!

Besides a few small things it looks good. We can go over it in more detail when you open the PR.

TheTechromancer commented 3 months ago

Closing as this was merged in https://github.com/blacklanternsecurity/bbot/pull/1402.