search

projectdiscovery / shuffledns

MassDNS wrapper written in go to enumerate valid subdomains using active bruteforce as well as resolve subdomains with wildcard filtering and easy input-output support.
https://projectdiscovery.io
GNU General Public License v3.0
1.26k stars 180 forks source link

shuffledns ignores NXDOMAIN records with valid CNAME entry #18

Closed streaak closed 3 years ago

streaak commented 4 years ago

Hello team, The above edge case helps us identify domains which can be taken over. However shuffledns filters out these domains due to the NXDOMAIN status. Note that this could be a massdns issue as discussed personally. Let me know if you need more information.

Best regards. streaak

streaak commented 4 years ago

Just some additional information, this doesnt look like a massdns issue, I tested this using updated resolvers list- Running massdns -r resolvers.txt -o S -t A Returns even NXDOMAIN status domains having a CNAME record along with the normal ones.

Regards, streaak

ehsandeep commented 4 years ago

Hey @streaak,

Thanks for adding more info on this, will try to get this sorted asap.

SbIm commented 4 years ago

Just some additional information, this doesnt look like a massdns issue, I tested this using updated resolvers list- Running massdns -r resolvers.txt -o S -t A Returns even NXDOMAIN status domains having a CNAME record along with the normal ones.

Regards, streaak

Hi, streaak, do you mean this kinds of scenarios? 

dig xxxxxxxx.target.com
xxxxxxxx.target.com.    IN  CNAME xxxxxxx.a.com
xxxxxxx.a.com.      IN     NXDOMAIN

massdns will return xxxxxxxx.target.com but shuffledns will filter it right?

streaak commented 4 years ago

Hey @sblm, Not exactly that format. It's like this-

$ dig sts.jet.com

; <<>> DiG 9.11.5-P4-5.1ubuntu2.1-Ubuntu <<>> sts.jet.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 8955
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;sts.jet.com.           IN  A

;; ANSWER SECTION:
sts.jet.com.        299 IN  CNAME   jetadfs.trafficmanager.net.

;; AUTHORITY SECTION:
trafficmanager.net. 29  IN  SOA tm1.msft.net. hostmaster.trafficmanager.net. 2003080800 900 300 2419200 30

;; Query time: 36 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Sun Apr 19 16:54:56 CEST 2020

Here you can see status: NXDOMAIN and CNAME pointing to trafficmanager.

massdns will return xxxxxxxx.target.com but shuffledns will filter it right?

Yes, that was my experience.

ehsandeep commented 4 years ago

Hi @SbIm @streaak,

sts.jet.com get filtered because we validate the host in with shuffledns and sts.jet.com is not a valid host as it has no A record associated with it, anyway for your specific use case you can use dnsprobe and it will detect jetadfs.trafficmanager.net

root@b0x:~# cat jet.txt | dnsprobe -r cname 

       __                            __       
  ____/ /___  _________  _________  / /_  ___ 
 / __  / __ \/ ___/ __ \/ ___/ __ \/ __ \/ _ \
/ /_/ / / / (__  ) /_/ / /  / /_/ / /_/ /  __/
\__,_/_/ /_/____/ .___/_/   \____/_.___/\___/ 
               /_/                                                                    

        projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
shop.jet.com shop-jet-com.rsys5.com.
images.jet.com images.jet.com.edgekey.net.
batman-assets.torbit.jet.com batman-assets.torbit.predc-jet.com.akadns.net.
www.swagstore.jet.com ext-cust.squarespace.com.
m.batman-api-staging.jet.com batman-api.jet.com.edgekey-staging.net.
careers.jet.com jet.prod.wmtcdev.com.
partner.jet.com partner.jet.com.edgekey.net.
batman-api.jet.com batman-api.jet.com.edgekey.net.
goto.jet.com goto-jet-com.ct.impactradius.com.
developer.jet.com ssl.readmessl.com.
eastus2-api.torbit.jet.com eastus2-api.torbit-dc.predc-jet.com.akadns.net.
westus.torbit.jet.com westus.torbit.predc-jet.com.akadns.net.
webhook.jet.com webhook.jet.com-v1.edgekey.net.
clicks.jet.com mandrillapp.com.
westus-api.torbit.jet.com westus-api.torbit-dc.predc-jet.com.akadns.net.
sts.jet.com jetadfs.trafficmanager.net.
tech.jet.com medium.tech.jet.com.edgekey.net.
alfred-api.jet.com batman-api.jet.com.edgekey.net.
merchant-api.jet.com merchant-api2.jet.com.edgekey.net.
webhook-stg.jet.com webhook.jet.com-v1.edgekey-staging.net.
go.jet.com cname.bitly.com.
m.batman-api.jet.com batman-api.jet.com.edgekey.net.
www.jet.com evsan.jet.com.edgekey.net.
link.jet.com cb.sailthru.com.
www.careers.jet.com jet.prod.wmtcdev.com.
eastus2.torbit.jet.com eastus2.torbit.predc-jet.com.akadns.net.
batmancdn.jet.com batmancdn.jet.com.edgekey.net.
merchant-api-staging.jet.com merchant-api2.jet.com.edgekey-staging.net.
streaak commented 4 years ago

Hey @bauthard That's not my use case as of now. It was to remove the wildcard domains using shuffledns and the tool helped me a lot with that case. However while doing so, it removed potential takeover domains. NXDOMAIN status occurs when there is no A record so just considering a domain with a CNAME record would also benefit many others.

Best regards, streaak

gy741 commented 3 years ago

Hello, @bauthard @streaak

I think this problem will improve if json output such as massdns is supported.

39 massdns json output

{
  "name": "exmpale.com.",
  "type": "A",
  "class": "IN",
  "status": "NXDOMAIN",
  "data": {
    "authorities": [
      {
        "ttl": 899,
        "type": "SOA",
        "class": "IN",
        "name": "exmpale.com.",
        "data": "ns-1714.awsdns-22.co.uk. awsdns-hostmaster.amazon.com. 1 7200 900 1209600 86400"
      }
    ]
  },
  "resolver": "8.8.8.8:53"
}
plenumlab commented 3 years ago

This is not an edge case, it allows for subdomain takeovers on specific cloud services please fix this, those CNAME records should be flagged as potential subdomain takeovers

https://hackerone.com/reports/389783

ehsandeep commented 3 years ago

Hey @plenumlab,

We are well aware of this, the point is shuffledns aims to detect valid host, not vulnerable takeovers, we have nuclei-template to detect this specific case.

You can simply detect this takeover, with nuclei -l subdomains.txt -t dns/dead-host-with-cname.yaml, we will surely consider this when we work on shuffledns in the future, but not on priority as this can be easily detected with nuclei.

streaak commented 3 years ago

Hey @bauthard I believe the issue is not with the detection of takeovers but rather removing the above cases from the output. When we use shuffledns as intended, it removes these domains from the final list which would hence be an issue. However, filtering invalid domains along with adding the above cases would help a lot.

adityathebe commented 3 years ago

However, filtering invalid domains along with adding the above cases would help a lot.

Any plan on adding this feature ?

ehsandeep commented 3 years ago

Hey @adityathebe @streaak,

https://github.com/projectdiscovery/shuffledns/blob/2c66f4987600807a6c75b0993051b5dba37e2124/pkg/massdns/process.go#L92

shuffedns executes following massdns command massdns -o Snl -t A -r $resolvers $input-list -w $output.

-o Snl eliminates the records having NXDOMAIN status code, and we are using Snl output switch for the reasons listed here https://github.com/projectdiscovery/shuffledns/issues/20, with that said this feature can not be supported due to limitations and issue mentioned in the referenced issue.

adityathebe commented 3 years ago

Hey @bauthard

massdns will still get the result even with those flags.

I have set up a test dns record for this

❯ dig whatisdaisydoingthereinnepal.csvops.guru
; <<>> DiG 9.16.10 <<>> whatisdaisydoingthereinnepal.csvops.guru
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 29497
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 1, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;whatisdaisydoingthereinnepal.csvops.guru. IN A

;; ANSWER SECTION:
whatisdaisydoingthereinnepal.csvops.guru. 1799 IN CNAME whatisdaisydoingthereinnepal.trafficmanager.net.

;; AUTHORITY SECTION:
trafficmanager.net. 30  IN  SOA tm1.dns-tm.com. hostmaster.trafficmanager.net. 2003080800 900 300 2419200 30

;; Query time: 453 msec
;; SERVER: 192.168.254.254#53(192.168.254.254)
;; WHEN: Sun Dec 27 19:37:44 +0545 2020
;; MSG SIZE  rcvd: 191

Massdns

$ echo "whatisdaisydoingthereinnepal.csvops.guru" | massdns --quiet -t A -o Snl -r resolvers.txt
whatisdaisydoingthereinnepal.csvops.guru. CNAME whatisdaisydoingthereinnepal.trafficmanager.net.
zealsham commented 3 years ago

Hello team, did this ever get sorted out . @streaak did you find a way around this. curious as this is indeed a critical issue.