anshumanbh / brutesubs

An automation framework for running multiple open sourced subdomain bruteforcing tools (in parallel) using your own wordlists via Docker Compose
MIT License
257 stars 72 forks source link

resolve doesn't work when the subdomains have weird characters in them like unicode chars, encoded chars, etc. #1

Closed anshumanbh closed 7 years ago

anshumanbh commented 8 years ago

Modify the shell script to remove all subdomains such that they only begin with a-z or 0-9

dhaval17 commented 8 years ago

But what about subdomains like è. Let's say https://è.dr4cun0.com/ Your [a-z0-9] won't include è I guess

anshumanbh commented 8 years ago

@dhaval17 you bring up a good point. Thats the trade off when you work with different tools as opposed to writing your own. In this case, resolve doesn't play well with such subdomains from what I can tell. I will have to think of something to get this resolved. Do you have any suggestions in mind?

dhaval17 commented 8 years ago

@anshumanbh Here's a small list of characters allowed in sub domain name you can look into

But the problem I faced was dig converts https://è.dr4cun0.com/ to https://\195\168.dr4cun0.com/ which obviously didn't resolve

Don't know how python resolver would react

anshumanbh commented 8 years ago

@dhaval17 The resolver (resolve.go) I am using is actually written in GO - https://github.com/majek/goplayground/blob/master/resolve/resolve.go. I am not using python's resolver at all

dhaval17 commented 8 years ago

@anshumanbh Did it resolve è.dr4cun0.com because I checked gethostbyname in php and it doesn't

anshumanbh commented 8 years ago

@dhaval17 Looks like it did but it didn't find an IP. Are you sure that subdomain can actually be resolved?

Server: 8.8.8.8:53, sending delay: 8.333333ms (120 pps), retry delay: 1s è.dr4cun0.com, www.dr4cun0.com, 104.25.158.34 104.25.159.34 Resolved 2 domains in 0.086s. Average retries 1.000. Domains per second: 23.229

dhaval17 commented 8 years ago

@anshumanbh yes it should screen shot 2016-09-29 at 2 34 11 am 1

both x.dr4cun0.com and è.dr4cun0.com should behave similarly

dig x.dr4cun0.com +short

104.25.159.34 104.25.158.34

anshumanbh commented 8 years ago

@dhaval17 weird..resolve doesn't work in that case:

(env)usxxbharta1m1:test bharta1$ echo x.dr4cun0.com | resolve Server: 8.8.8.8:53, sending delay: 8.333333ms (120 pps), retry delay: 1s x.dr4cun0.com, 104.25.158.34 104.25.159.34 Resolved 1 domains in 0.125s. Average retries 1.000. Domains per second: 7.976 (env)usxxbharta1m1:test bharta1$ echo è.dr4cun0.com | resolve Server: 8.8.8.8:53, sending delay: 8.333333ms (120 pps), retry delay: 1s è.dr4cun0.com, Resolved 1 domains in 0.100s. Average retries 1.000. Domains per second: 9.973

dhaval17 commented 8 years ago

@anshumanbh Actually resolver for Go is good, atleast it says "Resolved 1 domains" wrt to other languages

Node.js :

{ Error: queryA ENODATA è.dr4cun0.com at errnoException (dns.js:28:10) at QueryReqWrap.onresolve as oncomplete code: 'ENODATA', errno: 'ENODATA', syscall: 'queryA', hostname: 'è.dr4cun0.com' }

Can you try ç.dr4cun0.com ?

anshumanbh commented 8 years ago

@dhaval17 Returning no IP is worse than throwing an error. At least in node.js, you know something went wrong. If you use resolve, there won't be any error. Hence, you would think ç.dr4cun0.com doesn't even exist. False Negatives are bad.

anshumanbh commented 8 years ago

@dhaval17 resolve will say that for every domain, whether it actually exists or not..unless it spits out an IP as well, it basically means it couldn't resolve

dhaval17 commented 8 years ago

@anshumanbh Oh that's misleading output then I though "Resolved 1 domains" meant Go has resolved a domain while couldn't fetch the IPs

dhaval17 commented 8 years ago

@anshumanbh I tried dig, php, Node.js, Python none of them seems to resolve è.dr4cun0.com while browsers seems to have no problem

Time to use áàäčçđéèêŋńñóòôöšŧüžæøå for admin consoles

anshumanbh commented 8 years ago

@dhaval17 touche. good to know this!

dhaval17 commented 8 years ago

@anshumanbh It seems like browsers are able to resolve è.dr4cun0.com because it's first converted to xn--8ca.dr4cun0.com

dig xn--8ca.dr4cun0.com +short

104.25.159.34 104.25.158.34

anshumanbh commented 8 years ago

@dhaval17 very interesting! but, how do we do this in scripts now?

dhaval17 commented 8 years ago

@anshumanbh idna can work here I think

Also there a nice npm module punycode.js in node.js

anshumanbh commented 7 years ago

@dhaval17 Finally, got some time to get this fixed. I managed to write a quick and dirty script to resolve domains here -https://github.com/anshumanbh/brutesubs/blob/master/scripts/isresolveable.go

I now use that to resolve the domains. It does resolve punycode as well now.

Try running it by go run isresolveable.go <file-with-domains>

dhaval17 commented 7 years ago

@anshumanbh

root@dr4cun0:/tmp# go run isresolveable.go a.txt 
google.com

root@dr4cun0:/tmp# cat a.txt 
google.com
dr4cun0.com.mx
è.dr4cun0.com
dr4cun0.côm

The correct list of resolvable domains should have had è.dr4cun0.com as well

anshumanbh commented 7 years ago

@dhaval17 This is working fine.

SDGL141bd20ad:scripts abhartiya$ cat a.txt google.com dr4cun0.com.mx è.dr4cun0.com dr4cun0.côm SDGL141bd20ad:scripts abhartiya$ go run isresolveable.go a.txt google.com

è.dr4cun0.com

For those domains that cannot be resolved, I am printing out a blank line.

dhaval17 commented 7 years ago

@anshumanbh Yes I get that

But in my case I don't see è.dr4cun0.com in output Maybe I am missing a package

Will check again