many fake records - Githubissues

yam23 commented 6 years ago

Hi,

I was running your tool on my vps , it was running fine for quite sometime but yesterday I installed it on new VPS and I am getting weird result. there is one weird server(142.54.173.92) which is sending fake records , this server does not listed on resolvers.txt(using from your repo). It looks like something is wrong. I have tried to traceroute this ip but last 10 hopes are not visible and i have also tried to see code massdns but it is in compiled format.

I have copied your repo from my local system to vps wihtout using make command and once i received weird result , i delete entire folder and clone from online using git clone command and use make command .......still i receive this weird records from this weird server ...... It looks like two possible scenario , one of the resolver from list is sending records to this fake dns server(142.54.173.92) which is sending fake records as result. second could be somewhere in my vps , this records are stored due to using existing copy of the folder and re installation is not removing it .....i grep dns but i could not find any susceptible file

./subbrute.py names.txt example.com | ./bin/massdns -r resolvers.txt -t A -a -o -c 100 -w 18_dec_1.txt -

[18_dec_1.txt]

Note : i have changed result to example.com in above command and output file attached. I really appreciate if you can check on your side that if you are receiving this server in output or something wrong with massdns compiled file or can you provide me any lead to resolve this issue ?

Thanks, Anis

yam23 commented 6 years ago

It looks like all verisign dns servers are sending fake records .......they have put captcha on their website and it identifies those record as automated request. ....... I think we need to find solution for that as it will happen in future with all dns records :( ....need to send request at some interval .... find output of massdns as below that lead to above conclusion....

204.117.214.10:53 1513717321 NXDOMAIN gambler.example.com. IN A

example.com.    86242   IN  SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284843445 7200 3600 1209600 86400

216.194.28.33:53 1513717321 NXDOMAIN ajsupreme.example.com. IN A

example.com.    9878    IN  SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284843445 7200 3600 1209600 86400

216.194.28.33:53 1513717321 NXDOMAIN fpogate.example.com. IN A

example.com.    10642   IN  SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284843445 7200 3600 1209600 86400

216.175.203.51:53 1513717321 NXDOMAIN hpwadfg.example.com. IN A

example.com.    81999   IN  SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284843445 7200 3600 1209600 86400

216.52.65.33:53 1513717321 NXDOMAIN akk.example.com. IN A

example.com.    10642   IN  SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284843445 7200 3600 1209600 86400

208.78.24.238:53 1513717321 NXDOMAIN b2.example.com. IN A

example.com.    3600    IN  SOA a4.nstld.com. dnssupport.verisign-grs.com. 2284843445 7200 3600 1209600 86400

208.67.222.220:53 1513717321 NXDOMAIN mx43.example.com. IN A

yam23 commented 6 years ago

After backtracking , i finally found that this dns resolver 218.248.240.23 in resolver.txt file is giving noerror record with wrong ip address....it is belong to BSNL banglore, India. It looks like someone has corrupt dns record of this server and using it as bot or BSNL banglore has put some protection against automated dns request....you can verify from your side if it gives similar output to you....i have learned a lot today about DNS by backtracking this error..... still need to figure out and get clean result from massdns....working on it ......

blechschmidt commented 6 years ago

If you are operating with a concurrency of only 100 lookups, you might consider running a local unbound resolver which is more reliable. You might as well consider using the public DNS resolver list from https://public-dns.info or alternatively use the extraction script for resolvers from Censys scans which will still yield about a million usable resolver IP addresses after having verified their connectivity and superficial integrity by resolving known DNS names and filtering out resolvers that produce invalid replies.

A check allowing to optionally filter out IP addresses that do not belong to the resolvers in use has been implemented by 92fd870425956e14f32d93b8a62a249d17a8abe4 on the highspeed branch which is supposed to be merged into the master branch soon.

yam23 commented 6 years ago

It looks like I have lot to learn about DNS :) ... Thanks for your suggestion but I need to do my homework before I comment any further :) ... I do not want to waste your precious time. I really appreciate your reply as always

ehsandeep commented 6 years ago

Hi @blechschmidt,

Facing similar problem with resolvers reliability due to lot's fake result, so right now what I'm doing is taking output using existing resolvers and resolving again with google or Cloudflare DNS and comparing the result to remove bad resolvers from the list, and the list got sorted from 900 to around 350 active resolver, but could you suggest best way to sorting out bad resolvers from existing list or public-dns list other then comparing two result? additionally when you say getting some active resolvers from Censys scans do you mean zmap-results.csv.lz4 this file or the other one?

and the bigger question is that generally, everyone suggests running massdns with the existing list of resovlers.txt and again run with some reliable resolver to get the accurate result, it might be a good idea but don't you think resovlers.txt might drop some active subdomain at the 1st place only?

blechschmidt commented 6 years ago

Hi,

I do not know of any better way to check resolvers other than resolving known DNS names and compare the responses of the DNS servers to the expected result. To filter out bad resolvers, I usually query a set of known A, AAAA and MX records. In addition, I perform some IP address queries which I expect to return NXDOMAIN in order to filter out resolvers that deliver IP addresses to ad servers. As massdns does not perform iteration itself, the only way to ensure that massdns delivers reliable results consists in using reliable resolvers. Originally, massdns was not designed to be a subdomain enumeration tool, I just realized it could be used for this purpose as well. The Censys scan file I mean is available at https://censys.io/data/53-dns-lookup-full_ipv4. I don't recommend to use resolvers.txt anymore. Resolvers started to degrade quickly since (and because of?) the release of massdns.

It could indeed happen that a query is dropped silently by a resolver. If a resolver returns a NOERROR instead of a proper error code due to rate limiting although the queried record does exist, massdns will not notice and this specific subdomain will be missed. You could use a large enough set of resolvers and resolve every name multiple times in order to reduce this risk, but this will likely increase the false positive rate as well.

Maybe the IPv6 rate limiting evasion approach mentioned in the README is interesting in your case.

I might be able to release the code of my high-performance non-authoritative server performing iterations eventually, but at the moment it's incomplete. Currently, if you really need reliable results, I can only suggest to use known reliable resolvers (Google, Cloudflare etc.) or run a local unbound instance, which I experienced can resolve up to 20,000 queries per second on a dedicated server.

ehsandeep commented 6 years ago

Thank you so much for the detailed information, and for sure this can be used for the subdomain enumeration at mass level with the list of good resolvers of course and yes right now using the same approach to filter the bad one and will take look for the unbound as well.

I might be able to release the code of my high-performance non-authoritative server performing iterations eventually.

will wait for this as well, thanks for making and maintaining this project actively :)

ehsandeep commented 6 years ago

Hi @blechschmidt,

I was thinking more about solving this issue in a most effective way, and came up with an idea!

is it possible to resolve single domain with all given resolvers with massdns at the moment? NO? I mean each resolver will resolve the single domain and store the output, as we know the expected output for the given domain, we can filter out all the resolvers which will produce any other result than expected one? let me know what's your views on this?

yam23 commented 6 years ago

Hi,

what i did as workaround is that , whatever result i got after using massdns , I am again verifying those list(some hundreds list of item) with valid resolver using some bash script, that way we are not loosing the speed of massdns and we are also getting verified result.

Hope that helps !!

Thanks, Anis

On Wed, May 30, 2018 at 2:29 PM, madaratech notifications@github.com wrote:

Hi @blechschmidt https://github.com/blechschmidt,

I was thinking more about solving this issue in a most effective way, and came up with an idea!

is it possible to resolve single domain with all given resolvers with massdns at the moment? NO? I mean each resolver will resolve the single domain and store the output, as we know the expected output for the given domain, we can filter out all the resolvers which will produce any other result than expected one? let me know what's your views on this?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/blechschmidt/massdns/issues/22#issuecomment-393307798, or mute the thread https://github.com/notifications/unsubscribe-auth/AdjkU4noHkghHcf9r0KKtUE_tCOz3fjMks5t3wElgaJpZM4RHh0b .

--

Thanks, Anis, CISSP MISAM, IEEE-President, Ethical Hacker at Hacker0ne

plenumlab commented 6 years ago

Hi, I have ran into this issue but the workaround I found is a bad resolver would always return same ip address for all domains/subdomains so I simply wrote a script that count occurrences of the same ip if it is more the 5 then remove those records problem solved