search

projectdiscovery / shuffledns

MassDNS wrapper written in go to enumerate valid subdomains using active bruteforce as well as resolve subdomains with wildcard filtering and easy input-output support.
https://projectdiscovery.io
GNU General Public License v3.0
1.29k stars 184 forks source link

Intermediate output file can grow too large #20

Closed faizal3199 closed 4 years ago

faizal3199 commented 4 years ago

Currently, shuffledns doesn't set --output/-o option while calling massdns due to which massdns uses default Full Text Output mode. This mode includes Header and Question section to be included in massdns's output for every DNS query. This extra information is not needed for further processing and can be avoided.

Setting --output/-o to Sn can help reduce intermediate file size to a great extent.

faizal3199 commented 4 years ago

Can I make a pull request for this?

Ice3man543 commented 4 years ago

@faizal3199 actually the issue is that massdns full output is needed by the parser to correctly parse the output and handle various cases of DNS resolution like multiple CNAMES, NS, etc.

The parser that shuffledns uses is currently https://github.com/projectdiscovery/shuffledns/tree/master/pkg/parser. If you can figure out a way to make it work without breaking other things, that would be great for sure!

faizal3199 commented 4 years ago

@Ice3man543 I looked through parser's code. With --output option set to Snl, massdns will add an extra newline between two different DNS replies, thus adding an empty line and allowing shuffledns to differentiate between two DNS answer section.

As for handling A, NS and CNAME records, the code logic needs not to be changed much.

Ice3man543 commented 4 years ago

@faizal3199, thanks for looking into it so quickly. That makes sense, using Snl mode. This is not on our priority list for now, but if you want to make a PR to add this feature, by all means go for it. I'd be more than happy to help you getting it merged!

Ice3man543 commented 4 years ago

Nice work @faizal3199. I tested and it worked well.