search

AlephNullSK / dnsgen

Generates combination of domain names from the provided input.
MIT License
884 stars 115 forks source link

Remove Duplicates #10

Closed adityathebe closed 1 year ago

adityathebe commented 4 years ago

Dnsgen outputs a lot of duplicates

Input File

example.com
hey3.example.com
hey2.example.com
hey1.example.com

Command

dnsgen --fast targets.txt

Output

hey3.example.com
hey4.example.com
hey5.example.com
hey1.example.com
hey0.example.com
hey2.example.com
hey3.example.com
hey4.example.com
hey0.example.com
hey4.example.com
hey5.example.com
hey6.example.com
hey2.example.com
hey1.example.com
hey0.example.com

Desired output

hey0.example.com
hey1.example.com
hey2.example.com
hey3.example.com
hey4.example.com
hey5.example.com
hey6.example.com
trashbruder commented 3 years ago

Add a sort -u to your output and you'll get your desired output.

adityathebe commented 3 years ago

sort -u buffers the output which means dnsgen needs to generate all the million combinations before it can sort things out. Can't keep things flowing that way