Generates lists of live hosts and URLs for targeting, automating the usage of Massdns, Masscan and nmap to filter out unreachable hosts
Given an input file of domain names, this script will automate the usage of MassDNS to filter out unresolvable hosts, and then pass the results on to Masscan to confirm that the hosts are reachable and on which ports. The script will then generate a list of full URLs to be used for further targeting (passing into tools like gobuster or dirsearch, or making HTTP requests), a list of reachable domain names, and a list of reachable IP addresses. As an optional last step, you can run an nmap version scan on this reduced host list, verifying that the earlier reachable hosts are up, and gathering service information from their open ports.
This script is especially useful for large domain sets, such as subdomain enumerations gathered from an apex domain with thousands of subdomains. With these large lists, an nmap scan would simply take too long. The goal here is to first use the less accurate, but much faster, MassDNS to quickly reduce the size of your input list by removing unresolvable domains. Then, Masscan will be able to take the output from MassDNS, and further confirm that the hosts are reachable, and on which ports. The script will then parse these results and generate lists of the live hosts discovered.
Now, the list of hosts should be reduced enough to be suitable for further scanning/testing. If you want to go a step further, you can tell the script to run an nmap scan on the list of reachable hosts, which should take more reasonable amount of time with the shorter list of hosts. After running nmap, any false positives given from Masscan will be filtered out. Raw nmap output will be stored in the regular nmap XML format, and additional information from the version detection will be added to a SQLite database.
If using the nmap scan option, this tool assumes that you already have nmap installed
Note: Running the install script is only needed if you do not already have MassDNS and Masscan installed, or if you would like to reinstall them inside this repo. If you do not run the script, you can provide the paths to the respective executables as arguments. The script additionally expects that the resolvers list included with MassDNS be located at {massDNS_directory}/lists/resolvers.txt
.
git clone https://github.com/allyomalley/LiveTargetsFinder.git
cd LiveTargetsFinder
sudo pip3 install -r requirements.txt
(OPTIONAL)
chmod +x install_deps.sh
./install_deps.sh
If you do not already have MassDNS and Masscan installed, and would prefer to install them yourself, see the documentation for instructions:
I have only tested this script on macOS and Linux - the python script itself should work on a Windows machine, though I believe the installation for MassDNS and Masscan will differ.
python3 liveTargetsFinder.py [domainList] [options]
Flag | Description | Default | Required |
---|---|---|---|
--target-list |
Input file containing list of domains, e.g google.com | Yes | |
--massdns-path |
Path to the MassDNS executable, if non-default | ./massdns/bin/massdns | No |
--masscan-path |
Path to the Masscan executable, if non-default | ./masscan/bin/masscan | No |
--nmap |
Run an nmap version detection scan on the gathered live hosts | Disabled | No |
--db-path |
If using the --nmap option, supply the path to the database you would like to append to (will be created if does not exist) | output/liveTargetsFinder.sqlite3 | No |
-s
, (--hashmap-size
) in particular was chosen for performance reasons - you will likely be able to increase this.-c 25 -o J -r ./massdns/lists/resolvers.txt -s 100 -w massdnsOutput -t A targetHosts
--max-rate
argument for Masscan - you will likely want to adjust this.
-iL ipFile -oD masscanOutput --open-only --max-rate 5000 -p80,443 --max-retries 10
--script http-server-header.nse,http-devframework.nse,http-headers -sV -T4 -p80,443 -oX {output.xml}
Did run install script:
python3 liveTargetsFinder.py --target-list victim_domains.txt
Did NOT run the install script:
python3 liveTargetsFinder.py --target-list victim_domains.txt --massdns-path ../massdns/bin/massdns --masscan-path ../masscan/bin/masscan
Perform an nmap scan and write to/append to the default DB path (liveTargetsFinder.sqlite3)
python3 liveTargetsFinder.py --target-list victim_domains.txt --nmap
Perform an nmap scan and write to/append to the specified database
python3 liveTargetsFinder.py --target-list victim_domains.txt --nmap --db-path serviceinfo_victim.sqlite3
Input: victimDomains.txt
File | Description | Examples |
---|---|---|
output/victimDomains_targetUrls.txt | List of reachable, live URLs | https://github.com, http://github.com |
output/victimDomains_domains_alive.txt | List of live domain names | github.com, google.com |
output/victimDomains_ips_alive.txt | List of live IP addresses | 10.1.0.200, 52.3.1.166 |
Supplied or default DB Path | SQLite database storing live hosts and information about their services running | |
output/victimDomains_massdns.txt | The raw output from MassDNS, in ndjson format | |
output/victimDomains_masscan.txt | The raw output from Masscan, in ndjson format | |
output/victimDomains_nmap.txt | The raw output from nmap, in XML format |