[Feature Request] Data Source Implementation Status

[fork-while-fork]

Amass Data Sources

Now that amass has support for API keys, I've decided to organize a list of all the data sources to track the development progress for each one. This can also serve as a living document to track future data sources. See the key below the table for more information on each field.

If there are any errors in the table, please let me know, I'll keep it updated.

Data Source	Basic	Advanced	URL
archiveit	Implemented	N/A	https://wayback.archive-it.org/all
archivetoday	Implemented	N/A	http://archive.is
arquivo	Implemented	N/A	http://arquivo.pt/wayback
ask	Implemented	Not Implemented	https://www.ask.com/web
baidu	Implemented	Not Implemented	https://www.baidu.com/s
bing	Implemented	Not Implemented	http://www.bing.com/search
censys	Implemented	#70	https://www.censys.io/domain/%s/table
certdb	N/A	#75	https://certdb.com/
certspotter	Implemented	Not Implemented	https://certspotter.com/api/v0/certs?domain=%s
commoncrawl	Implemented	N/A	http://index.commoncrawl.org/
crtsh	Implemented	N/A	https://crt.sh/
dnsdb	#73	#73	http://www.dnsdb.org/
dnsdumpster	Implemented	N/A	https://dnsdumpster.com/
dnstable	Implemented	N/A	https://dnstable.com/domain/%s
dogpile	Implemented	N/A	http://www.dogpile.com/search/web
entrust	Implemented	N/A	https://ctsearch.entrust.com/api/v1/certificates
exalead	Implemented	N/A	http://www.exalead.com/search/web/results/
findsubdomains	Implemented	N/A	https://findsubdomains.com/subdomains-of/%s
google	Implemented	Not Implemented	https://www.google.com/search
hackertarget	Implemented	Not Implemented	http://api.hackertarget.com/hostsearch/?q=%s
locarchive	Implemented	N/A	http://webarchive.loc.gov/all
netcraft	Implemented	N/A	https://searchdns.netcraft.com/?restriction=site+ends+with&host=%s
openukarchive	Implemented	N/A	http://www.webarchive.org.uk/wayback/archive
ptrarchive	Implemented	N/A	http://ptrarchive.com/tools/search3.htm?label=%s&date=ALL
riddler	Implemented	Not Implemented	https://riddler.io/search?q=pld:%s
robtex	Implemented	Not Implemented	https://freeapi.robtex.com/
sitedossier	Implemented	N/A	http://www.sitedossier.com/parentdomain/%s
securitytrails	Not Implemented	#70	https://api.securitytrails.com/v1/domain/%s/subdomains
threatcrowd	Implemented	N/A	https://www.threatcrowd.org/searchApi/v2/domain/report/?domain=%s
ukgovarchive	Implemented	N/A	http://webarchive.nationalarchives.gov.uk
virustotal	Implemented	Not Implemented	https://www.virustotal.com/en/domain/%s/information/
wayback	Implemented	Not Implemented	http://web.archive.org/web
yahoo	Implemented	Not Implemented	https://search.yahoo.com/search
google transparency report	Not Implemented	Not Implemeneted	https://transparencyreport.google.com/https/certificates
viewdns	Not Formally Implemented	Not Implemented	https://viewdns.info/
passive total	N/A	#74	https://api.passivetotal.org/api/docs/
cisco umbrella	N/A	#73	https://docs.umbrella.com/umbrella-api/docs
facebook certificate transparency	N/A	Not Implemented	https://developers.facebook.com/tools/ct/
shodan	N/A	#73	https://www.shodan.io/
binary edge	N/A	#73	https://api.binaryedge.io
twitter	N/A	#73	https://api.twitter.com/
urlscan	N/A	#74	https://urlscan.io/api
circl	N/A	#74	https://www.circl.lu/pdns
bufferover	#104	N/A	dns.bufferover.run/dns

Key

Term	Description
Implemented	Implemented in some capacity. This does not necessarily mean that amass leverages all the data the source provides.
Not Formally Implemented	Amass uses some data sources in an ad hoc manner in instances where collection of certain data hasn't been formalized (i.e. reverse whois)
Basic	Does not require authenticated access. The sources implemented this way typically scrape web pages for data
Advanced	Requires some sort of authenticated access such as an API key or some other credentials.
N/A	Does not support this type of access (i.e. the service does not support an API key)

[jpastuszuk]

@fork-while-fork - Seems like dnsdb.org domain is not serving it's purpose anymore

[caffix]

Closing this issue