Fewer domains returned from `intel` when IPv6 is enabled

[Thomascountz]

Originally reported by elderx via Discord

Short Description

When running amass intel -whois with IPv6 enabled, fewer domains are returned. Once IPv6 is disabled, the expected number of domains are returned.

Steps to Reproduce

Confirm IPv6 is enabled.

$ sudo sysctl net.ipv6.conf | grep disable

net.ipv6.conf.all.disable_ipv6 = 0
net.ipv6.conf.all.disable_policy = 0
net.ipv6.conf.default.disable_ipv6 = 0
net.ipv6.conf.default.disable_policy = 0
net.ipv6.conf.eth0.disable_ipv6 = 0
net.ipv6.conf.eth0.disable_policy = 0
net.ipv6.conf.lo.disable_ipv6 = 0
net.ipv6.conf.lo.disable_policy = 0

Run amass intel against example.com and store the result into a new file.

$ amass intel -whois -d example.com > amass_test_ipv6_enabled

Disabled IPv6 and confirm.

$ sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1 && \
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1 && \
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.lo.disable_ipv6 = 1

$ sudo sysctl net.ipv6.conf | grep disable

net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.all.disable_policy = 0
net.ipv6.conf.default.disable_ipv6 = 1
net.ipv6.conf.default.disable_policy = 0
net.ipv6.conf.eth0.disable_ipv6 = 1
net.ipv6.conf.eth0.disable_policy = 0
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.lo.disable_policy = 0

Run amass intel against example.com and store the result into a new file.

$ amass intel -whois -d example.com > amass_test_ipv6_disabled

Compare both runs. When IPv6 is enabled, 315 fewer domains are returned.

$ wc -l amass_test_ipv6_enabled
105  amass_test_ipv6_enabled

$ wc -l amass_test_ipv6_disabled
420  amass_test_ipv6_disabled

For reference.

$ amass intel -list
Data Source               | Type                    | Available
--------------------------------------------------------------------------------
360PassiveDNS               api
ASNLookup                   api                         *
AbuseIPDB                   scrape                      *
Ahrefs                      api
AlienVault                  api                         *
Alterations                 alt                         *
AnubisDB                    api                         *
ArchiveIt                   archive                     *
Arquivo                     archive                     *
Ask                         scrape                      *
AskDNS                      scrape                      *
BGPTools                    misc                        *
BGPView                     api                         *
Baidu                       scrape                      *
BeVigil                     api
BigDataCloud                api                         *
BinaryEdge                  api                         *
Bing                        scrape                      *
Brute Forcing               brute                       *
BufferOver                  api                         *
BuiltWith                   api                         *
C99                         api
CIRCL                       api
Censys                      cert
CertCentral                 cert
CertSpotter                 cert                        *
Chaos                       api
Cloudflare                  api                         *
CommonCrawl                 crawl                       *
Crtsh                       cert                        *
DNSDB                       api
DNSDumpster                 scrape                      *
DNSHistory                  scrape                      *
DNSRepo                     api
DNSSpy                      scrape                      *
DNSlytics                   api
Deepinfo                    api
Detectify                   api
Digitorus                   cert                        *
DuckDuckGo                  scrape                      *
FOFA                        api
FacebookCT                  cert
FullHunt                    api                         *
Gists                       scrape                      *
GitHub                      api
GitLab                      api
Google                      scrape                      *
GoogleCT                    cert                        *
Greynoise                   api                         *
HAW                         archive                     *
HackerOne                   scrape                      *
HackerTarget                api                         *
Hunter                      api                         *
HyperStat                   scrape                      *
IPdata                      api                         *
IPinfo                      api
IntelX                      api
LeakIX                      api
Maltiverse                  api                         *
Mnemonic                    api                         *
Netlas                      api                         *
NetworksDB                  api                         *
ONYPHE                      api
PKey                        scrape                      *
PassiveTotal                api
Pastebin                    api                         *
PentestTools                api
PublicWWW                   crawl                       *
Pulsedive                   api                         *
Quake                       api
RADb                        api                         *
RapidDNS                    scrape                      *
Riddler                     scrape                      *
Robtex                      api                         *
SOCRadar                    api
Searchcode                  api                         *
Searx                       scrape                      *
SecurityTrails              api                         *
ShadowServer                misc                        *
Shodan                      api                         *
SiteDossier                 scrape                      *
SonarSearch                 api                         *
Spamhaus                    api
SpyOnWeb                    scrape                      *
Spyse                       api
Sublist3rAPI                api                         *
Synapsint                   scrape                      *
TeamCymru                   misc                        *
ThreatBook                  api
ThreatCrowd                 api                         *
ThreatMiner                 api                         *
Twitter                     api
UKWebArchive                archive                     *
URLScan                     api                         *
Umbrella                    api
VirusTotal                  api                         *
Wayback                     archive                     *
WhoisXMLAPI                 api
Yahoo                       scrape                      *
Yandex                      api
ZETAlytics                  api
ZoomEye                     api

[Elderx]

Originally when i tested this, it was from enum command. When ipv6 was enabled on virtual machine is took much longer to complete running (even if it completed at all) and number of returned subdomains was also much lower compared to when i disabled ipv6.