search

owasp-amass / amass

In-depth attack surface mapping and asset discovery
https://owasp.org/www-project-amass/
Other
11.63k stars 1.85k forks source link

Multipe sources getting 403 Forbidden/404 Not Found/503 Service Unavailable/"no such host" responses in log file #950

Open prisoner881 opened 1 year ago

prisoner881 commented 1 year ago

From my amass.log file:

"20:55:34.041797 Sublist3rAPI: https://api.sublist3r.com/search.php?domain=somedomain.com: Get "https://api.sublist3r.com/search.php?domain=somedomain.com": dial tcp: lookup api.sublist3r.com on 172.26.128.1:53: no such host
20:55:34.041848 Sublist3rAPI: vertical request to service failed: Get "https://api.sublist3r.com/search.php?domain=somedomain.com": dial tcp: lookup api.sublist3r.com on 172.26.128.1:53: no such host
20:55:34.054965 AbuseIPDB: get_ip request to service returned with status code: 403 Forbidden
20:55:34.210207 Greynoise: vertical request to service returned with status: 404 Not Found
20:55:35.052066 DNSHistory: vertical request to service returned with status code: 403 Forbidden
20:55:36.141506 Pulsedive: vertical request to service returned with status: 400 Bad Request
20:55:38.602192 Arquivo: vertical request to service returned with status code: 503 Service Unavailable
20:55:45.024356 Searx: https://northboot.xyz/search: Post "https://northboot.xyz/search": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
20:55:45.024403 Searx: scrape: Post "https://northboot.xyz/search": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
20:55:47.024397 CommonCrawl: https://index.commoncrawl.org/CC-MAIN-2023-06-index?fl=url&output=json&url=%2A.somedomain.com: Get "https://index.commoncrawl.org/CC-MAIN-2023-06-index?fl=url&output=json&url=%2A.somedomain.com": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
20:55:47.024444 CommonCrawl: scrape: Get "https://index.commoncrawl.org/CC-MAIN-2023-06-index?fl=url&output=json&url=%2A.somedomain.com": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

I get that CommonCrawl is timing out, but the other services look like they're rejecting outright. All of the above comes from the default amass install, no config.yaml or datasources.yaml are involved.

SmallKingfisher commented 12 months ago

Same thing here and in my case it doesn't give any results at all. This worked fine, but since a few days it is broken.

amass enum -passive -d tesla.com
No names were discovered
amass -version
v4.0.0
zesty commented 11 months ago

for the grab bag of datasource issues: passivetotal was acquired by MS, and ... is no more?

pak0s commented 4 months ago

@zesty @SmallKingfisher @prisoner881 were you able to find any solution?

zesty commented 4 months ago

@pak0s nope

krlabs commented 2 months ago

We have an absolutely identical problem. The logs are full of access errors for API services, although the configuration file config.yaml and datasources.yaml are filled in. The application is installed via Go.

00:26:47.894971 DNSHistory: vertical request to service returned with status code: 403 Forbidden
00:26:47.914128 DNSDB: vertical request to service returned with status: 403 forbidden
00:26:48.056845 IntelX: search request to service returned with status: 403 Forbidden
00:26:48.195123 Pulsedive: vertical request to service returned with status: 429 Too Many Requests
00:26:48.876284 FOFA: error in vertical service response: [820031] F点余额不足
00:26:48.888219 IPinfo: as_info request to service returned with status: 401 Unauthorized
00:26:48.921021 Querying WhoisXMLAPI for kr-labs.com.ua subdomains
00:26:49.005361 DNSDB: vertical request to service returned with status: 403 forbidden
00:26:49.521942 Censys: vertical request to service returned with status code: 404 Not Found

In conclusion, there are no results.

If you install Amass from the Kali Linux repository, everything works fine.