xnl-h4ck3r / waymore

Find way more from the Wayback Machine, Common Crawl, Alien Vault OTX, URLScan & VirusTotal!
MIT License
1.72k stars 195 forks source link

Error #22

Closed 0xPugal closed 1 year ago

0xPugal commented 1 year ago
root@ubuntu-do:~/tools/waymore# python3 waymore.py -i targets.txt -mode U                                                            [210/210]

 _ _ _       _   _ ____                                                                                                                                        
| | | |_____| | | /    \  ___   ____ _____                                                                                                                     
| | | (____ | | | | | | |/ _ \ / ___) ___ |                                                                                                                    
| | | / ___ | |_| | | | | |_| | |   | |_| |                                                                                                                    
 \___/\_____|\__  |_|_|_|\___/| |   | ____/                                                                                                                    
            (____/   by Xnl-h4ck3r  \_____)                                                                                                                    

No value for "URLSCAN_API_KEY" in config.yml - consider adding (you can get a FREE api key at urlscan.io)                                                      
Getting links from 2 archive.org API requests (this can take a while for some do[ ERR ] archive.org connection error for page 1                                
Links found on archive.org: 1                                                                                                                                  

Getting links from the latest 93 commoncrawl.org index collections (this can takExtra links found on commoncrawl.org: 0                                        

Extra links found on alienvault.com: 0                                                                                                                         

[ ERR ] unable to get links from urlscan.io: HTTPSConnectionPool(host='urlscan.io', port=443): Max retries exceeded with url: /api/v1/search/?q=domain:secure.p
laid.com&size=10000 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eaba9f60>: Failed to establish a new connection:
 [Errno -3] Temporary failure in name resolution'))                                                                                                            
Links found for *.secure.target.com: 1 🤘                                                                                                                       

No value for "URLSCAN_API_KEY" in config.yml - consider adding (you can get a FREE api key at urlscan.io)                                                      
[ ERR ] unable to get links from archive.org: HTTPSConnectionPool(host='web.archive.org', port=443): Max retries exceeded with url: /cdx/search/cdx?url=*.app.q
uovo.com/*&collapse=&fl=timestamp,original,mimetype,statuscode,digest&filter=!mimetype:warc/revisit%7Ctext/css%7Cimage/jpeg%7Cimage/jpg%7Cimage/png%7Cimage/svg
%5C+xml%7Cimage/gif%7Cimage/tiff%7Cimage/webp%7Cimage/bmp%7Cimage/vnd%7Cimage/x%5C-icon%7Cimage/vnd%5C.microsoft%5C.icon%7Cfont/ttf%7Cfont/woff%7Cfont/woff2%7C
font/x%5C-woff2%7Cfont/x%5C-woff%7Cfont/otf%7Caudio/mpeg%7Caudio/wav%7Caudio/webm%7Caudio/aac%7Caudio/ogg%7Caudio/wav%7Caudio/webm%7Cvideo/mp4%7Cvideo/mpeg%7Cv
ideo/webm%7Cvideo/ogg%7Cvideo/mp2t%7Cvideo/webm%7Cvideo/x%5C-msvideo%7Cvideo/x%5C-flv%7Capplication/font%5C-woff%7Capplication/font%5C-woff2%7Capplication/x%5C
-font%5C-woff%7Capplication/x%5C-font%5C-woff2%7Capplication/vnd%5C.ms%5C-fontobject%7Capplication/font%5C-sfnt%7Capplication/vnd%5C.android%5C.package%5C-arch
ive%7Cbinary/octet%5C-stream%7Capplication/octet%5C-stream%7Capplication/pdf%7Capplication/x%5C-font%5C-ttf%7Capplication/x%5C-font%5C-otf%7Cvideo/webm%7Cvideo
/3gpp%7Capplication/font%5C-ttf%7Caudio/mp3%7Caudio/x%5C-wav%7Cimage/pjpeg%7Caudio/basic&filter=!statuscode:404%7C301%7C302&page=&showNumPages=True (Caused by 
NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eaba99c0>: Failed to establish a new connection: [Errno -3] Temporary failure i
n name resolution'))                                                                                                                                           
Getting links from the latest 93 commoncrawl.org index collections (this can takExtra links found on commoncrawl.org: 2                                        

[ ERR ] unable to get links from alienvault.com: HTTPSConnectionPool(host='otx.alienvault.com', port=443): Max retries exceeded with url: /api/v1/indicators/do
main/app.target.com/url_list?limit=500&page=&showNumPages=True (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eabab9
10>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))                                                                  
[ ERR ] unable to get links from urlscan.io: HTTPSConnectionPool(host='urlscan.io', port=443): Max retries exceeded with url: /api/v1/search/?q=domain:app.quov
o.com&size=10000 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eabab970>: Failed to establish a new connection: [E
rrno -3] Temporary failure in name resolution'))
Links found for *.app.target.com: 2 🤘

No value for "URLSCAN_API_KEY" in config.yml - consider adding (you can get a FREE api key at urlscan.io)
[ ERR ] unable to get links from archive.org: HTTPSConnectionPool(host='web.archive.org', port=443): Max retries exceeded with url: /cdx/search/cdx?url=*.demo.
target.com/*&collapse=&fl=timestamp,original,mimetype,statuscode,digest&filter=!mimetype:warc/revisit%7Ctext/css%7Cimage/jpeg%7Cimage/jpg%7Cimage/png%7Cimage/sv
g%5C+xml%7Cimage/gif%7Cimage/tiff%7Cimage/webp%7Cimage/bmp%7Cimage/vnd%7Cimage/x%5C-icon%7Cimage/vnd%5C.microsoft%5C.icon%7Cfont/ttf%7Cfont/woff%7Cfont/woff2%7
Cfont/x%5C-woff2%7Cfont/x%5C-woff%7Cfont/otf%7Caudio/mpeg%7Caudio/wav%7Caudio/webm%7Caudio/aac%7Caudio/ogg%7Caudio/wav%7Caudio/webm%7Cvideo/mp4%7Cvideo/mpeg%7C
video/webm%7Cvideo/ogg%7Cvideo/mp2t%7Cvideo/webm%7Cvideo/x%5C-msvideo%7Cvideo/x%5C-flv%7Capplication/font%5C-woff%7Capplication/font%5C-woff2%7Capplication/x%5
C-font%5C-woff%7Capplication/x%5C-font%5C-woff2%7Capplication/vnd%5C.ms%5C-fontobject%7Capplication/font%5C-sfnt%7Capplication/vnd%5C.android%5C.package%5C-arc
hive%7Cbinary/octet%5C-stream%7Capplication/octet%5C-stream%7Capplication/pdf%7Capplication/x%5C-font%5C-ttf%7Capplication/x%5C-font%5C-otf%7Cvideo/webm%7Cvide
o/3gpp%7Capplication/font%5C-ttf%7Caudio/mp3%7Caudio/x%5C-wav%7Cimage/pjpeg%7Caudio/basic&filter=!statuscode:404%7C301%7C302&page=&showNumPages=True (Caused by
 NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eabec250>: Failed to establish a new connection: [Errno -3] Temporary failure 
in name resolution'))
Getting links from the latest 93 commoncrawl.org index collections (this can tak[ ERR ] Common Crawl connection error for index https://index.commoncrawl.org/C
C-MAIN-2013-20-index
[ ERR ] Common Crawl connection error for index https://index.commoncrawl.org/CC-MAIN-2021-04-index
[ ERR ] Common Crawl connection error for index https://index.commoncrawl.org/CC-MAIN-2018-51-index
Extra links found on commoncrawl.org: 2                                                                                                                        

[ ERR ] unable to get links from alienvault.com: HTTPSConnectionPool(host='otx.alienvault.com', port=443): Max retries exceeded with url: /api/v1/indicators/do
main/demo.target.com/url_list?limit=500&page=&showNumPages=True (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eabed
150>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
[ ERR ] unable to get links from urlscan.io: HTTPSConnectionPool(host='urlscan.io', port=443): Max retries exceeded with url: /api/v1/search/?q=domain:demo.pla
id.com&size=10000 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35eabed4b0>: Failed to establish a new connection: [
Errno -3] Temporary failure in name resolution'))
Links found for *.target.com: 2 🤘                                    
xnl-h4ck3r commented 1 year ago

Hi @0xPugazh This doesn't seem to be an issue with the tool, but maybe a DNS or connecction issue on your side because it is unable to connect to any of the API provider URLs. Can you check that you have an internet connection and that you are able to connect to web.archive.org, otx.alienvault.com, etc from your machine?

xnl-h4ck3r commented 1 year ago

Hi @0xPugazh, I'll assume no response means it's probably not an issue anymore, and was uinlikely to be an issue wirth waymore itself anyway, so I'll close this one. Thanks