报错文本
HTTPSConnectionPool(host='transparencyreport.google.com', port=443): Max retries exceeded with url: /transparencyreport/api/v3/httpsreport/ct/certsearch?include_expired=true&include_subdomains=true&domain=vuln.cn (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object
GoogleQuery module took 15.0 seconds found 0 subdomains
HTTPSConnectionPool(host='api.hackertarget.com', port=443): Max retries exceeded with url: /hostsearch/?q=vuln.cn (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFABDD088>, 'Connection to api.hackertarget.com timed out. (connect timeout=15)'))
HackerTargetQuery module took 15.2 seconds found 0 subdomains
HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB1DA988>, 'Connection to www.google.com timed out. (connect timeout=15)'))
GoogleSearch module took 15.0 seconds found 0 subdomains
HTTPSConnectionPool(host='search.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB21B888>, 'Connection to search.yahoo.com timed out. (connect timeout=15)'))
YahooSearch module took 15.5 seconds found 0 subdomains
HTTPSConnectionPool(host='www.search.ask.com', port=443): Max retries exceeded with url: /web?q=site%3A.vuln.cn&page=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB00E788>, 'Connection to www.search.ask.com timed out. (connect timeout=15)'))
是否使用了最新代码 是
Bug描述 在config/setting文件中配置代理如下 enable_request_proxy = True # 是否使用代理(全局开关) proxy_all_module = True # 代理所有模块 proxy_partial_module = ['GoogleQuery', 'AskSearch', 'DuckDuckGoSearch', 'GoogleAPISearch', 'GoogleSearch', 'YahooSearch', 'YandexSearch', 'CrossDomainXml', 'ContentSecurityPolicy'] # 代理自定义的模块 request_proxy_pool = [{'http': 'http://127.0.0.1:10080' }] 启动程序后依旧无法使用上述api模块
运行环境
如何复现 复现步骤(选写) 修改config/setting.py文件 enable_request_proxy = True # 是否使用代理(全局开关) proxy_all_module = True # 代理所有模块 proxy_partial_module = ['GoogleQuery', 'AskSearch', 'DuckDuckGoSearch', 'GoogleAPISearch', 'GoogleSearch', 'YahooSearch', 'YandexSearch', 'CrossDomainXml', 'ContentSecurityPolicy'] # 代理自定义的模块 request_proxy_pool = [{'http': 'http://127.0.0.1:10080' }]
复现命令(必写) py -3 oneforall.py --target vuln.cn run
报错文本 HTTPSConnectionPool(host='transparencyreport.google.com', port=443): Max retries exceeded with url: /transparencyreport/api/v3/httpsreport/ct/certsearch?include_expired=true&include_subdomains=true&domain=vuln.cn (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object GoogleQuery module took 15.0 seconds found 0 subdomains HTTPSConnectionPool(host='api.hackertarget.com', port=443): Max retries exceeded with url: /hostsearch/?q=vuln.cn (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFABDD088>, 'Connection to api.hackertarget.com timed out. (connect timeout=15)')) HackerTargetQuery module took 15.2 seconds found 0 subdomains HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB1DA988>, 'Connection to www.google.com timed out. (connect timeout=15)')) GoogleSearch module took 15.0 seconds found 0 subdomains HTTPSConnectionPool(host='search.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB21B888>, 'Connection to search.yahoo.com timed out. (connect timeout=15)')) YahooSearch module took 15.5 seconds found 0 subdomains HTTPSConnectionPool(host='www.search.ask.com', port=443): Max retries exceeded with url: /web?q=site%3A.vuln.cn&page=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB00E788>, 'Connection to www.search.ask.com timed out. (connect timeout=15)'))
预期结果 正确输出所有api爬取结果
实际结果 无法使用google等需要代理的api