shmilylty / OneForAll

OneForAll是一款功能强大的子域收集工具
GNU General Public License v3.0
8.09k stars 1.28k forks source link

在使用代理下,无法使用setting.py文件下代理模块中的api #221

Open KrystianLi opened 2 years ago

KrystianLi commented 2 years ago

是否使用了最新代码

Bug描述 在config/setting文件中配置代理如下 enable_request_proxy = True # 是否使用代理(全局开关) proxy_all_module = True # 代理所有模块 proxy_partial_module = ['GoogleQuery', 'AskSearch', 'DuckDuckGoSearch', 'GoogleAPISearch', 'GoogleSearch', 'YahooSearch', 'YandexSearch', 'CrossDomainXml', 'ContentSecurityPolicy'] # 代理自定义的模块 request_proxy_pool = [{'http': 'http://127.0.0.1:10080' }] 启动程序后依旧无法使用上述api模块

运行环境

如何复现 复现步骤(选写) 修改config/setting.py文件 enable_request_proxy = True # 是否使用代理(全局开关) proxy_all_module = True # 代理所有模块 proxy_partial_module = ['GoogleQuery', 'AskSearch', 'DuckDuckGoSearch', 'GoogleAPISearch', 'GoogleSearch', 'YahooSearch', 'YandexSearch', 'CrossDomainXml', 'ContentSecurityPolicy'] # 代理自定义的模块 request_proxy_pool = [{'http': 'http://127.0.0.1:10080' }]

复现命令(必写) py -3 oneforall.py --target vuln.cn run

报错文本 HTTPSConnectionPool(host='transparencyreport.google.com', port=443): Max retries exceeded with url: /transparencyreport/api/v3/httpsreport/ct/certsearch?include_expired=true&include_subdomains=true&domain=vuln.cn (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object GoogleQuery module took 15.0 seconds found 0 subdomains HTTPSConnectionPool(host='api.hackertarget.com', port=443): Max retries exceeded with url: /hostsearch/?q=vuln.cn (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFABDD088>, 'Connection to api.hackertarget.com timed out. (connect timeout=15)')) HackerTargetQuery module took 15.2 seconds found 0 subdomains HTTPSConnectionPool(host='www.google.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB1DA988>, 'Connection to www.google.com timed out. (connect timeout=15)')) GoogleSearch module took 15.0 seconds found 0 subdomains HTTPSConnectionPool(host='search.yahoo.com', port=443): Max retries exceeded with url: / (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB21B888>, 'Connection to search.yahoo.com timed out. (connect timeout=15)')) YahooSearch module took 15.5 seconds found 0 subdomains HTTPSConnectionPool(host='www.search.ask.com', port=443): Max retries exceeded with url: /web?q=site%3A.vuln.cn&page=1 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x0000004EFB00E788>, 'Connection to www.search.ask.com timed out. (connect timeout=15)'))

预期结果 正确输出所有api爬取结果

实际结果 无法使用google等需要代理的api

shmilylty commented 1 year ago

如果需要只代理自定义部分模块,需要设置proxy_all_module = False,代理不成功可以先检查一下代理是否正常工作。