search

shmilylty / OneForAll

OneForAll是一款功能强大的子域收集工具
GNU General Public License v3.0
8.22k stars 1.3k forks source link

由于系统缓冲区空间不足或队列已满，不能执行套接字上的操作。 #195

Closed prohk closed 2 years ago

prohk commented 3 years ago

是否使用了最新代码 是

Bug描述 批量探测扫出的域名存活的时候异常退出了，域名数量在1W多

运行环境

如何复现 复现步骤（选写）

复现命令（必写） oneforall.py --targets target.txt run

报错文本 Traceback (most recent call last): File "F:\python\lib\threading.py", line 932, in _bootstrap_inner self.run() File "F:\python\lib\threading.py", line 870, in run self._target(*self._args, self._kwargs) File "D:\渗透工具\web渗透工具\信息收集\OneForAll-4.4\common\request.py", line 209, in save db.insert_table(name, new_info) File "D:\渗透工具\web渗透工具\信息收集\OneForAll-4.4\common\database.py", line 91, in insert_table self.conn.query( File "D:\渗透工具\web渗透工具\信息收集\OneForAll-4.4\common\records.py", line 335, in query cursor = self._conn.execute(text(query), params) # TODO: PARAMS GO HERE File "F:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1011, in execute return meth(self, multiparams, params) File "F:\python\lib\site-packages\sqlalchemy\sql\elements.py", line 298, in _execute_on_connection return connection._execute_clauseelement(self, multiparams, params) File "F:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1124, in _execute_clauseelement ret = self._execute_context( File "F:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1316, in _execute_context self._handle_dbapi_exception( File "F:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1514, in _handle_dbapiexception util.raise(exc_info[1], with_traceback=excinfo[2]) File "F:\python\lib\site-packages\sqlalchemy\util\compat.py", line 182, in raise raise exception File "F:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1276, in _execute_context self.dialect.do_execute( File "F:\python\lib\site-packages\sqlalchemy\engine\default.py", line 593, in do_execute cursor.execute(statement, parameters) MemoryError Request Progress: 66%|███████████▏ | 7278/11026 [21:22<16:40:05, 16.01s/it]Fatal Python error: Cannot r ecover from MemoryErrors while normalizing exceptions. Python runtime state: initialized

Thread 0x000049b4 (most recent call first): File "F:\python\lib\copy.py", line 278 in _reconstruct File "F:\python\lib\copy.py", line 102 in copy File "F:\python\lib\site-packages\requests\cookies.py", line 352 in update File "F:\python\lib\site-packages\requests\cookies.py", line 544 in merge_cookies File "F:\python\lib\site-packages\requests\sessions.py", line 444 in prepare_request File "F:\python\lib\site-packages\requests\sessions.py", line 519 in request File "F:\python\lib\site-packages\requests\sessions.py", line 546 in get File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 135 in get_resp File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 145 in request File "F:\python\lib\threading.py", line 870 in run File "F:\python\lib\threading.py", line 932 in _bootstrap_inner File "F:\python\lib\threading.py", line 890 in _bootstrap

Thread 0x00003aa8 (most recent call first): File "F:\python\lib\copy.py", line 278 in _reconstruct File "F:\python\lib\copy.py", line 102 in copy File "F:\python\lib\site-packages\requests\cookies.py", line 352 in update File "F:\python\lib\site-packages\requests\cookies.py", line 544 in merge_cookies File "F:\python\lib\site-packages\requests\sessions.py", line 444 in prepare_request File "F:\python\lib\site-packages\requests\sessions.py", line 519 in request File "F:\python\lib\site-packages\requests\sessions.py", line 546 in get File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 135 in get_resp File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 145 in request File "F:\python\lib\threading.py", line 870 in run File "F:\python\lib\threading.py", line 932 in _bootstrap_inner File "F:\python\lib\threading.py", line 890 in _bootstrap

Thread 0x00005018 (most recent call first): File "F:\python\lib\site-packages\requests\cookies.py", line 343 in set_cookie File "F:\python\lib\site-packages\requests\cookies.py", line 352 in update File "F:\python\lib\site-packages\requests\cookies.py", line 544 in merge_cookies File "F:\python\lib\site-packages\requests\sessions.py", line 444 in prepare_request File "F:\python\lib\site-packages\requests\sessions.py", line 519 in request File "F:\python\lib\site-packages\requests\sessions.py", line 546 in get File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 135 in get_resp File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 145 in request File "F:\python\lib\threading.py", line 870 in run File "F:\python\lib\threading.py", line 932 in _bootstrap_inner File "F:\python\lib\threading.py", line 890 in _bootstrap

Thread 0x00004e7c (most recent call first): File "F:\python\lib\http\cookiejar.py", line 1214 in vals_sorted_by_key File "F:\python\lib\http\cookiejar.py", line 1219 in deepvalues File "F:\python\lib\http\cookiejar.py", line 1228 in deepvalues File "F:\python\lib\http\cookiejar.py", line 1228 in deepvalues File "F:\python\lib\site-packages\requests\cookies.py", line 351 in update File "F:\python\lib\site-packages\requests\cookies.py", line 544 in merge_cookies File "F:\python\lib\site-packages\requests\sessions.py", line 444 in prepare_request File "F:\python\lib\site-packages\requests\sessions.py", line 519 in request File "F:\python\lib\site-packages\requests\sessions.py", line 546 in get File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 135 in get_resp File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 145 in request File "F:\python\lib\threading.py", line 870 in run File "F:\python\lib\threading.py", line 932 in _bootstrap_inner File "F:\python\lib\threading.py", line 890 in _bootstrap

Thread 0x00005294 (most recent call first): File "F:\python\lib\http\cookiejar.py", line 1214 in vals_sorted_by_key File "F:\python\lib\http\cookiejar.py", line 1219 in deepvalues File "F:\python\lib\http\cookiejar.py", line 1228 in deepvalues File "F:\python\lib\http\cookiejar.py", line 1228 in deepvalues File "F:\python\lib\site-packages\requests\cookies.py", line 351 in update File "F:\python\lib\site-packages\requests\cookies.py", line 544 in merge_cookies File "F:\python\lib\site-packages\requests\sessions.py", line 444 in prepare_request File "F:\python\lib\site-packages\requests\sessions.py", line 519 in request File "F:\python\lib\site-packages\requests\sessions.py", line 546 in get File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 135 in get_resp File "D:\u6e17\u900f\u5de5\u5177\web\u6e17\u900f\u5de5\u5177\u4fe1\u606f\u6536\u96c6\OneForAll-4.4\common\request.py ", line 145 in request File "F:\python\lib\threading.py", line 870 in run File "F:\python\lib\threading.py", line 932 in _bootstrap_inner File "F:\python\lib\threading.py", line 890 in _bootstrap

预期结果 正常执行

实际结果 清晰而简洁的实际结果描述（选写，如出现什么错误）

屏幕截图 image image

日志上传 上传oneforall.log日志文件（复杂问题建议上传）

其他补充 关于bug的其他一些补充说明

Bronya-Rayi commented 3 years ago

https://blog.dreace.top/2020/Solve-An-operation-on-a-socket-could-not-be-performed-because-the-system-lacked-sufficient-buffer-space-or-because-a-queue-was-full/

改完注册表后再把config/setting.py里的请求线程数量调低就好了，虽然存活验证的会慢点，但是好在稳