search

qd-today / qd

QD [v20240210] —— HTTP请求定时任务自动执行框架 base on HAR Editor and Tornado Server
https://qd-today.github.io/qd/
MIT License
4.46k stars 577 forks source link

[Bug] 无法更新公共模板缓存 #336

Closed BenMake closed 1 year ago

BenMake commented 2 years ago

Verify steps

QianDao Version

20220911

Bug on OS

Linux

Bug on Platform

Docker/Linux 32位

To Reproduce

登录qiandao->公共模板->强制更新

Describe the Bug

已注册仓库 https://github.com/qiandao-today/templates

公共模板一直是空白的， 无法更新， 强制更新和清除缓存都试过

QianDao config

No response

QianDao log

[I 221126 23:57:05 tornado.access web:2271] 304 GET /subscribe/1/ (10.0.1.30) 303.19ms
[W 221126 23:57:35 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/tpls_history.json [Warning] Timeout during request
[I 221126 23:57:35 tornado.access web:2271] 200 GET /subscribe/1/updating/ (10.0.1.30) 30408.96ms
    打开链接错误https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/tpls_history.json
[E 221126 23:57:35 qiandao.Web.Handler subscribe:141] UserID: 1 update Subscribe failed! Reason: 
[I 221127 00:00:20 tornado.access web:2271] 302 POST /subscribe/refresh/1/?op=clear (10.0.1.30) 128.00ms
[I 221126 23:57:36 tornado.access web:2271] 200 GET /subscribe/1/ (10.0.1.30) 97.87ms
[W 221127 00:00:50 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/tpls_history.json [Warning] Timeout while connecting
[I 221127 00:00:51 tornado.access web:2271] 304 GET /subscribe/1/updating/ (10.0.1.30) 30415.73ms
[E 221127 00:00:51 qiandao.Web.Handler subscribe:141] UserID: 1 update Subscribe failed! Reason:

Expected behavior

希望可以修复

Screenshots

No response

BenMake commented 2 years ago

[I 221126 23:57:05 tornado.access web:2271] 304 GET /subscribe/1/ (10.0.1.30) 303.19ms [W 221126 23:57:35 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/tpls_history.json [Warning] Timeout during request [I 221126 23:57:35 tornado.access web:2271] 200 GET /subscribe/1/updating/ (10.0.1.30) 30408.96ms 打开链接错误https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/tpls_history.json [E 221126 23:57:35 qiandao.Web.Handler subscribe:141] UserID: 1 update Subscribe failed! Reason: [I 221127 00:00:20 tornado.access web:2271] 302 POST /subscribe/refresh/1/?op=clear (10.0.1.30) 128.00ms [I 221126 23:57:36 tornado.access web:2271] 200 GET /subscribe/1/ (10.0.1.30) 97.87ms [W 221127 00:00:50 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/tpls_history.json [Warning] Timeout while connecting [I 221127 00:00:51 tornado.access web:2271] 304 GET /subscribe/1/updating/ (10.0.1.30) 30415.73ms [E 221127 00:00:51 qiandao.Web.Handler subscribe:141] UserID: 1 update Subscribe failed! Reason: [W 221127 00:05:25 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/189%E5%A4%A9%E7%BF%BC%E4%BA%91%E7%9B%98-%E8%B4%A6%E5%8F%B7%E7%89%88.har [Warning] Timeout during request [I 221127 00:04:33 tornado.access web:2271] 200 GET /subscribe/1/ (10.0.1.30) 264.87ms [I 221127 00:04:32 tornado.access web:2271] 302 POST /subscribe/refresh/1/?op=refresh (10.0.1.30) 167.70ms [W 221127 00:05:55 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E7%B2%BE%E6%98%93%E8%AE%BA%E5%9D%9B.har [Warning] Timeout while connecting [W 221127 00:06:25 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/leaguehd%E6%9F%A0%E6%AA%ACPT.har [Warning] Timeout during request [W 221127 00:06:55 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E7%BB%88%E7%82%B9%E8%AE%BA%E5%9D%9B.har [Warning] Timeout during request [W 221127 00:07:25 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E5%93%A5%E7%89%B9%E5%8A%A8%E6%BC%AB%E7%8E%8B%E5%9B%BD.har [Warning] Timeout while connecting [W 221127 00:07:55 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E8%90%8C%E5%87%BA%E8%A1%80%E5%8A%A8%E6%BC%AB%E8%AE%BA%E5%9D%9B.har [Warning] Timeout during request [W 221127 00:08:26 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E8%8A%B1%E7%81%AB%E5%AD%A6%E5%9B%AD.har [Warning] Timeout while connecting [I 221127 00:08:49 tornado.access web:2271] 200 GET /subscribe/1/updating/ (10.0.1.30) 256576.09ms [SQL: SELECT task.id, task.tplid, task.userid, task.disabled, task.retry_count, task.success_count, task.failed_count, task.last_failed_count, task.ctime, task.mtime, task.ontimeflg, task.ontime, task._groups, task.pushsw, task.newontime, task.init_env, task.env, task.session, task.retry_interval, task.last_success, task.last_failed, task.next, task.note [E 221127 00:08:54 qiandao.Worker worker:334] Schedule Producer get tasks failed! (sqlite3.OperationalError) database is locked [E 221127 00:08:49 qiandao.Web.Handler subscribe:141] UserID: 1 update Subscribe failed! Reason: [Errno 104] Connection reset by peer WHERE task.next <= ?] FROM task (Background on this error at: https://sqlalche.me/e/14/e3q8) [parameters: (1669478929.8409173,)] [I 221127 00:08:55 tornado.access web:2271] 200 GET /subscribe/1/ (10.0.1.30) 5676.27ms [E 221127 00:09:05 tornado.application ioloop:760] Exception in callback functools.partial(<function _HTTPConnection.init.. at 0xb3efc388>, <Task finished name='Task-1974' coro=<_HTTPConnection.run() done, defined at /usr/lib/python3.10/site-packages/tornado/simple_httpclient.py:290> exception=SSLZeroReturnError(6, 'TLS/SSL connection has been closed (EOF) (_ssl.c:997)')>) Traceback (most recent call last): File "/usr/lib/python3.10/site-packages/tornado/simple_httpclient.py", line 340, in run ret = callback() File "/usr/lib/python3.10/site-packages/tornado/ioloop.py", line 740, in _run_callback gen.convert_yielded(self.run()), lambda f: f.result() File "/usr/lib/python3.10/site-packages/tornado/simple_httpclient.py", line 287, in stream = await self.tcp_client.connect( File "/usr/lib/python3.10/site-packages/tornado/tcpclient.py", line 288, in connect stream = await stream.start_tls( File "/usr/lib/python3.10/site-packages/tornado/iostream.py", line 1389, in _do_ssl_handshake File "/usr/lib/python3.10/ssl.py", line 1342, in do_handshake self._sslobj.do_handshake() self.socket.do_handshake() ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:997) [E 221127 00:09:20 tornado.application ioloop:760] Exception in callback functools.partial(<function _HTTPConnection.init.. at 0xb3efc2f8>, <Task finished name='Task-1703' coro=<_HTTPConnection.run() done, defined at /usr/lib/python3.10/site-packages/tornado/simple_httpclient.py:290> exception=ConnectionResetError(104, 'Connection reset by peer')>) [W 221127 00:09:45 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/189%E5%A4%A9%E7%BF%BC%E4%BA%91%E7%9B%98-%E8%B4%A6%E5%8F%B7%E7%89%88.har [Warning] Timeout while connecting [E 221127 00:10:05 tornado.application ioloop:760] Exception in callback functools.partial(<function _HTTPConnection.init.. at 0xb3a3f538>, <Task finished name='Task-1865' coro=<_HTTPConnection.run() done, defined at /usr/lib/python3.10/site-packages/tornado/simple_httpclient.py:290> exception=ConnectionResetError(104, 'Connection reset by peer')>) [W 221127 00:11:17 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E5%93%A5%E7%89%B9%E5%8A%A8%E6%BC%AB%E7%8E%8B%E5%9B%BD.har [Warning] Timeout during request [W 221127 00:10:47 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E7%BB%88%E7%82%B9%E8%AE%BA%E5%9D%9B.har [Warning] Timeout while connecting [W 221127 00:10:16 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E7%B2%BE%E6%98%93%E8%AE%BA%E5%9D%9B.har [Warning] Timeout while connecting ConnectionResetError: [Errno 104] Connection reset by peer [W 221127 00:11:50 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E8%8A%B1%E7%81%AB%E5%AD%A6%E5%9B%AD.har [Warning] Timeout while connecting [W 221127 00:12:20 qiandao.Http.Fetcher fetcher:476] GET https://cdn.jsdelivr.net/gh/qiandao-today/templates@master/%E8%90%8C%E5%B9%BB%E4%B9%8B%E4%B9%A1.har [Warning] Timeout while connecting

BenMake commented 2 years ago

board2:~:# docker exec -it qiandao bash bash-5.1# ping cdn.jsdelivr.net PING cdn.jsdelivr.net (151.101.109.229): 56 data bytes 64 bytes from 151.101.109.229: seq=2 ttl=52 time=121.359 ms 64 bytes from 151.101.109.229: seq=5 ttl=52 time=124.019 ms 64 bytes from 151.101.109.229: seq=9 ttl=52 time=118.589 ms 64 bytes from 151.101.109.229: seq=17 ttl=52 time=118.036 ms 64 bytes from 151.101.109.229: seq=18 ttl=52 time=119.401 ms

a76yyyy commented 2 years ago

怀疑是jsdelivr可以ping通,但是下载很慢? 错误显示是超时30s

BenMake commented 1 year ago

怀疑是jsdelivr可以ping通,但是下载很慢? 错误显示是超时30s

从日志中看到 cdn.jsdelivr.net timeout 但是通过ping 和 浏览器访问均是可以获得文件的， 公共模版请求是哪里发出的， 想排查一下

a76yyyy commented 1 year ago

请更新最新源码后尝试, 如仍未成功, 请开启debug模式提取日志

a76yyyy commented 1 year ago

如果还有问题请reopen 这个 issue