Closed LionYu66 closed 2 months ago
配置文件如下: common:
proxy_pool:
enable: false
# ip池地址,参考 https://github.com/jhao104/proxy_pool
proxy_pool_url: http://192.168.100.1:5010
query_task:
name: 任务_bilibili
enable: false
type: bilibili
intervals_second: 60
begin_time: "00:00"
end_time: "23:59"
target_push_name_list:
enable_dynamic_check: true
enable_living_check: true
uid_list:
cookie: ""
payload: ""
name: 任务_weibo enable: false type: weibo intervals_second: 60 begin_time: "00:00" end_time: "23:59" target_push_name_list:
uid_list:
name: 任务_xhs enable: true type: xhs intervals_second: 601 begin_time: "09:30" end_time: "23:59" target_push_name_list:
profile_id_list:
name: 任务_douyin enable: true type: douyin intervals_second: 60 begin_time: "21:30" end_time: "23:59" target_push_name_list:
signature_server_url: http://192.168.100.1:7878
username_list:
sec_uid_list:
douyin_id_list:
push_channel:
enable: false
type: serverChan_turbo
send_key:
corp_id:
agent_id:
corp_secret:
key:
access_token:
app_id:
app_secret:
receive_id_type:
receive_id:
webhook_key:
api_token:
chat_id:
key:
web_server_url:
webhook_url: https://xxx.com?title={{title}}&content={{content}}
request_method: GET
smtp_host:
smtp_port:
smtp_ssl: true
smtp_tls: false
sender_email:
sender_password:
receiver_email:
1、小红书频率过高会导致被ban一小会儿,你看日志,可以发现是好几小时才ban一次,所以这一点可以忽略不计:
stderr: 2024-06-25 09:56:59,216 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52
stderr: 2024-06-25 15:58:21,558 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52
stderr: 2024-06-25 20:49:26,920 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52
stderr: 2024-06-26 09:51:44,619 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52
stderr: 2024-06-26 20:44:12,589 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52
2、抖音的视频更新检测现在挂了,我还不知道怎么修复,可以先关上(开播检测是正常的)
不对,我看错了,竟然是小红书动态的id变了,我看看代码
辛苦了,顺便想问下以下这样配置是关闭抖音的视频更新检测吗? 然后时间如果想设置成今晚22:00到明天凌晨3:00该如何填写,之前我直接填写begin_time: "22:00",end_time: "03:00"似乎不能生效
signature_server_url: http://192.168.100.1:7878
username_list:
sec_uid_list:
douyin_id_list:
1、唉,现在小红书的接口不返回笔记id了,先用标题判断是否变化吧(存在误判的可能),后面再研究研究,代码已提交;
2、enable_dynamic_check: false
是关闭视频更新检测的意思;
3、目前不支持跨日期的时间段
好的辛苦作者了
在6月24号正常推送一次后,突然开始无法获取作者的作品列表,报错如下:
stderr: 2024-06-24 17:53:50,324 - 8-MainThread - query_xhs.py[line: 85] - INFO: 【小红书-查询动态状态-任务_xhs】【repboy】上一条动态id[667819c5000000001c026e0f],本条动态id[66793707000000001f005f45] stderr: 2024-06-24 17:53:50,709 - 8-MainThread - query_xhs.py[line:102] - INFO: 【小红书-查询动态状态-任务_xhs】【repboy】动态有更新,准备推送:【修勾日记|给你们介绍下新朋友(蓝胖子)】#贝特恩奶粉[话题 stderr: 2024-06-24 17:53:52,965 - 8-MainThread - bark.py[line: 17] - INFO: 【推送_推送通道_Bark】成功 stderr: 2024-06-24 20:34:38,479 - 8-MainThread - util.py[line: 28] - ERROR: 【小红书-查询动态状态-任务_xhs】:HTTPSConnectionPool(host='www.xiaohongshu.com', port=443): Read timed out. (read timeout=10) stderr: Traceback (most recent call last): stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 386, in _make_request stderr: self._validate_conn(conn) stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1040, in _validate_conn stderr: conn.connect() stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connection.py", line 414, in connect stderr: self.sock = ssl_wrapsocket( stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/util/ssl.py", line 449, in ssl_wrap_socket stderr: ssl_sock = _ssl_wrap_socketimpl( stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/util/ssl.py", line 493, in _ssl_wrap_socket_impl stderr: return ssl_context.wrap_socket(sock, server_hostname=server_hostname) stderr: File "/usr/local/lib/python3.9/ssl.py", line 501, in wrap_socket stderr: return self.sslsocket_class._create( stderr: File "/usr/local/lib/python3.9/ssl.py", line 1074, in _create stderr: self.do_handshake() stderr: File "/usr/local/lib/python3.9/ssl.py", line 1343, in do_handshake stderr: self._sslobj.do_handshake() stderr: socket.timeout: _ssl.c:1116: The handshake operation timed out stderr: stderr: During handling of the above exception, another exception occurred: stderr: stderr: Traceback (most recent call last): stderr: File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 486, in send stderr: resp = conn.urlopen( stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 785, in urlopen stderr: retries = retries.increment( stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/util/retry.py", line 550, in increment stderr: raise six.reraise(type(error), error, _stacktrace) stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/packages/six.py", line 770, in reraise stderr: raise value stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 703, in urlopen stderr: httplib_response = self._make_request( stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 389, in _make_request stderr: self._raise_timeout(err=e, url=url, timeout_value=conn.timeout) stderr: File "/usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout stderr: raise ReadTimeoutError( stderr: urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='www.xiaohongshu.com', port=443): Read timed out. (read timeout=10) stderr: stderr: During handling of the above exception, another exception occurred: stderr: stderr: Traceback (most recent call last): stderr: File "/app/common/util.py", line 26, in requests_get stderr: response = requests.get(url, headers=headers, params=params, proxies=proxies, timeout=10, verify=False) stderr: File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 73, in get stderr: return request("get", url, params=params, kwargs) stderr: File "/usr/local/lib/python3.9/site-packages/requests/api.py", line 59, in request stderr: return session.request(method=method, url=url, kwargs) stderr: File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 589, in request stderr: resp = self.send(prep, send_kwargs) stderr: File "/usr/local/lib/python3.9/site-packages/requests/sessions.py", line 703, in send stderr: r = adapter.send(request, kwargs) stderr: File "/usr/local/lib/python3.9/site-packages/requests/adapters.py", line 532, in send stderr: raise ReadTimeout(e, request=request) stderr: requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='www.xiaohongshu.com', port=443): Read timed out. (read timeout=10) stderr: 2024-06-25 09:56:59,216 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52 stderr: 2024-06-25 15:58:21,558 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52 stderr: 2024-06-25 20:49:26,920 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52 stderr: 2024-06-26 09:51:44,619 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52 stderr: 2024-06-26 20:44:12,589 - 8-MainThread - query_xhs.py[line: 52] - ERROR: 【小红书-查询动态状态-任务_xhs】请求返回数据为空,profile_id:61582f8f000000000201da52
其后进行重启报错如下:
stderr: 2024-06-26 22:14:17,514 - 8-MainThread - config.py[line: 18] - INFO: 加载配置common: {'proxy_pool': {'enable': False, 'proxy_pool_url': 'http://192.168.100.1:5010'}} stderr: 2024-06-26 22:14:17,649 - 8-MainThread - config.py[line: 23] - INFO: 加载配置query_task: [{}] stderr: 2024-06-26 22:14:17,649 - 8-MainThread - config.py[line: 28] - INFO: 加载配置push_channel: [{}] stderr: 2024-06-26 22:14:17,649 - 8-MainThread - main.py[line: 12] - INFO: 开始初始化推送通道 stderr: 2024-06-26 22:14:17,649 - 8-MainThread - main.py[line: 19] - INFO: 初始化推送通道: 推送通道_Bark,通道类型: bark stderr: 2024-06-26 22:14:17,649 - 8-MainThread - main.py[line: 23] - INFO: 初始化查询任务 stderr: 2024-06-26 22:14:17,649 - 8-MainThread - main.py[line: 28] - INFO: 初始化查询任务: 任务_xhs,任务类型: xhs stderr: 2024-06-26 22:14:18,187 - 8-MainThread - query_xhs.py[line: 79] - INFO: 【小红书-查询动态状态-任务_xhs】【repboy】动态初始化:deque(['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', ''], maxlen=20) stderr: 2024-06-26 22:14:18,188 - 8-MainThread - main.py[line: 28] - INFO: 初始化查询任务: 任务_douyin,任务类型: douyin stderr: 2024-06-26 22:14:18,309 - 8-MainThread - query_douyin.py[line: 80] - INFO: 【抖音-生成ttwid-任务_douyin】成功:1%7CLjE84jNiWSa4HM379KEsl_bsmCKdWWJvf4YOYCdbpdA%7C1719411258%7Cd63e6bf866824583e18ee0f37f91c72e6fdaa998648a5e8a1420b61e666d151c stderr: 2024-06-26 22:14:18,309 - 8-MainThread - cache.py[line: 7] - INFO: [本地缓存]将: ttwid -> 1%7CLjE84jNiWSa4HM379KEsl_bsmCKdWWJvf4YOYCdbpdA%7C1719411258%7Cd63e6bf866824583e18ee0f37f91c72e6fdaa998648a5e8a1420b61e666d151c 存入缓存中 stderr: 2024-06-26 23:10:37,718 - 8-MainThread - config.py[line: 18] - INFO: 加载配置common: {'proxy_pool': {'enable': True, 'proxy_pool_url': 'http://192.168.100.1:5010'}} stderr: 2024-06-26 23:10:37,718 - 8-MainThread - proxy.py[line: 23] - INFO: 【ip池】已启用,地址: http://192.168.100.1:5010 stderr: 2024-06-26 23:10:37,853 - 8-MainThread - config.py[line: 23] - INFO: 加载配置query_task: [{}] stderr: 2024-06-26 23:10:37,853 - 8-MainThread - config.py[line: 28] - INFO: 加载配置push_channel: [{}] stderr: 2024-06-26 23:10:37,853 - 8-MainThread - main.py[line: 12] - INFO: 开始初始化推送通道 stderr: 2024-06-26 23:10:37,853 - 8-MainThread - main.py[line: 19] - INFO: 初始化推送通道: 推送通道_Bark,通道类型: bark stderr: 2024-06-26 23:10:37,853 - 8-MainThread - main.py[line: 23] - INFO: 初始化查询任务 stderr: 2024-06-26 23:10:37,853 - 8-MainThread - main.py[line: 28] - INFO: 初始化查询任务: 任务_xhs,任务类型: xhs stderr: 2024-06-26 23:10:47,430 - 8-MainThread - proxy.py[line: 64] - INFO: 【ip池】尝试多次均未获取到有效ip stderr: 2024-06-26 23:10:48,090 - 8-MainThread - query_xhs.py[line: 79] - INFO: 【小红书-查询动态状态-任务_xhs】【repboy】动态初始化:deque(['', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', '', ''], maxlen=20) stderr: 2024-06-26 23:10:48,090 - 8-MainThread - main.py[line: 28] - INFO: 初始化查询任务: 任务_douyin,任务类型: douyin stderr: 2024-06-26 23:10:48,239 - 8-MainThread - query_douyin.py[line: 80] - INFO: 【抖音-生成ttwid-任务_douyin】成功:1%7Cj07X8-sbRT2KavzfC3EUMkg4YwjOFI4IQ6DFr_jcckg%7C1719414648%7C52fb5230857c141177a8e8e39cc3fbf60cdd9b696f69d5f59f0aec1e6bcd4fba stderr: 2024-06-26 23:10:48,239 - 8-MainThread - cache.py[line: 7] - INFO: [本地缓存]将: ttwid -> 1%7Cj07X8-sbRT2KavzfC3EUMkg4YwjOFI4IQ6DFr_jcckg%7C1719414648%7C52fb5230857c141177a8e8e39cc3fbf60cdd9b696f69d5f59f0aec1e6bcd4fba 存入缓存中 stderr: 2024-06-26 23:11:03,761 - 8-MainThread - proxy.py[line: 64] - INFO: 【ip池】尝试多次均未获取到有效ip stderr: 2024-06-26 23:11:03,957 - 8-MainThread - query_douyin.py[line: 41] - ERROR: 【抖音-查询任务-任务_douyin】出错:Expecting value: line 1 column 1 (char 0) stderr: Traceback (most recent call last): stderr: File "/app/query_task/query_douyin.py", line 36, in query stderr: self.query_dynamic(self.username_list[i], self.sec_uid_list[i]) stderr: File "/app/query_task/query_douyin.py", line 93, in query_dynamic stderr: result = json.loads(str(response.content, "utf-8")) stderr: File "/usr/local/lib/python3.9/json/init.py", line 346, in loads stderr: return _default_decoder.decode(s) stderr: File "/usr/local/lib/python3.9/json/decoder.py", line 337, in decode stderr: obj, end = self.raw_decode(s, idx=_w(s, 0).end()) stderr: File "/usr/local/lib/python3.9/json/decoder.py", line 355, in raw_decode stderr: raise JSONDecodeError("Expecting value", s, err.value) from None stderr: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)