BiliBili UP主动态抓取问题（未发现稳定复现的条件）

E1xP commented 7 months ago

路由地址

/bilibili/user/dynamic/:uid/:routeParams?

完整路由地址

bilibili/user/dynamic/1049200059

预期是什么？

正常抓取

实际发生了什么？

已正确配置B站Cookie，未改变环境变量后重启docker容器可正常抓取，一段时间后均无法抓取，调试需要Puppeteer。复现情况未发现稳定触发条件

部署

自建

部署相关信息

OS: Linux，DockerImages: diygod/rsshub:2024-04-04, Docker: v24.0.7

额外信息

[2024-04-11 20:00:45.873] info: <-- GET /bilibili/user/dynamic/1049200059
[2024-04-11 20:00:45.875] error: Error in /bilibili/user/dynamic/1049200059: Could not find Chrome (ver. 123.0.6312.86). This can occur if either
 1. you did not perform an installation before running the script (e.g. `npx puppeteer browsers install chrome`) or
 2. your cache path is incorrectly configured (which is: /app/node_modules/.cache/puppeteer).
For (2), check out our guide on configuring puppeteer at https://pptr.dev/guides/configuration.
[2024-04-11 20:00:45.876] info: --> GET /bilibili/user/dynamic/1049200059 503 3ms

这不是重复的 issue

[X] 我已经搜索了现有 issue，以确保该错误尚未被报告。

github-actions[bot] commented 7 months ago

Searching for maintainers:

/bilibili/user/dynamic/:uid/:routeParams?: @DIYgod @zytomorrow @CaoMeiYouRen @JimenezLi

To maintainers: if you are not willing to be disturbed, list your username in scripts/workflow/test-issue/call-maintainer.js. In this way, your username will be wrapped in an inline code block when tagged so you will not be notified.

If all routes can not be found, the issue will be closed automatically. Please use NOROUTE for a route-irrelevant issue or leave a comment if it is a mistake. 如果所有路由都无法匹配，issue 将会被自动关闭。如果 issue 和路由无关，请使用 NOROUTE 关键词，或者留下评论。我们会重新审核。

E1xP commented 7 months ago

重启容器后可正常抓取

> rsshub@1.0.0 start
> cross-env NODE_ENV=production tsx lib/index.ts

(node:47) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
[2024-04-11 20:09:19.774] info: 🎉 RSSHub is running on port 1200! Cheers!
[2024-04-11 20:09:19.777] info: 💖 Can you help keep this open source project alive? Please sponsor 👉 https://docs.rsshub.app/sponsor
[2024-04-11 20:09:19.777] info: 🔗 Local: 👉 http://localhost:1200
[2024-04-11 20:09:19.777] info: 🔗 Network: 👉 http://172.17.0.2:1200
[2024-04-11 20:09:52.690] info: <-- GET /bilibili/user/dynamic/1049200059
[2024-04-11 20:09:54.105] info: --> GET /bilibili/user/dynamic/1049200059 200 1s`

目前抓取间隔：每小时约12次；最短503时间：8小时，最长出现时间约15小时

Qixingchen commented 7 months ago

检测到风控后会使用 Puppeteer 来绕过风控,但你的 docker 没有 Puppeteer 所以就报错了

CaoMeiYouRen commented 7 months ago

目前的逻辑是如果检测到 -352 就会禁用掉配置项中的 cookie ，所以应该是你触发了 -352 风控

E1xP commented 7 months ago

目前的逻辑是如果检测到 -352 就会禁用掉配置项中的 cookie ，所以应该是你触发了 -352 风控

明白了，触发就因为没有Puppeteer报错，重启就重新启用了cookie。那目前修复方案是使用带chromium-bundled的镜像，对吧

DIYgod / RSSHub