Closed forchannot closed 1 year ago
目前小样本实验中,直接访问 https://m.bilibili.com/opus/{dynamic_id}
不会出现验证码,而通过 https://m.bilibili.com/dynamic/{dynamic_id}
或 https://t.bilibili.com/{dynamic_id}
跳转到前者会大概率出现验证码。
需更多样本测试
可将移动端截图功能中的URL添加opus/
暂时避免此问题。
那电脑端有解决办法吗
电脑端同样存在 https://www.bilibili.com/opus/{dynamic_id}
链接,但我没测试过
(莫不是阿b终于发现动态翻译成dynamic太蠢了)
async def get_dynamic_screenshot_mobile(dynamic_id):
"""移动端动态截图"""
url = f"https://m.bilibili.com/dynamic/{dynamic_id}"
和
async def get_dynamic_screenshot_pc(dynamic_id):
"""电脑端动态截图"""
url = f"https://t.bilibili.com/{dynamic_id}"
请问下是把上述两个部分分别改为
https://m.bilibili.com/opus/{dynamic_id}
和 https://t.bilibili.com/opus/{dynamic_id}
吗
移动端可以直接改,我刚看了一下电脑端改链接的话样式就变了,更接近移动端样式,不存在.card
,所以仍然会截全图
如果一定要用电脑端的话,或许可以添加:
if await page.query_selector(".geetest_panel"):
return None
来避免截图,并在下一次爬取时重试。
好的,感谢,我试一下,我发现好像www.bilibili.com/opus/
会直接跳转到t.bilibili.com/
,似乎并没有出现样式改变
好的,感谢,我试一下,我发现好像
www.bilibili.com/opus/
会直接跳转到t.bilibili.com/
,似乎并没有出现样式改变
群友测试反馈改成/opus/
会有很多问题😫,例如:
/dynamic/
链接所以还是建议用刷新重试的方法😢:
if await page.query_selector(".geetest_panel"):
return None
另外,修改UA可能会降低出验证码的概率
这行代码加在哪里呢
这行代码加在哪里呢
我改了一些地方所以不确定原本应该是第几行,但你应该能在get_dynamic_screenshot_pc
和get_dynamic_screenshot_mobile
函数内部找到判断404页面的两行代码,例如:
# 动态被删除或者进审核了
if page.url == "https://m.bilibili.com/404":
return None
加到这两行代码的前面或后面都可以。
好的感谢
期待反馈
目前关注的主播都还没有更新动态,等动态更新样本多了再来反馈
今天一天都还挺稳定的,不知道是因为我改了ua还是加了
if page.url == "https://m.bilibili.com/404":
return None
的原因
今天一天都还挺稳定的,不知道是因为我改了ua还是加了
if page.url == "https://m.bilibili.com/404": return None
的原因
今天不是某个二字游戏前瞻,我这边仍然有验证码
看来是lz的解决方法有效果了
今天前瞻十几个都没遇到验证码,暂时稳定,ua我是直接去抓的物理手机上的真实ua,然后也加上了那段代码
我这边使用的pc端截图,但是我发现源代码没有PC端的UA设置,加上之后验证码消失了,目前恢复正常,需要再观察几天. ps:我试过了部署到其他ip的服务器,确认ip地址不同,依然无法解决这个问题,确认不是ip引起的风控. pps:我修改了移动端的地址,也就是改为opus,配置文件改为使用移动端后,发现样式和pc端一样,目前不知道是配置文件未生效还是其它问题,待验证
pps:我修改了移动端的地址,也就是改为opus,配置文件改为使用移动端后,发现样式和pc端一样,目前不知道是配置文件未生效还是其它问题,待验证
mobile加上刷新重试的代码的话会无限获取动态,然而一直发不出来。换成pc后成功发出并且无验证码。 ps:mobile这边加不加/opus/都会无限获取
我一直没用移动端,也只是改了代码,一直用的电脑端截图,反正是电脑端添加上UA就正常了
mobile加上刷新重试的代码的话会无限获取动态,然而一直发不出来。换成pc后成功发出并且无验证码。 ps:mobile这边加不加/opus/都会无限获取
修改UA可降低出现验证码概率
手机样式,尝试更改UA,可以解决,一两天后也会重新出现验证码。目前我的解决办法就是一天改一次UA。
手机样式,尝试更换了opus和修改了AU,还添加了代码,目前动态截图稳定,就是视频推送动态依旧是验证码
06-05 12:12:22 [ERROR] haruka_bot | 截取动态时发生错误:https://m.bilibili.com/dynamic/803588020310638661
Traceback (most recent call last):
File "E:\IDEAyes\BOT\nonebot2\bot.py", line 60, in <module>
nonebot.run(app="__mp_main__:app")
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\nonebot\__init__.py", line 309, in run
get_driver().run(*args, **kwargs)
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\nonebot\drivers\fastapi.py", line 198, in run
uvicorn.run(
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\uvicorn\main.py", line 569, in run
server.run()
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\uvicorn\server.py", line 60, in run
return asyncio.run(self.serve(sockets=sockets))
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 633, in run_until_complete
self.run_forever()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 321, in run_forever
super().run_forever()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 600, in run_forever
self._run_once()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1896, in _run_once
handle._run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\apscheduler\executors\base_py3.py", line 30, in run_coroutine_job
retval = await job.func(*job.args, **job.kwargs)
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\haruka_bot\plugins\pusher\dynamic_pusher.py", line 74, in dy_sched
image = await get_dynamic_screenshot(dynamic_id)
File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\haruka_bot\utils\browser.py", line 40, in get_dynamic_screenshot
return await get_dynamic_screenshot_mobile(dynamic_id)
> File "E:\IDEAyes\BOT\nonebot2\venv\lib\site-packages\haruka_bot\utils\browser.py", line 102, in get_dynamic_screenshot_mobile
assert card
AssertionError
修改了UA之后,bot现在是在晚上经常会出现空图或者动态不存在的截图情况… 不知道是ip问题还是UA问题,如果是UA问题,求助一下各位有没有什么办法能够添加多个UA啊(?)谢谢!
等修复吧
---原始邮件--- 发件人: @.> 发送时间: 2023年6月13日(周二) 晚上7:14 收件人: @.>; 抄送: @.**@.>; 主题: Re: [SK-415/HarukaBot] 【错误回报】动态推送出现验证码,且高频率复现 (Issue #366)
尝试了此issue中的所有方法,更改opus、更改UA、添加代码、更改截图方式,而且均单独测试过,仍无法解决 以下为报错 06-13 19:11:02 [INFO] haruka_bot | 检测到新动态(806664067293904931):德卡拉先生的信箱(322610207) 06-13 19:11:08 [ERROR] haruka_bot | 截取动态时发生错误:https://m.bilibili.com/dynamic/806664067293904931 Traceback (most recent call last): File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in run_code exec(code, run_globals) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\Scripts\hb.exe_main.py", line 7, in sys.exit(main()) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in call return self.main(args, kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main rv = self.invoke(ctx) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1657, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke return ctx.invoke(self.callback, ctx.params) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke return callback(args, *kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\cli_init.py", line 16, in run run() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\cli\bot.py", line 35, in run nonebot.run(app="haruka_bot.cli.bot:app") File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\nonebot_init.py", line 309, in run get_driver().run(args, kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\nonebot\drivers\fastapi.py", line 198, in run uvicorn.run( File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\main.py", line 578, in run server.run() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\server.py", line 61, in run return asyncio.run(self.serve(sockets=sockets)) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run return loop.run_until_complete(main) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 633, in run_until_complete self.run_forever() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 321, in run_forever super().run_forever() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 600, in run_forever self._run_once() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1896, in _run_once handle._run() File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run self._context.run(self._callback, self._args) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\apscheduler\executors\base_py3.py", line 30, in run_coroutine_job retval = await job.func(job.args, job.kwargs) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\plugins\pusher\dynamic_pusher.py", line 72, in dy_sched image = await get_dynamic_screenshot(dynamic_id) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\utils\browser.py", line 38, in get_dynamic_screenshot return await get_dynamic_screenshot_mobile(dynamic_id)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\utils\browser.py", line 100, in get_dynamic_screenshot_mobile assert card AssertionError
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>
修改了UA之后,bot现在是在晚上经常会出现空图或者动态不存在的截图情况… 不知道是ip问题还是UA问题,如果是UA问题,求助一下各位有没有什么办法能够添加多个UA啊(?)谢谢!
自己解决了 换了个百度蜘蛛的UA() 用魔法对抗魔法
好像有效(?)再观察几天
---原始邮件--- 发件人: @.> 发送时间: 2023年6月18日(周日) 上午10:25 收件人: @.>; 抄送: @.**@.>; 主题: Re: [SK-415/HarukaBot] 【错误回报】动态推送出现验证码,且高频率复现 (Issue #366)
修改了UA之后,bot现在是在晚上经常会出现空图或者动态不存在的截图情况… 不知道是ip问题还是UA问题,如果是UA问题,求助一下各位有没有什么办法能够添加多个UA啊(?)谢谢!
自己解决了 换了个百度蜘蛛的UA() 用魔法对抗魔法
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: @.***>
确实,把移动端的UA改成Mozilla/5.0 (Linux;u;Android 4.2.2;zh-cn;) AppleWebKit/534.46 (KHTML,like Gecko)Version/5.1 Mobile Safari/10600.6.3 (compatible; Baiduspider/2.0;+http://www.baidu.com/search/spider.html) 可解
如果一定要用电脑端的话,或许可以添加:
if await page.query_selector(".geetest_panel"): return None
来避免截图,并在下一次爬取时重试。
加了这玩意好像有概率死循环
06-23 21:05:17 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:05:24 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:05:31 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:05:37 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:05:44 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:05:51 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:05:57 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:03 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:10 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:16 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:22 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:28 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:34 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
06-23 21:06:40 [INFO] haruka_bot | 检测到新动态(810386176411172886):原神(401742377)
今天前瞻,就一直这样
fix 分支修复了该问题
fix 分支修复了该问题
尝试了删除原文件并使用fix分支重新安装bot,仍然弹出验证码。不过此前该issue中所有解决方法都尝试过均未能解决问题
@SasoriXizixu
新分支理论上100%解决了 mobile 的截图验证码问题,你这个失败请发下日志,不然我给你抽张塔罗牌算算命?
fix 分支修复了该问题
尝试了删除原文件并使用fix分支重新安装bot,仍然弹出验证码。不过此前该issue中所有解决方法都尝试过均未能解决问题
另外,fix分支一直有新更新,可以再拉下新代码试试
@SasoriXizixu
新分支理论上100%解决了 mobile 的截图验证码问题,你这个失败请发下日志,不然我给你抽张塔罗牌算算命?
实在不好意思,以下是日志
06-28 12:15:21 [ERROR] haruka_bot | 截取动态时发生错误:https://m.bilibili.com/dynamic/812123812542808089
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\Scripts\hb.exe\__main__.py", line 7, in <module>
sys.exit(main())
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\click\core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\cli\__init__.py", line 16, in run
run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\cli\bot.py", line 35, in run
nonebot.run(app="haruka_bot.cli.bot:app")
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\nonebot\__init__.py", line 309, in run
get_driver().run(*args, **kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\nonebot\drivers\fastapi.py", line 198, in run
uvicorn.run(
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\main.py", line 578, in run
server.run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\uvicorn\server.py", line 61, in run
return asyncio.run(self.serve(sockets=sockets))
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\runners.py", line 44, in run
return loop.run_until_complete(main)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 633, in run_until_complete
self.run_forever()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\windows_events.py", line 321, in run_forever
super().run_forever()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 600, in run_forever
self._run_once()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\base_events.py", line 1896, in _run_once
handle._run()
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\asyncio\events.py", line 80, in _run
self._context.run(self._callback, *self._args)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\apscheduler\executors\base_py3.py", line 30, in run_coroutine_job
retval = await job.func(*job.args, **job.kwargs)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\plugins\pusher\dynamic_pusher.py", line 72, in dy_sched
image = await get_dynamic_screenshot(dynamic_id)
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\utils\browser.py", line 38, in get_dynamic_screenshot
return await get_dynamic_screenshot_mobile(dynamic_id)
> File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\haruka_bot\utils\browser.py", line 100, in get_dynamic_screenshot_mobile
assert card
AssertionError
运行环境
win server 2022
HarukaBot 版本
1.5.4
go-cqhttp 版本
1.0.1
错误描述
动态推送出现验证码
![Screenshot_2023-05-11-12-12-19-689_com tencent mobileqq](https://github.com/SK-415/HarukaBot/assets/100580891/8e2dd9c1-ffcb-4ea1-a815-61931188b1b4)
报错日志(如果有)