bug - Githubissues

SharkPika commented 7 months ago

非常好的爬虫但是4年后再用还是会有bug unexpected end of stream on null重新尝试获取 https://pixiv.net/ajax/search/artworks/genshin?word=genshin&mode=r18&p=1&type=all&lang=zh&s_mode=s_type unexpected end of stream on null重新尝试获取 https://pixiv.net/ajax/search/artworks/genshin?word=genshin&mode=r18&p=1&type=all&lang=zh&s_mode=s_type unexpected end of stream on null重新尝试获取 https://pixiv.net/ajax/search/artworks/genshin?word=genshin&mode=r18&p=1&type=all&lang=zh&s_mode=s_type unexpected end of stream on null重新尝试获取 https://pixiv.net/ajax/search/artworks/genshin?word=genshin&mode=r18&p=1&type=all&lang=zh&s_mode=s_type

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffcb9a1bf00, pid=31196, tid=25208
#
# JRE version: Java(TM) SE Runtime Environment (17.0.2+8) (build 17.0.2+8-LTS-86)
# Java VM: Java HotSpot(TM) 64-Bit Server VM (17.0.2+8-LTS-86, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# V  [jvm.dll+0xbf00]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# D:\QQ\1127367472\FileRecv\Source\PixivSpider-master\build\libs\hs_err_pid31196.log
#
# If you would like to submit a bug report, please visit:
#   https://bugreport.java.com/bugreport/crash.jsp
#

jht3QAQ commented 7 months ago

这仓库都四年没维护了2333 如果让我维护我可能会选择用python重写(逃)

SharkPika commented 7 months ago

其实我想学学java怎么做爬虫才搜到你这个仓库的

jht3QAQ commented 6 months ago

润python吧代码量起码能少一半如果单论爬虫的话无非就是生成请求然后去下载罢了生成请求获取图片列表->遍历图片列表去下载翻翻我这里源代码能找到我之前找的一些P站的API 像下面这些之类的

https://www.pixiv.net/ajax/search/artworks/{kw}?word={kw}&mode=safe&p=1&type=all&&lang=zh&s_mode=s_tag
https://www.pixiv.net/ajax/user/{uid}/profile/all?lang=zh
https://www.pixiv.net/ajax/user/{uid}/profile/illusts?ids[]={ids}work_category=illustManga&is_first_page=1&lang=zh
https://www.pixiv.net/ajax/illust/{pid}/pages?lang=zh

如果你想做p站的爬虫应该能用的上这些api返回的都是json 解析一下就能找到自己想要的数据具体点下面的链接就能懂了

https://www.pixiv.net/ajax/search/artworks/genshin?word=genshin&mode=r18&p=1&type=all&lang=zh&s_mode=s_type
https://www.pixiv.net/ajax/user/52542337/profile/all?lang=zh
https://www.pixiv.net/ajax/user/52542337/profile/illusts?ids[]=118201101&ids[]=117777980&work_category=illustManga&is_first_page=1&lang=zh
https://www.pixiv.net/ajax/illust/118201101/pages?lang=zh

就这么构造请求去获取json 获取根据画师/关键字获取图片列表再根据图片列表获取图片的url 然后下载就好

jht3QAQ / PixivSpider

bug #1