vicety / Pixiv-Crawler

Scrapy框架下的pixiv多功能爬虫
GNU General Public License v3.0
48 stars 12 forks source link

指定画师时无法爬取 #8

Closed llll1004 closed 6 years ago

llll1004 commented 6 years ago

1 start urls login successfully found result(s) 2018-11-20 14:42:16 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.pixiv.net/member_illust.php?id=3967264> (referer: https://accounts.pixiv.net/login?lang=zh&source=pc&view_type=page&ref=wwwtop_accounts_index) Traceback (most recent call last): File "D:\Python\lib\site-packages\scrapy\utils\defer.py", line 102, in iter_errback yield next(it) File "D:\Python\lib\site-packages\scrapy\spidermiddlewares\offsite.py", line 30, in process_spider_output for x in result: File "D:\Python\lib\site-packages\scrapy\spidermiddlewares\referer.py", line 339, in return (_set_referer(r) for r in result or ()) File "D:\Python\lib\site-packages\scrapy\spidermiddlewares\urllength.py", line 37, in return (r for r in result or () if _filter(r)) File "D:\Python\lib\site-packages\scrapy\spidermiddlewares\depth.py", line 58, in return (r for r in result or () if _filter(r)) File "D:\Pixiv-Crawler-master\pixiv_beta\spiders\pixiv-beta.py", line 176, in artist "Artist: {0}".format(response.css("._user-profile-card .profile a.user-name::text").extract_first('unknown'))) File "D:\Pixiv-Crawler-master\pixiv_beta\spiders\pixiv-beta.py", line 302, in update_process raise UnmatchError("collection_num not matched") pixiv_beta.utils.PixivError.UnmatchError: collection_num not matched Crawling complete, got 0 data

vicety commented 6 years ago

P站又改网页了……这次的类名做了混淆处理 image

改起来有点麻烦,短期内是没有空更新了,抱歉

llll1004 commented 6 years ago

好的,辛苦了~