请求参数异常，请升级客户端后重试!

ZhiWeiCui commented 3 years ago

读取 Cookies 文件登录成功 3 proxies have been fetched 3 proxies have been fetched Traceback (most recent call last): File "F:/My/Git/ZhihuCrawler/zhihu_spider.py", line 35, in begin_crawl_usr crawl_usr(crawler, data[0], data[1][0], data[1][1]) File "F:/My/Git/ZhihuCrawler/zhihu_spider.py", line 62, in crawl_usr for data in json['data']: KeyError: 'data' retries exceeded json : {'error': {'code': 10002, 'message': '10002:请求参数异常，请升级客户端后重试'}}

MogicianXD commented 3 years ago

请确认是否安装了jsdom

npm install jsdom

ZhiWeiCui commented 3 years ago

已经安装了，如果不安装会直接提示缺少jsdom模块，而且已经登录成功了。安装过以后在zhihu_login.py中是这样的配置：

@staticmethod
def _encrypt(form_data: dict):
    with open('./encrypt.js') as f:
        js = execjs.compile(f.read(), cwd=r'C:\Users\Administrator\AppData\Roaming\npm\node_modules')
        return js.call('b', urlencode(form_data))

wait表中插入的初始数据是这样的：

INSERT INTO wait(uid, utoken, url) VALUE ("1da75b85900e00adb072e91c56fd9149", "xiaoxiaodouzi", "https://www.zhihu.com/api/v4/people/xiaoxiaodouzi")

找不到原因，不知道是知乎更新了反爬虫策略，还是我使用的姿势不对。

MogicianXD commented 3 years ago

应该是知乎更新了反爬虫机制

MogicianXD commented 3 years ago

要么不使用api了，改成爬取整个页面，解析dom树，从元素里抽取数据。这个项目我不再维护了，您有时间的话可以fork后修改。

MogicianXD / ZhihuCrawler

请求参数异常，请升级客户端后重试! #2