hect0x7 / JMComic-Crawler-Python

Python API for JMComic | 提供Python API访问禁漫天堂,同时支持网页端和移动端 | 禁漫天堂GitHub Actions下载器🚀
https://jmcomic.readthedocs.io/zh-cn/latest/option_file_syntax/#
MIT License
791 stars 1.85k forks source link

一个关于无效Json的Bug #196

Closed Yunxi-awa closed 8 months ago

Yunxi-awa commented 8 months ago

以下是报错信息,复现起来可能有点困难,压测了6个小时发现的Bug(另外怀疑Html端也有相似问题,但我这边网络很迷,没法测试 可以在解析JSON时判断行是否有效,建议在option设置一个选项能控制遇到此类错误是直接抛异常还是继续执行

2024-01-15 20:52:11:【dler.exception】superDownloader Exit with exception: (<class 'json.decoder.JSONDecodeError'>, JSONDecodeError('Expecting value: line 2 column 1 (char 1)'))
Exception in thread Thread-4:
Traceback (most recent call last):
  File "E:\Python\3.11.6\Lib\threading.py", line 1045, in _bootstrap_inner
    self.run()
  File "E:\PythonProject\JMDownload\new\apiGet.py", line 127, in run
    jmcomic.download_album(aid, getOption, superDownloader)
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\api.py", line 48, in download_album
    dler.download_album(jm_album_id)
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_downloader.py", line 58, in download_album
    album = client.get_album_detail(album_id)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_impl.py", line 607, in get_album_detail
    return self.fetch_detail_entity(album_id,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_impl.py", line 174, in cache_wrapper
    result = func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_impl.py", line 648, in fetch_detail_entity
    resp = self.req_api(
           ^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_impl.py", line 854, in req_api
    self.require_resp_success(resp, url)
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_impl.py", line 890, in require_resp_success
    resp.require_success()
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_interface.py", line 43, in require_success
    if self.is_not_success:
       ^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_interface.py", line 24, in is_not_success
    return not self.is_success
               ^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_interface.py", line 103, in is_success
    return super().is_success and self.json()['code'] == 200
                                  ^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\common\util\decorator_util.py", line 63, in func_exec
    attr = func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\jmcomic\jm_client_interface.py", line 89, in json
    return self.resp.json()
           ^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\site-packages\curl_cffi\requests\models.py", line 129, in json
    return loads(self.content, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "E:\Python\3.11.6\Lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 2 column 1 (char 1)
hect0x7 commented 8 months ago

这个我有遇到过,大概率是禁漫的Could not connect to mysql! Please check your database settings!

Yunxi-awa commented 8 months ago

这个我有遇到过,大概率是禁漫的Could not connect to mysql! Please check your database settings!

大佬知道有曲线救国的方法吗

hect0x7 commented 8 months ago

这个我有遇到过,大概率是禁漫的Could not connect to mysql! Please check your database settings!

大佬知道有曲线救国的方法吗

遇到这个就说明禁漫服务器内部异常,这次请求失败了。 可以优化一下,当遇到这种情况时,走重试机制。

hect0x7 commented 8 months ago

下个版本会把这种情况加入重试处理

hect0x7 commented 8 months ago

v2.5.1: 已增加对json格式返回值检查。 如果发现返回值应该是json格式但不是,则会强制抛出异常,该异常会被请求重试机制catch到,从而走请求重试的逻辑。 为了减少性能开销,只对少数情况做了简单检查。外界可以扩展此检查,参考如下代码:

from jmcomic import *

# 自定义Client
class MyApiClient(JmApiClient):
    def raise_if_resp_should_retry(self, resp):
        """
        在这个方法对resp响应对象进行检查,
        如果觉得不满意想重试,可以抛出异常
        如果满意,返回resp
        """
        return super().raise_if_resp_should_retry(resp)

JmModuleConfig.register_client(MyApiClient)

create_option(...)