JoeanAmier / TikTokDownloader

TikTok 主页/合辑/直播/视频/图集/原声;抖音主页/视频/图集/收藏/直播/原声/合集/评论/账号/搜索/热榜数据采集工具
https://github.com/JoeanAmier/TikTokDownloader/wiki/Documentation
GNU General Public License v3.0
7.15k stars 1.16k forks source link

[BUG] UnicodeDecodeError: 'utf-8' codec can't decode byte ... in position 3: invalid start byte #245

Open Johnserf-Seed opened 1 month ago

Johnserf-Seed commented 1 month ago

描述: 在调用__request_data_get方法时,在默认的DATA_HEADERS配置下,该方法尝试使用response.json()解码数据,导致UnicodeDecodeError。此错误回溯表明,问题是由尝试解码的接口数据不符合UTF-8规范的JSON数据引起的。

复现步骤:

预期行为: 期望代码能够优雅地处理非UTF-8编码的数据,并在无法解码为JSON时提供合适的处理方式,如返回原始字节数据或者其他适当的格式。同时,应记录详细的错误日志以便调试和排查问题。

实际行为: 当尝试JSON解码不符合UTF-8规范的字节数据时,出现UnicodeDecodeError

错误回溯:

Error Traceback ```bash Traceback (most recent call last): File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\main.py", line 11, in run(main()) File "C:\Users\WDAGUtilityAccount\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 194, in run return runner.run(main) ^^^^^^^^^^^^^^^^ File "C:\Users\WDAGUtilityAccount\AppData\Local\Programs\Python\Python312\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\WDAGUtilityAccount\AppData\Local\Programs\Python\Python312\Lib\asyncio\base_events.py", line 687, in run_until_complete return future.result() ^^^^^^^^^^^^^^^ File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\main.py", line 7, in main await downloader.run() File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\TikTokDownloader.py", line 328, in run await self.main_menu(safe_pop(self.default_mode)) File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\TikTokDownloader.py", line 212, in main_menu await self.compatible(default_mode) File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\TikTokDownloader.py", line 296, in compatible await self.__function_menu[n][1]() File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\TikTokDownloader.py", line 220, in complete await example.run(self.default_mode) await self.__function[n][1](safe_pop(self.default_mode)) File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\main_complete.py", line 534, in detail_interactive await self.__detail_secondary_menu(self.__function_detail, select) File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\main_complete.py", line 555, in __detail_secondary_menu await menu[n][1](record) File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\main_complete.py", line 564, in __detail_inquire await self.input_links_acquisition(tiktok, ids, record) File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\application\main_complete.py", line 605, in input_links_acquisition await Detail( File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\interface\detail.py", line 45, in run return await super().run( ^^^^^^^^^^^^^^^^^^ File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\interface\template.py", line 119, in run await self.run_single( File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\interface\template.py", line 160, in run_single if data := await self.request_data( ^^^^^^^^^^^^^^^^^^^^^^^^ File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\interface\template.py", line 250, in request_data return await self.__request_data_get(url, params, headers or self.headers, ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "c:\Users\WDAGUtilityAccount\Desktop\TikTokDownloader-master\src\interface\template.py", line 314, in __request_data_get return response.json() ^^^^^^^^^^^^^^^ File "C:\Users\WDAGUtilityAccount\AppData\Local\Programs\Python\Python312\Lib\site-packages\httpx\_models.py", line 764, in json return jsonlib.loads(self.content, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\WDAGUtilityAccount\AppData\Local\Programs\Python\Python312\Lib\json\__init__.py", line 341, in loads s = s.decode(detect_encoding(s), 'surrogatepass') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa2 in position 3: invalid start byte ```

修复方法: 为解决这个问题,请修改src/custom/internal.py中的DATA_HEADERS,将AcceptAccept-Encoding头部修改为*/*

# 修改src/custom/internal.py中的头部
DATA_HEADERS = {
    'Accept': '*/*',
    'Accept-Encoding': '*/*',
    # 其他需要的头部
}

这个修改确保服务器以可正确解码的数据响应,而无需假设特定的编码方式。