Evil0ctal / Douyin_TikTok_Download_API

🚀「Douyin_TikTok_Download_API」是一个开箱即用的高性能异步抖音、快手、TikTok、Bilibili数据爬取工具,支持API调用,在线批量解析及下载。
https://douyin.wtf
Apache License 2.0
9.54k stars 1.46k forks source link

[BUG] 爬取用户主页作品问题 #509

Open qingmou opened 1 week ago

qingmou commented 1 week ago

发生错误的平台?

抖音

发生错误的端点?

Web APP

提交的输入值?

如:短视频链接

是否有再次尝试?

如:是,发生错误后X时间后错误依旧存在。

你有查看本项目的自述文件或接口文档吗?

如:有,并且很确定该问题是程序导致的。

爬取用户主页视频,web api是:/api/douyin/web/fetch_user_post_videos 例如:https://www.douyin.com/user/MS4wLjABAAAA3dRAmxkZfpdvYENgqTA0ewccirDkNEM4kBbUfqetEh8 爬取到的视频数量比实际数量少了,我检查找到了问题所在,如下: 问题出现文件在Douyin_TikTok_Download_API-main\crawlers\douyin\web\models.py的类class BaseRequestModel(BaseModel)中, 将

class BaseRequestModel(BaseModel):
    device_platform: str = "webapp"
    aid: str = "6383"
    channel: str = "channel_pc_web"
    pc_client_type: int = 1
    version_code: str = "190500"
    version_name: str = "19.5.0"
    cookie_enabled: str = "true"
    screen_width: int = 1920
    screen_height: int = 1080
    browser_language: str = "zh-CN"
    browser_platform: str = "Win32"
    browser_name: str = "Firefox"
    browser_version: str = "124.0"
    browser_online: str = "true"
    engine_name: str = "Gecko"
    engine_version: str = "122.0.0.0"
    os_name: str = "Windows"
    os_version: str = "10"
    cpu_core_num: int = 12
    device_memory: int = 8
    platform: str = "PC"
    # webid: str = "7388296161008862738"
    # downlink: int = 10
    # effective_type: str = "4g"
    # round_trip_time: int = 100
    msToken: str = TokenManager.gen_real_msToken()

修改为

class BaseRequestModel(BaseModel):
    device_platform: str = "webapp"
    aid: str = "6383"
    channel: str = "channel_pc_web"
    pc_client_type: int = 1
    version_code: str = "290100"
    version_name: str = "29.1.0"
    cookie_enabled: str = "true"
    screen_width: int = 1920
    screen_height: int = 1080
    browser_language: str = "zh-CN"
    browser_platform: str = "Win32"
    browser_name: str = "Chrome"
    browser_version: str = "130.0.0.0"
    browser_online: str = "true"
    engine_name: str = "Blink"
    engine_version: str = "130.0.0.0"
    os_name: str = "Windows"
    os_version: str = "10"
    cpu_core_num: int = 12
    device_memory: int = 8
    platform: str = "PC"
    downlink: str = "10"
    effective_type: str = "4g"
    from_user_page: str = "1"
    locate_query: str = "false"
    need_time_list: str = "1"
    pc_libra_divert: str = "Windows"
    publish_video_strategy_type: str = "2"
    round_trip_time: str = "0"
    show_live_replay_strategy: str = "1"
    time_list_query: str = "0"
    whale_cut_token: str = ""
    update_version_code: str = "170400"
    msToken: str = TokenManager.gen_real_msToken()

修改前获取到80+,修改后获取到110+,这样修改解决了问题。 以上在电脑测试。

Evil0ctal commented 1 week ago

嗯嗯,了解,我晚点去更新一下这个,感谢反馈!