[BiliBiliSearch] HTTPError 412: 'Precondition Failed'

lean-aa commented 2 years ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm reporting a bug unrelated to a specific site
[X] I've verified that I'm running yt-dlp version 2022.09.01 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue

Provide a description that is worded well enough to be understood

download from www.bilibili.com url is ok but download via bilisearch produce error

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

K:\temp>yt-dlp -vU --format "bv*+ba/b" --max-filesize 125m  --yes-playlist  "bilisearch12:毛不易"
[debug] Command-line config: ['-vU', '--format', 'bv*+ba/b', '--max-filesize', '125m', '--yes-playlist', 'bilisearch12: 毛不易']
[debug] Encodings: locale cp936, fs utf-8, pref cp936, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2022.09.01 [5d7c7d6] (win32_exe)
[debug] Python 3.8.10 (CPython 64bit) - Windows-10-10.0.19044-SP0
[debug] Checking exe version: ffmpeg -bsfs
[debug] Checking exe version: ffprobe -bsfs
[debug] exe versions: ffmpeg 2021-10-11-git-90a0da9f14-full_build-www.gyan.dev (setts), ffprobe 2021-10-11-git-90a0da9f14-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.06.15, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1670 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2022.09.01, Current version: 2022.09.01
yt-dlp is up to date (2022.09.01)
[debug] [BiliBiliSearch] Extracting URL: bilisearch12:毛不易
[download] Downloading playlist: 毛不易
[BiliBiliSearch] 毛不易: Extracting results from page 1
ERROR: Unable to download JSON metadata: HTTP Error 412: Precondition Failed (caused by <HTTPError 412: 'Precondition Failed'>); please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "yt_dlp\extractor\common.py", line 803, in _request_webpage
  File "yt_dlp\YoutubeDL.py", line 3656, in urlopen
  File "urllib\request.py", line 531, in open
  File "urllib\request.py", line 640, in http_response
  File "urllib\request.py", line 569, in error
  File "urllib\request.py", line 502, in _call_chain
  File "urllib\request.py", line 649, in http_error_default
urllib.error.HTTPError: HTTP Error 412: Precondition Failed

[BiliBiliSearch] Playlist 毛不易: Downloading 1 videos of 0
[download] Finished downloading playlist: 毛不易

c-basalt commented 1 year ago

HTTP 412 is the status code for reaching API rate limit on bilibili, and all API requests from the same IP address will be restricted for a certain amount of time.

hanxi commented 8 months ago

I try open this url in browser, It's can get right json.

https://api.bilibili.com/x/web-interface/search/type?context=&page=1&order=pubdate&keyword=%E5%91%A8%E6%9D%B0%E4%BC%A6&duration=0&tids_2=&__refresh__=true&search_type=video&tids=0&highlight=1

Need view https://www.bilibili.com/ to get cookie first.

curl 'https://api.bilibili.com/x/web-interface/search/type?context=&page=1&order=pubdate&keyword=%E5%91%A8%E6%9D%B0%E4%BC%A6&duration=0&tids_2=&__refresh__=true&search_type=video&tids=0&highlight=1' \
        -H $'cookie: buvid3=B316A51A-F640-D845-A87E-3AE6CCD098EE81984infoc; b_nut=1681592281; CURRENT_FNVAL=4048; _uuid=5C656C26-8135-BA96-9A9D-2B92810510DEB581243infoc; CURRENT_PID=35a6d580-dbd0-11ed-9490-9f9747584641; rpdid=|(um|Juulmk|0J\'uY)ulmkk)R; buvid4=7D238122-1DEE-A424-E4FD-01116A0EFB7F83076-023041604-Eg1Sg8zdHfupW32s9pBw%2Fw%3D%3D; fingerprint=34442aa86c5038f096a4281bb2c27db2; buvid_fp_plain=undefined; buvid_fp=4f6e181b5d54774c360e7cc8ece49e55; b_lsid=2DE29B4E_18D6502742B; bili_ticket=eyJhbGciOiJIUzI1NiIsImtpZCI6InMwMyIsInR5cCI6IkpXVCJ9.eyJleHAiOjE3MDcwNTU4ODAsImlhdCI6MTcwNjc5NjYyMCwicGx0IjotMX0.qgoRZHqpSsy0CVapMoANrc0Gt9Zop2ruJNcWgVTm0qU; bili_ticket_expires=1707055820; enable_web_push=DISABLE; header_theme_version=CLOSE; home_feed_column=5; browser_resolution=2560-918; bp_video_offset_2054571714=893171288527339541; sid=nvguvmnh'

c-basalt commented 8 months ago

I see. Adding a buvid3 cookie should fix this problem.

hanxi commented 8 months ago

I solved this problem by using --cookies parameter, according to this document https://github.com/yt-dlp/yt-dlp/wiki/FAQ#how-do-i-pass-cookies-to-yt-dlp Get cookies.txt file.

yt-dlp "bilisearch:周杰伦" --cookies cookies.txt

cookies.txt file like this

# Netscape HTTP Cookie File
# This file is generated by yt-dlp.  Do not edit.

.bilibili.com   TRUE    /       FALSE   1738333501      _uuid   13416438-8579-9913-91AE-34EFA613959509732infoc
.bilibili.com   TRUE    /       FALSE   0       b_lsid  FD0CEF910_18D67227BCC
.bilibili.com   TRUE    /       FALSE   1741357501      b_nut   1706797501
.bilibili.com   TRUE    /       FALSE   1738368491      browser_resolution      1091-918
.bilibili.com   TRUE    /       FALSE   1741357501      buvid3  101103CF-598D-69AD-C0A7-86ACD2E8BB5C08091infoc
.bilibili.com   TRUE    /       FALSE   1741357501      buvid4  BBC106B7-ED7D-615F-442C-F048FDD096B908091-024020114-ANawzDfwerU2QorLcdfITg%3D%3D
.bilibili.com   TRUE    /       FALSE   1741357511      buvid_fp        dd139ca05df9d1b8cd13ef76011d9aa4
.bilibili.com   TRUE    /       FALSE   1738368491      enable_web_push DISABLE
.bilibili.com   TRUE    /       FALSE   1738368491      header_theme_version    CLOSE
.bilibili.com   TRUE    /       FALSE   1738368491      home_feed_column        4
.bilibili.com   TRUE    /       FALSE   0       sid     1dh94bnf
www.bilibili.com        FALSE   /       FALSE   0       bmg_af_switch   1
www.bilibili.com        FALSE   /       FALSE   0       bmg_src_def_domain      i1.hdslb.com

hanxi commented 8 months ago

I see. Adding a buvid3 cookie should fix this problem.

You are right. Only need buvid3 cookie.

yt-dlp / yt-dlp