[BiliIntl] HTTP Error 403: Forbidden - Bilibili.tv unable to download audio

nothing2obvi commented 11 months ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I intentionally remove or skip any mandatory* field

Checklist

[X] I'm reporting that yt-dlp is broken on a supported site
[X] I've verified that I'm running yt-dlp version 2023.10.13 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region

South Korea

Provide a description that is worded well enough to be understood

I get an error when I try to download from Bilibili.tv.

ERROR: [BiliIntl] None: Unable to extract JSON-LD; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U

I do not have the latest (2023.10.13) version of yt-dlp, but I have the latest that is available when I use pip3 to install, which is stable@2023.03.04 [392389b7d] (pip).

I am located in the US but am using a VPN for South Korea to access the video.

Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] If using API, add 'verbose': True to YoutubeDL params instead
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output

[debug] Command-line config: ['-P', '/Volumes/Venti', '--cookies-from-browser', 'opera', '--embed-subs', '-vU', 'https://www.bilibili.tv/en/video/2049276024']
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.03.04 [392389b7d] (pip)
[debug] Python 3.10.0 (CPython x86_64 64bit) - macOS-12.6-x86_64-i386-64bit (OpenSSL 1.1.1l  24 Aug 2021)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[Cookies] Extracting cookies from opera
[debug] Extracting cookies from: "/Users/NAME/Library/Application Support/com.operasoftware.Opera/Cookies"
[debug] using find-generic-password to obtain password from OSX keychain
[Cookies] Extracted 1533 cookies from opera
[debug] cookie version breakdown: {'v10': 1518, 'other': 0, 'unencrypted': 15}
[debug] Proxy map: {}
[debug] Loaded 1786 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.13, Current version: stable@2023.03.04
[debug] Downloading _update_spec from https://github.com/yt-dlp/yt-dlp/releases/latest/download/_update_spec
ERROR: You installed yt-dlp with pip or using the wheel from PyPi; Use that to update
[BiliIntl] Extracting URL: https://www.bilibili.tv/en/video/2049276024
[BiliIntl] 2049276024: Downloading webpage
ERROR: [BiliIntl] None: Unable to extract JSON-LD; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 694, in extract
    ie_result = self._real_extract(url)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yt_dlp/extractor/bilibili.py", line 1087, in _real_extract
    **self._extract_video_metadata(url, video_id, season_id),
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yt_dlp/extractor/bilibili.py", line 1058, in _extract_video_metadata
    self._parse_video_metadata(video_data), self._search_json_ld(webpage, video_id), {
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 1481, in _search_json_ld
    raise RegexNotFoundError('Unable to extract JSON-LD')

bashonly commented 11 months ago

I do not have the latest (2023.10.13) version of yt-dlp,

Then this is not a valid bug report. Get the latest version

nothing2obvi commented 11 months ago

@bashonly

Okay, just tried it with the latest macOS binary. yt-dlp version stable@2023.10.13 [b634ba742]

My command is as follows, while using a VPN to appear to be in South Korea:

/Users/Name/yt-dlp_macos -P '/Volumes/Hard-Drive' --cookies-from-browser 'opera' -vU 'https://www.bilibili.tv/en/video/2049276024'

The video file (only) downloads, but then I get the following error. The video has no audio. (It has audio when I play it on the website.)

[debug] Command-line config: ['-P', '/Volumes/Hard-Drive', '--cookies-from-browser', 'opera', '-vU', 'https://www.bilibili.tv/en/video/2049276024']
Extracting cookies from opera
[debug] Extracting cookies from: "/Users/Name/Library/Application Support/com.operasoftware.Opera/Cookies"
[debug] using find-generic-password to obtain password from OSX keychain
Extracted 1551 cookies from opera
[debug] cookie version breakdown: {'v10': 1536, 'other': 0, 'unencrypted': 15}
[debug] Encodings: locale utf-8, fs utf-8, pref utf-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2023.10.13 [b634ba742] (darwin_exe)
[debug] Python 3.11.5 (CPython x86_64 64bit) - macOS-12.6-x86_64-i386-64bit (OpenSSL 3.0.10 1 Aug 2023)
[debug] exe versions: ffmpeg 6.0 (setts), ffprobe 6.0, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.19.0, brotli-1.1.0, certifi-2023.07.22, mutagen-1.47.0, sqlite3-3.42.0, websockets-11.0.3
[debug] Proxy map: {}
[debug] Loaded 1890 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Available version: stable@2023.10.13, Current version: stable@2023.10.13
Current Build Hash: 5d538669b078c77e6fc8ff914686079dfca4226113192cfc16f8643161a49046
yt-dlp is up to date (stable@2023.10.13)
[BiliIntl] Extracting URL: https://www.bilibili.tv/en/video/2049276024
[BiliIntl] 2049276024: Downloading webpage
WARNING: [BiliIntl] unable to extract JSON-LD ; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliIntl] unable to extract og:title; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliIntl] unable to extract og:description; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[BiliIntl] 2049276024: Downloading video formats
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, source, id
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 2049276024: Downloading 1 format(s): 12+2
[debug] Invoking http downloader on "https://upos-bstar1-mirrorakam.akamaized.net/iupxcodeboss/m9/xm/n220727ad29l10gz6lrey43ez1m5xmm9-1-141220110000.m4s?e=ig8euxZM2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1699085570&gen=playurlv2&os=akam&oi=3230589849&trid=be97eb4134e94c73a4629b70c714168ei&mid=1320796813&platform=pc&upsig=c45360249f4dfcc81d4a7c7df8d27a8e&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&hdnts=exp=1699085570~hmac=86e166032105906aaffd7f48357546869b4fe4d51db3f4efec6d0f58ca1ec60e&bvc=vod&nettype=0&orderid=0,1&logo=00000000&f=i_0_0"
[download] Destination: /Volumes/Venti/Goblin Special Episode： The Summoning PART 1 [2049276024].f12.mp4
[download] 100% of  436.42MiB in 00:01:10 at 6.19MiB/s
[debug] Invoking http downloader on "https://upos-bstar1-mirrorakam.akamaized.net/iupxcodeboss/m9/xm/n220727ad29l10gz6lrey43ez1m5xmm9-1-1d1301000022.m4s?e=ig8euxZM2rNcNbdlhoNvNC8BqJIzNbfqXBvEqxTEto8BTrNvN0GvT90W5JZMkX_YN0MvXg8gNEV4NC8xNEV4N03eN0B5tZlqNxTEto8BTrNvNeZVuJ10Kj_g2UB02J0mN0B5tZlqNCNEto8BTrNvNC7MTX502C8f2jmMQJ6mqF2fka1mqx6gqj0eN0B599M=&uipk=5&nbs=1&deadline=1699085570&gen=playurlv2&os=akam&oi=3230589849&trid=838285d2d12447fd8bb74f6f13643dfai&mid=1320796813&platform=pc&upsig=4e0d786f1b4e344045db486b036cab81&uparams=e,uipk,nbs,deadline,gen,os,oi,trid,mid,platform&hdnts=exp=1699085570~hmac=3342cb74bb3369ddd7c79eb4eef0da5315a9f5f98d42255091845bbeb9c14b03&bvc=vod&nettype=0&orderid=0,1&logo=00000000&f=i_0_0"
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "yt_dlp/networking/_urllib.py", line 410, in _send
  File "urllib/request.py", line 525, in open
  File "urllib/request.py", line 634, in http_response
  File "urllib/request.py", line 563, in error
  File "urllib/request.py", line 496, in _call_chain
  File "urllib/request.py", line 643, in http_error_default
urllib.error.HTTPError: HTTP Error 403: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 4051, in urlopen
  File "yt_dlp/networking/common.py", line 114, in send
  File "yt_dlp/networking/_helper.py", line 204, in wrapper
  File "yt_dlp/networking/common.py", line 325, in send
  File "yt_dlp/networking/_urllib.py", line 415, in _send
yt_dlp.networking.exceptions.HTTPError: HTTP Error 403: Forbidden

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 3369, in process_info
  File "yt_dlp/YoutubeDL.py", line 3108, in dl
  File "yt_dlp/downloader/common.py", line 455, in download
  File "yt_dlp/downloader/http.py", line 364, in real_download
  File "yt_dlp/downloader/http.py", line 120, in establish_connection
  File "yt_dlp/YoutubeDL.py", line 4070, in urlopen
yt_dlp.networking.exceptions._CompatHTTPError: HTTP Error 403: Forbidden

SirElderling commented 11 months ago

The 403 Forbidden is returned when yt-dlp attempts to download the audio part. Despite the warning, the issue has nothing to do with the JSON-LD related calls.

bashonly commented 11 months ago

see if adding a referer header helps, as suggested in this comment: https://github.com/yt-dlp/yt-dlp/issues/6640#issuecomment-1495394644

SirElderling commented 11 months ago

I'm a bit confused.

Until one hour ago, the best audio format download was failing with the 403 Forbidden error (the worst audio format always worked), but now it seems to work just fine without any changes.

To confirm, I tested from a different location (remote server) without additional parameters (ex. the suggested Referer header) and it still works...

Maybe someone else can test?

Note:

There is no need to add browser cookies - the video/audio is downloaded with the best quality out of the box.
I'm not sure if the this specific content is geo-blocked. I can extract it from a portuguese and german IP

bashonly commented 11 months ago

I am geo blocked

nothing2obvi commented 11 months ago

Weird, it suddenly worked for me, without the add-header option. Thank you both for your help!

darkralts commented 11 months ago

$ yt-dlp https://www.bilibili.tv/en/video/2046079257
[BiliIntl] Extracting URL: https://www.bilibili.tv/en/video/2046079257
[BiliIntl] 2046079257: Downloading webpage
WARNING: [BiliIntl] unable to extract JSON-LD ; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliIntl] unable to extract og:title; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliIntl] unable to extract og:description; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[BiliIntl] 2046079257: Downloading video formats
[info] 2046079257: Downloading 1 format(s): 12+2
ERROR: unable to download video data: HTTP Error 403: Forbidden

it still errors for me

decade7117 commented 10 months ago

$ yt-dlp https://www.bilibili.tv/en/video/2046079257
[BiliIntl] Extracting URL: https://www.bilibili.tv/en/video/2046079257
[BiliIntl] 2046079257: Downloading webpage
WARNING: [BiliIntl] unable to extract JSON-LD ; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliIntl] unable to extract og:title; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
WARNING: [BiliIntl] unable to extract og:description; please report this issue on  https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using  yt-dlp -U
[BiliIntl] 2046079257: Downloading video formats
[info] 2046079257: Downloading 1 format(s): 12+2
ERROR: unable to download video data: HTTP Error 403: Forbidden

it still errors for me

can you help me from start

yt-dlp / yt-dlp