yt-dlp / yt-dlp

A feature-rich command-line audio/video downloader
https://discord.gg/H5MNcFW63r
The Unlicense
90.93k stars 7.07k forks source link

[Youtube] 8k VR180 SBS videos no longer available #9903

Open R-S1978 opened 6 months ago

R-S1978 commented 6 months ago

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

Checklist

Region

germany

Provide a description that is worded well enough to be understood

New 8k VR180 SBS videos on YouTube are not longer downloadable, because the 8k format is not seen in the API JSON by yt-dlp as available format. This is since some weeks. The 8k format (2x4k SBS) is working for this video with YouTube App on Meta Quest 3, so it must be definitly there. But yt-dlp sees only the 4k (2x2k SBS) format. The 8k format is hidden somewhere.

Provide verbose output that clearly demonstrates the problem

Complete Verbose Output

[debug] Command-line config: ['--list-formats', 'https://www.youtube.com/watch?v=W51jdIrSsgk', '--extractor-args', 'youtube:player_client=all', '-vU']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version nightly@2024.05.08.232715 from yt-dlp/yt-dlp-nightly-builds [6b54cccdc] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.3.2-2021-02-02-full_build-www.gyan.dev, ffprobe 4.3.2-2021-02-02-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.31.0, sqlite3-3.35.5, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1810 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest
Latest version: nightly@2024.05.08.232715 from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@2024.05.08.232715 from yt-dlp/yt-dlp-nightly-builds)
[youtube] Extracting URL: https://www.youtube.com/watch?v=W51jdIrSsgk
[youtube] W51jdIrSsgk: Downloading webpage
[youtube] W51jdIrSsgk: Downloading ios player API JSON
[youtube] W51jdIrSsgk: Downloading ios embedded player API JSON
WARNING: [youtube] YouTube said: ERROR - Requested entity was not found.
WARNING: [youtube] HTTP Error 404: Not Found. Retrying (1/3)...
[youtube] W51jdIrSsgk: Downloading ios embedded player API JSON
WARNING: [youtube] YouTube said: ERROR - Requested entity was not found.
WARNING: [youtube] HTTP Error 404: Not Found. Retrying (2/3)...
[youtube] W51jdIrSsgk: Downloading ios embedded player API JSON
WARNING: [youtube] YouTube said: ERROR - Requested entity was not found.
WARNING: [youtube] HTTP Error 404: Not Found. Retrying (3/3)...
[youtube] W51jdIrSsgk: Downloading ios embedded player API JSON
WARNING: [youtube] YouTube said: ERROR - Requested entity was not found.
WARNING: [youtube] Unable to download API page: HTTP Error 404: Not Found (caused by <HTTPError 404: Not Found>)
[youtube] W51jdIrSsgk: Downloading ios music player API JSON
[youtube] W51jdIrSsgk: Downloading ios creator player API JSON
[youtube] W51jdIrSsgk: Downloading ios embedscreen player API JSON
[youtube] W51jdIrSsgk: Downloading android player API JSON
[youtube] W51jdIrSsgk: Downloading android embedded player API JSON
[youtube] W51jdIrSsgk: Downloading android music player API JSON
[youtube] W51jdIrSsgk: Downloading android creator player API JSON
[youtube] W51jdIrSsgk: Downloading android embedscreen player API JSON
[youtube] W51jdIrSsgk: Downloading web embedded client config
[youtube] W51jdIrSsgk: Downloading player 17fd9675
[youtube] W51jdIrSsgk: Downloading web embedded player API JSON
[youtube] W51jdIrSsgk: Downloading web music client config
[youtube] W51jdIrSsgk: Downloading web music player API JSON
[youtube] W51jdIrSsgk: Downloading web creator player API JSON
[youtube] W51jdIrSsgk: Downloading web embedscreen player API JSON
[youtube] W51jdIrSsgk: Downloading tv embedded player API JSON
[youtube] W51jdIrSsgk: Downloading mweb player API JSON
[youtube] W51jdIrSsgk: Downloading mweb embedscreen player API JSON
WARNING: [youtube] Skipping player responses from android/android_embedscreen clients (got player responses for video "aQvGIIdgFDM" instead of "W51jdIrSsgk")
[debug] Loading youtube-nsig.17fd9675 from cache
[debug] [youtube] Decrypted nsig 3FIf0FeDT3SuYx => -10eO2qq9RNVBA
[debug] Loading youtube-nsig.17fd9675 from cache
[debug] [youtube] Decrypted nsig 8J-KKQ4BRN590f => 2OT4zU8GM9RboA
[youtube] W51jdIrSsgk: Downloading m3u8 information
[youtube] W51jdIrSsgk: Downloading m3u8 information
[debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto
[debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id
[info] Available formats for W51jdIrSsgk:
ID      EXT   RESOLUTION FPS CH │   FILESIZE    TBR PROTO │ VCODEC           VBR ACODEC      ABR ASR MORE INFO
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
sb3     mhtml 48x27        1    │                   mhtml │ images                                   storyboard
sb2     mhtml 79x45        1    │                   mhtml │ images                                   storyboard
sb1     mhtml 159x90       1    │                   mhtml │ images                                   storyboard
sb0     mhtml 319x180      1    │                   mhtml │ images                                   storyboard
233     mp4   audio only        │                   m3u8  │ audio only           unknown             Default, IOS
234     mp4   audio only        │                   m3u8  │ audio only           unknown             Default, IOS
599     m4a   audio only      2 │  619.48KiB    31k https │ audio only           mp4a.40.5   31k 22k ultralow, MWEB, m4a_dash
600     webm  audio only      2 │  584.82KiB    29k https │ audio only           opus        29k 48k ultralow, MWEB, webm_dash
139-drc m4a   audio only      2 │  980.78KiB    49k https │ audio only           mp4a.40.5   49k 22k low, DRC, IOS, m4a_dash
249-drc webm  audio only      2 │  898.02KiB    45k https │ audio only           opus        45k 48k low, DRC, WEB, webm_dash
250-drc webm  audio only      2 │    1.16MiB    59k https │ audio only           opus        59k 48k low, DRC, WEB, webm_dash
139     m4a   audio only      2 │  980.78KiB    49k https │ audio only           mp4a.40.5   49k 22k low, IOS, m4a_dash
249     webm  audio only      2 │  878.15KiB    44k https │ audio only           opus        44k 48k low, ANDR-E, webm_dash
250     webm  audio only      2 │    1.13MiB    58k https │ audio only           opus        58k 48k low, ANDR-E, webm_dash
140-drc m4a   audio only      2 │    2.54MiB   130k https │ audio only           mp4a.40.2  130k 44k medium, DRC, IOS, m4a_dash
251-drc webm  audio only      2 │    2.35MiB   120k https │ audio only           opus       120k 48k medium, DRC, WEB, webm_dash
140     m4a   audio only      2 │    2.54MiB   130k https │ audio only           mp4a.40.2  130k 44k medium, IOS, m4a_dash
251     webm  audio only      2 │    2.30MiB   117k https │ audio only           opus       117k 48k medium, ANDR-E, webm_dash
597     mp4   256x144     15    │  629.80KiB    31k https │ avc1.4d400b      31k video only          144p, MWEB, mp4_dash
602     mp4   256x144     15    │ ~  2.09MiB   107k m3u8  │ vp09.00.10.08   107k video only          IOS
598     webm  256x144     15    │  718.95KiB    36k https │ vp9              36k video only          144p, MWEB, webm_dash
394     mp4   256x144     30    │  785.95KiB    39k https │ av01.0.00M.08    39k video only          144s, mesh, WEB, mp4_dash
269     mp4   256x144     30    │ ~  3.55MiB   181k m3u8  │ avc1.4D400C     181k video only          IOS
160     mp4   256x144     30    │    2.19MiB   112k https │ avc1.4D400C     112k video only          144s, mesh, IOS, mp4_dash
603     mp4   256x144     30    │ ~  3.30MiB   169k m3u8  │ vp09.00.11.08   169k video only          IOS
278     webm  256x144     30    │  890.51KiB    44k https │ vp09.00.11.08    44k video only          144s, mesh, IOS, webm_dash
395     mp4   426x240     30    │    1.41MiB    72k https │ av01.0.00M.08    72k video only          240s, mesh, WEB, mp4_dash
229     mp4   426x240     30    │ ~  6.51MiB   333k m3u8  │ avc1.4D4015     333k video only          IOS
133     mp4   424x240     30    │    4.80MiB   245k https │ avc1.4D4015     245k video only          240s, mesh, IOS, mp4_dash
604     mp4   426x240     30    │ ~  6.33MiB   324k m3u8  │ vp09.00.20.08   324k video only          IOS
242     webm  424x240     30    │    1.81MiB    92k https │ vp09.00.20.08    92k video only          240s, mesh, IOS, webm_dash
396     mp4   640x360     30    │    2.93MiB   149k https │ av01.0.01M.08   149k video only          360s, mesh, WEB, mp4_dash
230     mp4   640x360     30    │ ~ 15.25MiB   780k m3u8  │ avc1.4D401E     780k video only          IOS
134     mp4   640x360     30    │    5.25MiB   268k https │ avc1.4D401E     268k video only          360s, mesh, IOS, mp4_dash
18      mp4   640x360     30  2 │ ≈  8.14MiB   415k https │ avc1.42001E          mp4a.40.2       44k 360p, IOS-C
605     mp4   640x360     30    │ ~ 11.99MiB   613k m3u8  │ vp09.00.21.08   613k video only          IOS
243     webm  640x360     30    │    3.58MiB   183k https │ vp09.00.21.08   183k video only          360s, mesh, IOS, webm_dash
397     mp4   854x480     30    │    5.82MiB   297k https │ av01.0.04M.08   297k video only          480s, mesh, WEB, mp4_dash
231     mp4   854x480     30    │ ~ 25.33MiB  1296k m3u8  │ avc1.4D401F    1296k video only          IOS
135     mp4   852x480     30    │   11.07MiB   565k https │ avc1.4D401F     565k video only          480s, mesh, IOS, mp4_dash
606     mp4   854x480     30    │ ~ 20.62MiB  1055k m3u8  │ vp09.00.30.08  1055k video only          IOS
244     webm  852x480     30    │    7.03MiB   359k https │ vp09.00.30.08   359k video only          480s, mesh, IOS, webm_dash
22      mp4   1280x720    30  2 │ ≈ 24.53MiB  1251k https │ avc1.64001F          mp4a.40.2       44k 720p, ANDR-E
398     mp4   1280x720    30    │   13.81MiB   705k https │ av01.0.05M.08   705k video only          720s, mesh, WEB, mp4_dash
232     mp4   1280x720    30    │ ~ 45.96MiB  2351k m3u8  │ avc1.64001F    2351k video only          IOS
136     mp4   1280x720    30    │   26.47MiB  1351k https │ avc1.4D401F    1351k video only          720s, mesh, IOS, mp4_dash
609     mp4   1280x720    30    │ ~ 42.05MiB  2151k m3u8  │ vp09.00.31.08  2151k video only          IOS
247     webm  1280x720    30    │   16.59MiB   847k https │ vp09.00.31.08   847k video only          720s, mesh, IOS, webm_dash
270     mp4   1920x1078   30    │ ~ 93.08MiB  4761k m3u8  │ avc1.640028    4761k video only          IOS
614     mp4   1920x1078   30    │ ~ 59.09MiB  3023k m3u8  │ vp09.00.40.08  3023k video only          IOS
399     mp4   1920x1080   30    │   30.41MiB  1552k https │ av01.0.08M.08  1552k video only          1080s, mesh, WEB, mp4_dash
137     mp4   1920x1080   30    │   50.22MiB  2563k https │ avc1.640028    2563k video only          1080s, mesh, IOS, mp4_dash
248     webm  1920x1080   30    │   29.45MiB  1503k https │ vp09.00.40.08  1503k video only          1080s, mesh, IOS, webm_dash
620     mp4   2560x1438   30    │ ~160.97MiB  8234k m3u8  │ vp09.00.50.08  8234k video only          IOS
400     mp4   2560x1440   30    │  102.42MiB  5226k https │ av01.0.12M.08  5226k video only          1440s, mesh, WEB, mp4_dash
271     webm  2560x1440   30    │  119.39MiB  6092k https │ vp09.00.50.08  6092k video only          1440s, mesh, IOS, webm_dash
401     mp4   3840x2160   30    │  215.94MiB 11018k https │ av01.0.12M.08 11018k video only          2160s, mesh, WEB, mp4_dash
313     webm  3840x2160   30    │  287.64MiB 14677k https │ vp09.00.50.08 14677k video only          2160s, mesh, IOS, webm_dash
bashonly commented 6 months ago

This is a side-effect of #9554. VR180 formats are only available via the Android client(s). It's not possible for yt-dlp to get Android client formats anymore.

It's possible that the VR formats may be served to another client in the future, such as the upcoming client for VisionOS.

R-S1978 commented 6 months ago

This is a side-effect of #9554. VR180 formats are only available via the Android client(s). It's not possible for yt-dlp to get Android client formats anymore.

It's possible that the VR formats may be served to another client in the future, such as the upcoming client for VisionOS.

Thanks for this information. But why can the official youtube app on meta quest get the 8k format? Is it using some other protocol?

bashonly commented 6 months ago

Meta's VR gear uses the Android client

kclauhk commented 6 months ago

(this is not a real fix but I hope this may help in some use cases)

If you pass --extractor-args "youtube:player_client=android" to yt-dlp, it will extract the VR formats, but the format download links will expire after ~30 seconds and then start to return HTTP Error 403. i.e. you can successfully download the video in one attempt if download finishes within 30 seconds. If it cannot finish within 30 seconds, you simply run yt-dlp again to resume download until the whole video is downloaded.

For example, the video mentioned in this post:

>python -m yt_dlp "https://www.youtube.com/watch?v=W51jdIrSsgk" --extractor-args "youtube:player_client=android" -f 571+140
[youtube] Extracting URL: https://www.youtube.com/watch?v=W51jdIrSsgk
[youtube] W51jdIrSsgk: Downloading webpage
[youtube] W51jdIrSsgk: Downloading android player API JSON
[info] W51jdIrSsgk: Downloading 1 format(s): 571+140
[download] Destination: NIS FORTRESS Archaeological site and modern Jazz Museum SERBIA 8K 4K VR180 3D Travel [W51jdIrSsgk].f571.mp4
[download]  64.0% of  532.55MiB at   13.26MiB/s ETA 00:14ERROR: unable to download video data: HTTP Error 403: Forbidden

>python -m yt_dlp "https://www.youtube.com/watch?v=W51jdIrSsgk" --extractor-args "youtube:player_client=android" -f 571+140
[youtube] Extracting URL: https://www.youtube.com/watch?v=W51jdIrSsgk
[youtube] W51jdIrSsgk: Downloading webpage
[youtube] W51jdIrSsgk: Downloading android player API JSON
[info] W51jdIrSsgk: Downloading 1 format(s): 571+140
[download] Resuming download at byte 357378580
[download] Destination: NIS FORTRESS Archaeological site and modern Jazz Museum SERBIA 8K 4K VR180 3D Travel [W51jdIrSsgk].f571.mp4
[download] 100% of  532.55MiB in 00:00:17 at 31.31MiB/s
[download] Destination: NIS FORTRESS Archaeological site and modern Jazz Museum SERBIA 8K 4K VR180 3D Travel [W51jdIrSsgk].f140.m4a
[download] 100% of    2.54MiB in 00:00:00 at 9.66MiB/s
[Merger] Merging formats into "NIS FORTRESS Archaeological site and modern Jazz Museum SERBIA 8K 4K VR180 3D Travel [W51jdIrSsgk].mp4"
Deleting original file NIS FORTRESS Archaeological site and modern Jazz Museum SERBIA 8K 4K VR180 3D Travel [W51jdIrSsgk].f140.m4a (pass -k to keep)
Deleting original file NIS FORTRESS Archaeological site and modern Jazz Museum SERBIA 8K 4K VR180 3D Travel [W51jdIrSsgk].f571.mp4 (pass -k to keep)

You can automate it with a script, e.g. a simple example for Windows:

@echo off
set arg=%1
for %%i in (%*) do call :concat %%i
set argc=%arg:"=%
set resume=0

python -m yt_dlp %arg%

:resume
if %ERRORLEVEL%==0 (goto :eof) else (
  if %resume%==5 (goto :eof) else (
    if "%argc:--extractor-args youtube:player_client=%" neq "%argc:"=%" if "%argc:android=%" neq "%argc:"=%" (
      python -m yt_dlp %arg%
      set /a "resume=%resume%+1"
    ) else goto :eof
  )
)
goto :resume

:concat
set arg=%arg% %1
goto :eof

use it as follow:

C:\yt-dlp>ex.bat "https://www.youtube.com/watch?v=W51jdIrSsgk" --extractor-args "youtube:player_client=android" -f 571+140

This script will stop downloading if it resumes more than 5 times. Change the number if needed.

pukkandan commented 6 months ago

@coletdjnz What do you think about removing the params from code and returning android formats with lower priority when requested with player-client=android?

bashonly commented 6 months ago

What do you think about removing the params from code and returning android formats with lower priority when requested with player-client=android?

Now implemented in https://github.com/yt-dlp/yt-dlp/pull/9553

ghost commented 5 months ago

yt-dlp with default settings only shows 2k quality as best: yt-dlp -F "https://www.youtube.com/watch?v=c_jOI6Wboc4" ... 308 webm 2560x1438 60 │ 342.56MiB 10388k https │ vp9 10388k video only 1440p60, webm_dash

with specified user agent from firefox, its the same: yt-dlp --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0' -F "https://www.youtube.com/watch?v=c_jOI6Wboc4" 308 webm 2560x1438 60 │ 342.56MiB 10388k https │ vp9 10388k video only 1440p60, webm_dash

with user agent and cookies from browser, it shows 4k option: yt-dlp --user-agent 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:126.0) Gecko/20100101 Firefox/126.0' --cookies-from-browser firefox:path_to_profile -F "https://www.youtube.com/watch?v=c_jOI6Wboc4" ... 315 webm 3840x2160 60 │ 682.67MiB 20703k https │ vp9 20703k video only 2160s60, mesh, webm_dash

specifying both: client as android and cookies from browser shows 8k quality option: yt-dlp --extractor-args "youtube:player_client=android" --cookies-from-browser firefox:path_to_profile -F "https://www.youtube.com/watch?v=c_jOI6Wboc4" 571 mp4 7680x3840 60 │ 795.66MiB 24129k https │ av01.0.17M.08 24129k video only 4320s60, mesh, BROKEN, mp4_dash

this video is age restricted, which makes it require login cookie to show 8k quality: https://www.youtube.com/watch?v=c_jOI6Wboc4

with this video, only specifying client is enough to show 8k quality option: yt-dlp --extractor-args "youtube:player_client=android" -F "https://www.youtube.com/watch?v=Ypu5k5GYGjE"

R-S1978 commented 5 months ago

Interesting, thank you! 👍

Now it would be great when yt-dlp would be able to resume the download when the error message "HTTP Error 403: Forbidden" occurs. I think that would fix the problem? 😀

godlovesdavid commented 5 months ago

the player_client=androidoption worked for me, thanks @kclauhk ! i didnt even need to retry after 30s. i am using the --cookies-from-browser chrome which might've helped keep the connection alive.

kclauhk commented 5 months ago

i didnt even need to retry after 30s. i am using the --cookies-from-browser chrome which might've helped keep the connection alive.

DO NOT PASS YOUR COOKIES unless it is really necessary... or you may register a new account for the use of yt-dlp

See https://github.com/yt-dlp/yt-dlp/issues/10128#issuecomment-2155037169

rook218 commented 5 months ago

Thank you @kclauhk , this solution works for many videos.

I am having a problem with a video like this, though: https://www.youtube.com/watch?v=HI7mTIxNotQ

The option I get for downloading is -f 313 but that outputs a wonky format: https://i.imgur.com/tQhbIwU.png

All other formats look equivalent, just lower quality. Is there a trick to getting those to post-process into a proper SBS video?

princeadam001 commented 5 months ago

@rook218 That's a top/bottom 3D 360 video, not SBS 180. It looks like that because YouTube used to use the equi-angular cubemap (EAC) format for 360 videos (maybe still does for some) instead of the standard equirectangular format. https://github.com/ytdl-org/youtube-dl/issues/15267

There are some VR video players that support the EAC format (like Skybox), but if you want the equirectangular version you can download it by adding --user-agent "".

Example: yt-dlp --user-agent "" "https://www.youtube.com/watch?v=HI7mTIxNotQ" -f 313+338

tuber100 commented 4 months ago

@rook218 That's a top/bottom 3D 360 video, not SBS 180. It looks like that because YouTube used to use the equi-angular cubemap (EAC) format for 360 videos (maybe still does for some) instead of the standard equirectangular format. ytdl-org/youtube-dl#15267

There are some VR video players that support the EAC format (like Skybox), but if you want the equirectangular version you can download it by adding --user-agent "".

Example: yt-dlp --user-agent "" "https://www.youtube.com/watch?v=HI7mTIxNotQ" -f 313+338

Many thanks for the command. but the video downloaded is not stereoscopic but mono. how to download these cube format 360 videos in a 3d format recognized by default players (for e.g deovr, quest tv etc.)

princeadam001 commented 4 months ago

Many thanks for the command. but the video downloaded is not stereoscopic but mono. how to download these cube format 360 videos in a 3d format recognized by default players (for e.g deovr, quest tv etc.)

Which video are you trying to download? The video I used in the example downloads in top/bottom format for me using that command.

tuber100 commented 4 months ago

Which video are you trying to download? The video I used in the example downloads in top/bottom format for me using that command.

https://www.youtube.com/watch?v=ZgYSKbaM5fg or any 360 video i tried so far. yt-dlp downloads it in weird cube map format which no player recognizes. I just want it in sbs or top-bottom format

princeadam001 commented 4 months ago

Which video are you trying to download? The video I used in the example downloads in top/bottom format for me using that command.

https://www.youtube.com/watch?v=ZgYSKbaM5fg or any 360 video i tried so far. yt-dlp downloads it in weird cube map format which no player recognizes. I just want it in sbs or top-bottom format

That video is 2D 360, so there's no stereoscopic format available. I downloaded the 8K equirectangular (not cubemap) version of it using this command: yt-dlp --user-agent "" "https://www.youtube.com/watch?v=ZgYSKbaM5fg" -f 571+140

bashonly commented 3 months ago

As of version 2024.08.01 you can try --extractor-arg "youtube:player_client=android_vr"