Closed rredford6 closed 1 day ago
will be "fixed" by #9553 ( :cry: ). yt doesn't want us to have nice things
will be "fixed" by #9553 ( :cry: ). yt doesn't want us to have nice things
I'm pretty sure YouTube adding that has nothing to do with yt-dlp and is entirely because of ReVanced and yt-dlp is just collateral damage. As ReVanced modifies the official YouTube app (unlike NewPipe which is a separate app written from scratch), it's no longer enough for YouTube to just check if you are using the Android app, they now need to check if you are using a modded version of it.
To maintainers: Don't close the issue till release. From experience, keeping issue open seems to reduce number of duplicates
I can confirm this happening in New Zealand also (no VPN, native IPv4/IPv6):
[youtube] Extracting URL: https://youtu.be/luXZz9AqBbQ
[youtube] luXZz9AqBbQ: Downloading webpage
[youtube] luXZz9AqBbQ: Downloading ios player API JSON
[youtube] luXZz9AqBbQ: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "luXZz9AqBbQ")
[youtube] luXZz9AqBbQ: Downloading m3u8 information
[info] luXZz9AqBbQ: Downloading 1 format(s): 18
[download] Destination: r⧸AskOuija where they talk one LETTER at a time [luXZz9AqBbQ].mp4
[download] 100% of 10.50MiB in 00:00:02 at 4.20MiB/s
[Metadata] Adding metadata to "r⧸AskOuija where they talk one LETTER at a time [luXZz9AqBbQ].mp4"
`
Downloading the "aQvGIIdgFDM" video gives a file titled "Video Not Available [aQvGIIdgFDM].mp4", length of 5 minutes, black background, circled exclamation point on left, "The following content is not available in this app. Watch this content on the latest version of YouTube" on the right.
Whatever kind of black magic is YT trying to pull on us now, and why it is happening to Android client APIs specifically is currently a mystery to me. Why is this not happening on the iOS one as well?
UPDATE: Tested on the 'https://youtu.be/BaW_jenozKc' video as well, failed to replicate on that video:
[youtube] Extracting URL: https://youtu.be/BaW_jenozKc
[youtube] BaW_jenozKc: Downloading webpage
[youtube] BaW_jenozKc: Downloading ios player API JSON
[youtube] BaW_jenozKc: Downloading android player API JSON
[youtube] BaW_jenozKc: Downloading m3u8 information
[info] BaW_jenozKc: Downloading 1 format(s): 18
[download] Destination: youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].mp4
[download] 100% of 184.94KiB in 00:00:02 at 73.93KiB/s
[Metadata] Adding metadata to "youtube-dl test video "'⧸⧹ä↭𝕐 [BaW_jenozKc].mp4"
Possible A/B testing perhaps? Knowing YT's shady testing behaviour, it's certainly could be...
will be "fixed" by #9553
For an end-user is there any significant difference between that fix and simply using "--extractor-args youtube:player_client=ios,web" in your config?
The docs mention that
By default, ios,android,web is used, but tv_embedded and creator variants are added as required for age-gated videos.
and I'm not sure if explicitly specifying player clients overrides that behavior (and are those variants ever needed anyway if you're using cookies for a logged in youtube account?)
From experience, keeping issue open seems to reduce number of duplicates
That's probably because people like me tend to come here only when they have an issue themselves and don't realize that closed issues are hidden by default, and so think no one has posted about their issue yet.
From experience, keeping issue open seems to reduce number of duplicates
That's probably because people like me tend to come here only when they have an issue themselves and don't realize that closed issues are hidden by default, and so think no one has posted about their issue yet.
I know. We specifically ask to search closed issues, but many don't. It's what it is... Not trying to blame anyone. I was just passing the wisdom onto other maintainers since they are the ones handling most issues nowadays
- [x] I've searched known issues and the bugtracker for similar issues including closed ones. DO NOT post duplicates
For an end-user is there any significant difference between that fix and simply using "--extractor-args youtube:player_client=ios,web" in your config?
By default, ios,android,web is used, but tv_embedded and creator variants are added as required for age-gated videos.
and I'm not sure if explicitly specifying player clients overrides that behavior
No. The age-gate clients are still added when required. You could also simply ignore the warning.
(and are those variants ever needed anyway if you're using cookies for a logged in youtube account?)
It is not. If your account can directly access the video, we don't need any bypass
Whatever kind of black magic is YT trying to pull on us now, and why it is happening to Android client APIs specifically is currently a mystery to me.
Youtube is using Safetynet's AndroidGuard to validate that the requests are coming from their app; possibly in an effort to block revanced, though they have already implemented a bypass.
Possible A/B testing perhaps?
Yes, it happens randomly, though expect it to be completely blocked in few days
Fixed, but leaving open till release
This is happening in Venezuela also
@SantiiRepair try running yt-dlp --update-to master
, and if it still keeps happening, add --verbose
to your command and send the output
@SantiiRepair try running
yt-dlp --update-to master
, and if it still keeps happening, add--verbose
to your command and send the output
Thanks for response, but I got this:
Current version: stable@2024.03.10 from yt-dlp/yt-dlp
Latest version: master@2024.04.08.232116 from yt-dlp/yt-dlp-master-builds
Looks like GitHub master branch needs to be updated? Thx
That is not a complete verbose log
Closing since release is out
The sequel: https://github.com/yt-dlp/yt-dlp/issues/9725
Reopening this, since it's happening again. Note that as of today, it happens very infrequently. Presumably, this is due to some sort of A/B testing and the problem will become more prevalent eventually. It's likely that the new android player params' end-of-life date is near. This issue will be "fixed" by #9553
I have the same bug since it looks like the bug reappeared yesterday it keeps pausing on almost every download now it was fine 3 days ago. and it keeps saying
[debug] Command-line config: ['--remux-video', 'mp4', '-vU', 'https://www.youtube.com/watch?v=yUDc78znrLM'] [debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version nightly@2024.04.21.232710 from yt-dlp/yt-dlp-nightly-builds [ff38a011d] (pip) [debug] Python 3.12.2 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 3.0.13 30 Jan 2024) [debug] exe versions: ffmpeg 7.0-full_build-www.gyan.dev (setts), ffprobe 7.0-full_build-www.gyan.dev [debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.02.02, mutagen-1.47.0, requests-2.31.0, sqlite3-3.43.1, urllib3-2.2.1, websockets-12.0 [debug] Proxy map: {} [debug] Request Handlers: urllib, requests, websockets [debug] Loaded 1810 extractors [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp-nightly-builds/releases/latest Latest version: nightly@2024.04.21.232710 from yt-dlp/yt-dlp-nightly-builds yt-dlp is up to date (nightly@2024.04.21.232710 from yt-dlp/yt-dlp-nightly-builds) [youtube] Extracting URL: https://www.youtube.com/watch?v=yUDc78znrLM [youtube] yUDc78znrLM: Downloading webpage [youtube] yUDc78znrLM: Downloading ios player API JSON [youtube] yUDc78znrLM: Downloading android player API JSON WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "yUDc78znrLM") [debug] Loading youtube-nsig.9135c2ab from cache [debug] [youtube] Decrypted nsig LAauB_HxR-jWq4Kf => E42fDI0hSq25Rg [debug] Loading youtube-nsig.9135c2ab from cache [debug] [youtube] Decrypted nsig TMahUCnN2xYyyrhq => gfGiU_As3Byuhg [youtube] yUDc78znrLM: Downloading m3u8 information [debug] Sort order given by extractor: quality, res, fps, hdr:12, source, vcodec:vp9.2, channels, acodec, lang, proto [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, vcodec:vp9.2(10), channels, acodec, lang, proto, size, br, asr, vext, aext, hasaud, id [debug] Default format spec: bestvideo*+bestaudio/best [info] yUDc78znrLM: Downloading 1 format(s): 243+251 [debug] Invoking http downloader on "https://rr5---sn-8xgp1vo-p5qe.googlevideo.com/videoplayback?expire=1714043187&ei=0-QpZqlUu4SS5w-pqIzYBg&ip=71.171.34.118&id=o-ACBN7U_gWbgQwgm4V7JnhfuFglkYpmCa40ri08QltZ8P&itag=243&source=youtube&requiressl=yes&xpc=EgVo2aDSNQ%3D%3D&mh=Tl&mm=31%2C26&mn=sn-8xgp1vo-p5qe%2Csn-vgqsrn6l&ms=au%2Conr&mv=m&mvi=5&pl=18&initcwndbps=1317500&vprv=1&svpuc=1&mime=video%2Fwebm&gir=yes&clen=17118776&dur=528.026&lmt=1527102558874107&mt=1714020757&fvip=4&keepalive=yes&c=IOS&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cxpc%2Cvprv%2Csvpuc%2Cmime%2Cgir%2Cclen%2Cdur%2Clmt&sig=AJfQdSswRAIgSb0QmkfNQpG50j5e9J0cAVNeddAKKriEloTWk9esutACIApTXKd_Tb3jxufHcYQ7pI6SqZZFZFamr9SkFpjI8RMf&lsparams=mh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHWaYeowRQIgDasqd-6DN63YPcH770QJ_VLL0KbgLpuBAWUGdhhNAIgCIQCQ2N0zRjtCdSmKvuXPPuLqkNrTxx47YPVv5pvb916Jpw%3D%3D" [debug] File locking is not supported. Proceeding without locking [download] Destination: Thru The Mirror (1936) Mickey Mouse complete [yUDc78znrLM].f243.webm [download] 3.1% of 16.33MiB at 343.44KiB/s ETA 00:47
IT always says this for every download now: Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "yUDc78znrLM") and it always pasuses then retries
removing the android params seems work
@kclauhk the stream URL expires after 1 minute resulting in a 403 error during download if the params are removed
ok
how is this "{1: {1: 1}}" converted to "CgIIAQ=="?
how is this "{1: {1: 1}}" converted to "CgIIAQ=="?
It's protobuf. If you want to experiment with the params you could use https://github.com/Grub4K/qpb
For a temporary fix you can install the fix from this PR https://github.com/yt-dlp/yt-dlp/pull/9553:
python3 -m pip install -U pip hatchling wheel
python3 -m pip install --force-reinstall "yt-dlp[default] @ https://github.com/coletdjnz/yt-dlp-dev/archive/fix/youtube/remove-android-as-default.tar.gz"
FWIW, I'm not getting these intermittently, I've seen this warning now on all of the last ~ 30 videos or so from YT.
PS C:\> yt-dlp -U
Latest version: nightly@2024.04.28.232723 from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@2024.04.28.232723 from yt-dlp/yt-dlp-nightly-builds)
PS C:\>
Region is EU (DE).
Oh, and there is also a second warning (at least for me) now, and I couldn't find anything about this with a search in Issues here.
PS C:\> yt-dlp --youtube 'https://www.youtube.com/watch?v=zKRYqI84ov0'
[youtube] Extracting URL: https://www.youtube.com/watch?v=zKRYqI84ov0
[youtube] zKRYqI84ov0: Downloading webpage
[youtube] zKRYqI84ov0: Downloading ios player API JSON
[youtube] zKRYqI84ov0: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "zKRYqI84ov0")
[youtube] zKRYqI84ov0: Downloading m3u8 information
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
[youtube] zKRYqI84ov0: Downloading initial data API JSON
[MetadataParser] Did not find '[.]' in uploader
[MetadataParser] Did not find '[.]' in title
[MetadataParser] Did not find '[.]' in extractor
[MetadataParser] Did not find '[.]' in id
[MetadataParser] Changed extractor to: YouTube
[MetadataParser] Did not find '[.]' in channel
[info] zKRYqI84ov0: Downloading 1 format(s): 313+251
[download] Destination: [..]
I've just picked this video link from my front page, seems like it is the latest video upload from the WhistlinDiesel channel. Download (and video) still seem to work, btw.
also getting "android" warnings while running from Germany, Austria, Thailand
same warning from Hongkong
any update guys ?
You can simply ignore the warning. We don't have any way to get the android formats now. But for most users, this will have no effect. The warning will be removed in the next stable release.
Pls avoid commenting unless you have something useful to add.
@pukkandan What about the second warning (from my log 5 comments above)?
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
Not sure if it's related at all. Maybe it belongs into its own issue, not sure.
@Hrxn Was that a one-off, or consistent issue? If is is happening consistently, pls open a new issue. Otherwise, don't worry about it. Youtube API sometimes gives incomplete results and so we have fallbacks for it. The warning is just informing that a fallback was triggered
Definitely not a one-off thing for me, happened for the last dozen or so videos I tried..
But it seems that I cannot reproduce with --ignore-config
. I only get the android client response warning from this issue here then, as expected. Only with my normal cookies I get these two warnings again.
PS C:\Apps\Temp\2024> yt-dlp --freshcookies
Extracting cookies from chrome
Extracted 3264 cookies from chrome
Latest version: nightly@2024.05.05.232701 from yt-dlp/yt-dlp-nightly-builds
yt-dlp is up to date (nightly@2024.05.05.232701 from yt-dlp/yt-dlp-nightly-builds)
PS C:\Apps\Temp\2024> yt-dlp --ignore-config 'https://www.youtube.com/watch?v=NAwH6d1Gp9g'
[youtube] Extracting URL: https://www.youtube.com/watch?v=NAwH6d1Gp9g
[youtube] NAwH6d1Gp9g: Downloading webpage
[youtube] NAwH6d1Gp9g: Downloading ios player API JSON
[youtube] NAwH6d1Gp9g: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "NAwH6d1Gp9g")
[youtube] NAwH6d1Gp9g: Downloading m3u8 information
[info] NAwH6d1Gp9g: Downloading 1 format(s): 313+251
[download] Destination: Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f313.webm
[download] 100% of 989.63MiB in 00:00:41 at 23.74MiB/s
[download] Destination: Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f251.webm
[download] 100% of 10.54MiB in 00:00:00 at 23.26MiB/s
[Merger] Merging formats into "Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].webm"
Deleting original file Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f251.webm (pass -k to keep)
Deleting original file Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f313.webm (pass -k to keep)
PS C:\Apps\Temp\2024> ren '.\Audi Group B Rally Car vs THE STIG | TG Tunnel Run `[NAwH6d1Gp9g`].webm' '.\Audi Group B Rally Car vs THE STIG | TG Tunnel Run `[NAwH6d1Gp9g`]_NO_CONFIG_.webm'
PS C:\Apps\Temp\2024> yt-dlp --ignore-config --cookies .\yt-dlp.cookies.txt 'https://www.youtube.com/watch?v=NAwH6d1Gp9g'
[youtube] Extracting URL: https://www.youtube.com/watch?v=NAwH6d1Gp9g
[youtube] NAwH6d1Gp9g: Downloading webpage
[youtube] NAwH6d1Gp9g: Downloading ios player API JSON
[youtube] NAwH6d1Gp9g: Downloading android player API JSON
WARNING: [youtube] Skipping player responses from android clients (got player responses for video "aQvGIIdgFDM" instead of "NAwH6d1Gp9g")
[youtube] NAwH6d1Gp9g: Downloading m3u8 information
WARNING: [youtube] Incomplete data received in embedded initial data; re-fetching using API.
[youtube] NAwH6d1Gp9g: Downloading initial data API JSON
[info] NAwH6d1Gp9g: Downloading 1 format(s): 313+251
[download] Destination: Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f313.webm
[download] 100% of 989.63MiB in 00:00:49 at 19.93MiB/s
[download] Destination: Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f251.webm
[download] 100% of 10.54MiB in 00:00:00 at 18.72MiB/s
[Merger] Merging formats into "Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].webm"
Deleting original file Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f251.webm (pass -k to keep)
Deleting original file Audi Group B Rally Car vs THE STIG | TG Tunnel Run [NAwH6d1Gp9g].f313.webm (pass -k to keep)
PS C:\Apps\Temp\2024>
Does this even make any sense??
Creating a new issue for an unrelated problem makes sense
the issue has any update?
any update for this issue?
This is happening to me using python api while not happening calling yt-dlp
through os.system
or subprocess
(or in plain bash) using the same arguments. Weird!
This is happening to me using python api while not happening calling
yt-dlp
throughos.system
orsubprocess
(or in plain bash) using the same arguments. Weird!
does user agent affect this ?
does user agent affect this ?
No. And claims like the one above have no meaning if verbose logs are not shared
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
Checklist
Region
United States
Provide a description that is worded well enough to be understood
This is the crux of the issue:
This was previously solved in #4593. It was first rediscovered on https://github.com/yt-dlp/yt-dlp/pull/9546#issuecomment-2025347080
This can be reproduced on multiple YouTube videos. This warning is not seen on every download.
Provide verbose output that clearly demonstrates the problem
yt-dlp -vU <your command line>
)'verbose': True
toYoutubeDL
params instead[debug] Command-line config
) and insert it belowComplete Verbose Output