ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.3k stars 10.03k forks source link

YouTube age restriction #29578

Closed 1RandomDev closed 1 year ago

1RandomDev commented 3 years ago

Checklist

Verbose log

$ youtube-dl -v https://www.youtube.com/watch?v=irauhITDrsE
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.youtube.com/watch?v=irauhITDrsE']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.4.0-77-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[youtube] irauhITDrsE: Downloading webpage
[youtube] irauhITDrsE: Refetching age-gated info webpage
WARNING: unable to download video info webpage: HTTP Error 404: Not Found
ERROR: Sign in to confirm your age
This video may be inappropriate for some users.
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/extractor/youtube.py", line 1713, in _real_extract
    raise ExtractorError(reason, expected=True)
youtube_dl.utils.ExtractorError: Sign in to confirm your age
This video may be inappropriate for some users.

Description

When I try to download age restricted videos it, tells me to sign in to a verified account, which I don't have. An option to bypass this shit without a slow vpn or proxy would be great. I'm really sure there is a way of doing this, for example FreeTube can play all restricted videos without problems.

1RandomDev commented 3 years ago

I've just found a Tampermonkey script which also manages to bypass age restrictions very reliable: https://greasyfork.org/de/scripts/375525-youtube-age-verification-bypass

As I see it gathers the video info from https://www.youtube.com/get_video_info?html5=1&c=TVHTML5&cver=6.20180913&asv=3&video_id=.......&eurl=https://youtube.googleapis.com/v/

Maybe this can also be used in ytdl?

pukkandan commented 3 years ago

Duplicate of https://github.com/ytdl-org/youtube-dl/issues/29086

TLDR: This has been fixed in code, but is currently unreleased

mechalincoln commented 3 years ago

@pukkandan do we know when the next release is coming?

pukkandan commented 3 years ago

you'll have to ask the maintainers

rautamiekka commented 3 years ago

@pukkandan do we know when the next release is coming?

Just download the whole repo and run python youtube_dl/__main__.py with your flags/params like you did with a release one.

DMW007 commented 3 years ago

I tried to run the latest source code instead of the release with a sample age restricted video, but with the same result:

python youtube_dl/__main__.py "https://www.youtube.com/watch?v=JsTjPtGPOIE"
[youtube] JsTjPtGPOIE: Downloading webpage
[youtube] JsTjPtGPOIE: Refetching age-gated info webpage
WARNING: unable to download video info webpage: HTTP Error 404: Not Found
ERROR: Sign in to confirm your age
This video may be inappropriate for some users.

Also with --username and since that doesn't work also --password specified so that the request is authenticated (but the account has no id/credit card verification) doesn't help:

[youtube] Downloading login page
[youtube] Looking up account info
WARNING: Unable to look up account info: HTTP Error 400: Bad Request
[youtube] Tq92D6wQ1mg: Downloading webpage
[youtube] Tq92D6wQ1mg: Refetching age-gated info webpage
WARNING: unable to download video info webpage: HTTP Error 404: Not Found
ERROR: Sign in to confirm your age
This video may be inappropriate for some users.

Maybe this is because of googles 2FA stuff, which forces to enter a code on new devices sometimes.

RealDeviil commented 3 years ago

Request like this works as expected:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
Authorization: Bearer ya29.a0ARrdaM8ubnr5IhwHHNB2vZh-VcGZT2JaeVtOnF-7SZMqmnES0...

{"context":{"client":{"clientName":"ANDROID","clientVersion":"16.05"}},
"racyCheckOk":true,"videoId":"irauhITDrsE"}

@89z The only way this works is by sending a Bearer token in request header? Where/How are you getting this token?

In another issue, you also indicate that a valid YouTube Data API key must be included in the request header?

RealDeviil commented 3 years ago

@89z Thanks a lot. I can't find any age-restricted video that works by "only" providing valid X-Goog-Api-Key header value (without OAuth or cookies). Can you provide an example URL that works in this case?

(Note: Your example video is "not" age-restricted, so it does not even need the X-Goog-Api-Key header.)

Otherwise, it seems that using OAuth or login cookie is the only way to access the video source URLs. I.e., you must login into YouTube with username and password.

RealDeviil commented 3 years ago

@89z Thanks a lot. I can't find any age-restricted video that works by "only" providing valid X-Goog-Api-Key header value (without OAuth or cookies). Can you provide an example URL that works in this case?

q4qWMcQfOCc

This is an age-restricted video, but using only X-Goog-Api-Key header, I do not get any download URLs in the JSON response. I get video info, but no download URLs in that info. Do you?

This is what I get:

{ "responseContext": { "visitorData": "CgtLZGtTazZwUkVUTSjZzfGHBg%3D%3D", "serviceTrackingParams": [ { "service": "GFEEDBACK", "params": [ { "key": "is_viewed_live", "value": "False" }, { "key": "logged_in", "value": "0" }, { "key": "e", "value": "23884386,24064582,24016285,24037794,24021124,23946420,24060921,24042870,24030481,23929728,23891346,23857949,24058128,23966208,24068842,23996512,24057238,23891344,23983296,24030040,23996830,23974595,23744176,24049573,24028143,24000751,23882503,23918597,24059521,24056264,24049571,24019422,23858057,23934970,1714248,24049820,24050503,24067361,24001373,23968386,24007246,24043232,24045411,24053866,24056275,24036948,24004644,24058812,23944779,23998056" } ] }, { "service": "CSI", "params": [ { "key": "c", "value": "ANDROID" }, { "key": "cver", "value": "16.05" }, { "key": "yt_li", "value": "0" }, { "key": "GetPlayer_rid", "value": "0x84680bd1fc532805" } ] }, { "service": "GUIDED_HELP", "params": [ { "key": "logged_in", "value": "0" } ] }, { "service": "ECATCHER", "params": [ { "key": "client.version", "value": "16.05" }, { "key": "client.name", "value": "ANDROID" } ] }, { "service": "LISTNR", "params": [ { "key": "e", "value": "23996830,24024718,24050503,24049820,24064582,24036948,24030481" } ] } ] }, "trackingParams": "CAAQu2kiEwjYkeTCtfzxAhUqQnoFHewJDHA=", "playabilityStatus": { "status": "LOGIN_REQUIRED", "reason": "This video may be inappropriate for some users.", "errorScreen": { "confirmDialogRenderer": { "title": { "runs": [ { "text": "You must sign in to view this video" } ] }, "trackingParams": "CAIQxjgiEwjYkeTCtfzxAhUqQnoFHewJDHA=", "dialogMessages": [ { "runs": [ { "text": "This video may be inappropriate for some users." } ] } ], "confirmButton": { "buttonRenderer": { "style": "STYLE_BLUE_TEXT", "size": "SIZE_DEFAULT", "isDisabled": false, "text": { "runs": [ { "text": "Sign in" } ] }, "trackingParams": "CAQQ8FsiEwjYkeTCtfzxAhUqQnoFHewJDHA=" } }, "cancelButton": { "buttonRenderer": { "style": "STYLE_BLUE_TEXT", "size": "SIZE_DEFAULT", "isDisabled": false, "text": { "runs": [ { "text": "Go back" } ] }, "trackingParams": "CAMQ8FsiEwjYkeTCtfzxAhUqQnoFHewJDHA=" } } } }, "desktopLegacyAgeGateReason": 1, "reasonTitle": "You must sign in to view this video", "contextParams": "Q0FFU0FnZ0M=" }, "videoDetails": { "videoId": "q4qWMcQfOCc", "title": "Battlefield 2042 | Battlefield Portal Official Trailer", "lengthSeconds": "174", "keywords": [ "Battlefield", "Battlefield Portal", "Battlefield 2042", "Battlefield 2042 trailer", "Battlefield 2042 gameplay", "Battlefield 2042 official trailer", "Battlefield 2042 reveal trailer", "Battlefield 6", "Battlefield 6 trailer", "Battlefield 6 gameplay", "Battlefield 6 official trailer", "Battlefield 6 reveal trailer", "Battlefield V", "battlefield 5", "Battlefield buy", "Battlefield trailer", "Battlefield official trailer", "Battlefield 4", "Battlefield 1", "Battlefield series", "All Battlefield games" ], "channelId": "UCvNBXWGykQrWb7kPAn5eLUQ", "isOwnerViewing": false, "shortDescription": "Play Battlefield™ 2042 starting October 22, 2021\n\nDiscover Battlefield Portal – an all-new, community-driven platform where players will be able to change the rules of war by creating, sharing and discovering unexpected battles from Battlefield’s iconic past, present, and future.\n\nBattlefield 2042 is a first-person shooter that marks the return to the iconic all-out warfare of the franchise. Adapt and overcome in a near-future world transformed by disorder. Squad up and bring a cutting-edge arsenal into dynamically-changing battlegrounds supporting 128 players*, unprecedented scale, and epic destruction.\n\nPre-order now for early Open Beta access and more: http://x.ea.com/68944 \n\nFind out more about Battlefield: http://x.ea.com/68674\n\nSee the Battlefield 2042 Official Reveal Trailer: https://www.youtube.com/watch?v=ASzOzrB-a9E\n\nSee the Battlefield 2042 Official Gameplay Trailer: https://www.youtube.com/watch?v=WomAGoEh-Ss \n\nFollow us on Twitter: https://twitter.com/battlefield\nFollow us on Instagram: https://www.instagram.com/battlefield\nFollow us on Facebook: https://www.facebook.com/battlefield\n\n#Battlefield #Battlefield2042 #BattlefieldPortal\n*Supported on Xbox Series X|S, PC and PlayStation®5. Xbox One and PlayStation®4 will be locked at 64 players\n\n“WAR”\nOriginal song by: IDLES", "isCrawlable": true, "thumbnail": { "thumbnails": [ { "url": "https://i.ytimg.com/vi_webp/q4qWMcQfOCc/default.webp", "width": 120, "height": 90 }, { "url": "https://i.ytimg.com/vi_webp/q4qWMcQfOCc/mqdefault.webp", "width": 320, "height": 180 }, { "url": "https://i.ytimg.com/vi_webp/q4qWMcQfOCc/hqdefault.webp", "width": 480, "height": 360 }, { "url": "https://i.ytimg.com/vi_webp/q4qWMcQfOCc/sddefault.webp", "width": 640, "height": 480 } ] }, "averageRating": 4.9406896, "allowRatings": true, "viewCount": "2843524", "author": "Battlefield", "isPrivate": false, "isUnpluggedCorpus": false, "isLiveContent": false }, "playerSettingsMenuData": { "loggingDirectives": { "trackingParams": "CAEQtc4GIhMI2JHkwrX88QIVKkJ6BR3sCQxw", "visibility": { "types": "12" } } } }

RealDeviil commented 3 years ago

This is an age-restricted video, but using only X-Goog-Api-Key header, I do not get any download URLs in the JSON response. I get video info, but no download URLs in that info. Do you?

You have to use an embedded client:

POST /youtubei/v1/player HTTP/1.1
Host: www.youtube.com
X-Goog-Api-Key: AIzaSyAO_FJ2SlqU8Q4STEHLGCilw_Y9_11qcW8

{"context":{"client":{"clientName":"ANDROID_EMBEDDED_PLAYER",
"clientVersion":"16.05"}},"videoId":"q4qWMcQfOCc"}

This works fine. It's unfortunate that (it seems like) the majority of age-restricted videos now require a username and password for authorization. I'd much prefer to continue to download these videos as anonymous users, since -- unless you are using youtube-dl for your own personal, private use -- you otherwise have to require login credentials from people who may or may not want to volunteer that information. Or, you might generate a single login cookie file for all users consuming a given application of youtube-dl. In any case, logging in users just complicates the process. So, maybe someone can come up with a solution that is similar to the current (nonfunctioning) youtube-dl solution. That would be ideal.

I guess, to that end, maybe we just need to find another client and client version that can unlock other age-restricted videos? But, I expect it's just a matter of time before all clients and versions no longer work for this purpose.

So, maybe there is some other parameter and value we can pass in the request to indicate that the user is 18+ years old. I will keep looking.

coletdjnz commented 3 years ago

New tier 2 age-gate bypass using the new API (no auth required!): https://github.com/yt-dlp/yt-dlp/issues/574#issuecomment-887171136

AnonymousWebHacker commented 2 years ago

See https://github.com/Chocobozzz/PeerTube/issues/4837#issuecomment-1061259483

dirkf commented 2 years ago

See https://github.com/ytdl-org/youtube-dl/issues/30255#issuecomment-974822885.