Open levis-ineptias opened 2 months ago
Hi, @bashonly. Were you not able to reproduce the issue on your end? Is there anything I can do to help myself get a solution for this issue? Thank you.
[vimeo] XXXXXXXXX: Got HTTP Error 403 when using impersonate target "chrome-110:windows-10". If you are using a data center IP or VPN/proxy, your IP may be blocked; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
I'm trying to download about two dozen videos from Vimeo. I get this 403 error seemingly more often when I use yt-dlp 'vimeo.com/xxxxxxxx' --recode mp4
but I'm getting it frequently when just using yt-dlp 'vimeo.com/xxxxxxxxx'
as well.
I was able to download successfully until a few minutes ago. Now, when I use the same command that I have been using all along, I get the error
Is this happening every time now? Seems like either an IP block or possibly a rate-limit. I still can't reproduce it on my end.
I wonder how they are differentiating between yt-dlp traffic and web browser traffic if not by TLS fingerprinting. Maybe they are requiring newer user agents?
Hi, @bashonly. Thanks for the response.
Seems like an either an IP block or possibly a rate-limit.
Yes, I think it must be that. I am able to download the videos again now. I gave it a day's break and tried again yesterday, and I was able to download.
Maybe they are requiring newer user agents?
I am not sure what this means; I am new to all of this, so I am still learning.
Thanks once again for your reply 🙂
Is this happening every time now? Seems like either an IP block or possibly a rate-limit.
I connected to a VPN and the first download I tried gave me 403.
Seems likely that they are blocking DC/VPN/proxy IPs. BTW if you are new commenter to this issue, please include a verbose log with your report
[debug] Command-line config: ['-vU', '--referer', 'https://www.XXXXXXXXXXXXXXXXXXXXXXXXX', 'https://player.vimeo.com/video/XXXXXXXXXXXX']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.07.09 from yt-dlp/yt-dlp [7ead7332a] (pip)
[debug] Python 3.12.3 (CPython AMD64 64bit) - Windows-11-10.0.22631-SP0 (OpenSSL 3.0.13 30 Jan 2024)
[debug] exe versions: ffmpeg N-115582-gd55f5cba7b-20240606 (setts), ffprobe N-115582-gd55f5cba7b-20240606
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.06.02, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.45.1, urllib3-2.2.1, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1834 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.07.09 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.07.09 from yt-dlp/yt-dlp)
[vimeo] Extracting URL: https://player.vimeo.com/video/XXXXXXXXXXXXXXX
[vimeo] XXXXXXXXXXX: Downloading webpage
ERROR: [vimeo] XXXXXXXXXXXXXX: Got HTTP Error 403 when using impersonate target "chrome-110:windows-10". If you are using a data center IP or VPN/proxy, your IP may be blocked; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "C:\Users\XXXXXXXXXX\AppData\Local\Programs\Python\Python312\Lib\site-packages\yt_dlp\extractor\common.py", line 740, in extract
ie_result = self._real_extract(url)
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\XXXXXXXXXX\AppData\Local\Programs\Python\Python312\Lib\site-packages\yt_dlp\extractor\vimeo.py", line 859, in _real_extract
raise ExtractorError(
Before using the VPN, I got one video to work, and the second one gave 403.
@levis-ineptias @highschooldropout @jwidness
does adding -4
to your download command help at all? if not, then you could try -6
instead
With -4
, it gives the 429 error: Got HTTP Error 429 when using impersonate target "chrome-124:macos-14".
With -6
, it fails to even connect to github for the latest release information:
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
ERROR: Unable to obtain version info (<urllib3.connection.HTTPSConnection object at 0x00000200E8B83BC0>: Failed to resolve 'api.github.com' ([Errno 11001] getaddrinfo failed)); Please try again later or visit https://github.com/yt-dlp/yt-dlp/releases/latest
Howdy I've been watching this issue and just tried the -4
and it worked for two videos before giving me the error again.
[debug] Command-line config: ['-vU', '-4', '-f', 'bestvideo[ext=mp4]+bestaudio[ext=mp4]', '--refer', 'https://xxxxxxxxxxxxxxxxxxxx', 'https://player.vimeo.com/video/xxxxxxxxxxxxxxxxxxxxxx']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.07.16 from yt-dlp/yt-dlp [89a161e8c] (win_exe)
[debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.22631-SP0 (OpenSSL 1.1.1k 25 Mar 2021)
[debug] exe versions: ffmpeg N-116165-gcd2f8a22e9-20240707 (setts), ffprobe N-116165-gcd2f8a22e9-20240707
[debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1829 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: stable@2024.07.16 from yt-dlp/yt-dlp
yt-dlp is up to date (stable@2024.07.16 from yt-dlp/yt-dlp)
[vimeo] Extracting URL: https://player.vimeo.com/video/xxxxxxxxxxxxxxxxxxxxxx
[vimeo] xxxxxxxxx: Downloading webpage
ERROR: [vimeo] xxxxxxxxx: Got HTTP Error 429 when using impersonate target "chrome-110:windows-10". If you are using a data center IP or VPN/proxy, your IP may be blocked; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "yt_dlp\extractor\common.py", line 740, in extract
File "yt_dlp\extractor\vimeo.py", line 859, in _real_extract
This appears to be specifically related to yt-dlp, because in browser, or using streamlink it's fine. But 429 here using yt-dlp.
This appears to be specifically related to yt-dlp, because in browser, or using streamlink it's fine. But 429 here using yt-dlp.
What would really help the yt-dlp devs solve this problem is if people gave examples and shared URLs/logs/etc when claims like this are made.
Example: ./youtube-dl --force-ipv4 https://vimeo.com/988592349 --dump-json
ERROR: [vimeo] 988592349: Got HTTP Error 429 when using impersonate target "chrome-124:macos-14". If you are using a data center IP or VPN/proxy, your IP may be blocked; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
example: yt-dlp.exe --all-subs --write-info-json --write-description --write-thumbnail --add-metadata --embed-subs --embed-chapters https://player.vimeo.com/video/176280394 -referer https://www.************.com
I have a lot of vimeo IDs which do not work. I tried it 3 evenings in a row. Some of them download a day later. Some not
Try with --impersonate="chrome"
--impersonate="chrome"
didn't change anything for me, still 429
I'm using the python library (version yt-dlp==2024.7.25
) and I was fine for a while and then things weren't. I did a little investigating and experimenting. What it currently looks like is that:
chrome-110:windows-10
, regardless of the flags provided.Some example code:
import yt_dlp
from yt_dlp import ImpersonateTarget
import traceback
from random import choice
impersonate_target_list = (
# ImpersonateTarget(client='chrome', version='110', os='windows', os_version='10'),
ImpersonateTarget(client='chrome', version='107', os='windows', os_version='10'),
ImpersonateTarget(client='chrome', version='104', os='windows', os_version='10'),
ImpersonateTarget(client='chrome', version='101', os='windows', os_version='10'),
ImpersonateTarget(client='chrome', version='100', os='windows', os_version='10'),
ImpersonateTarget(client='chrome', version='99', os='windows', os_version='10'),
ImpersonateTarget(client='edge', version='101', os='windows', os_version='10'),
ImpersonateTarget(client='edge', version='99', os='windows', os_version='10'),
ImpersonateTarget(client='safari', version='15.5', os='macos', os_version='12'),
# ImpersonateTarget(client='safari', version='15.3', os='macos', os_version='11'),
# ImpersonateTarget(client='chrome', version='99', os='android', os_version='12')
ImpersonateTarget.from_str("chrome-99:android-12"),
ImpersonateTarget.from_str("safari-15.3:macos-11"),
)
def random_impersonate_target():
"""
Return a random impersonation target
"""
target: ImpersonateTarget = choice(impersonate_target_list)
return target
def download_vimeo_video(vimeo_url)
for i in range(200):
try:
vimeo_urls = [vimeo_url] # .replace("https")]
impersonate_as = random_impersonate_target()
print(f"Downloading {vimeo_url} as {impersonate_as}")
ydl_opts = {
'format': 'best',
"outtmpl": "%(title)s",
"simulate": False,
"verbose": True,
"continue": True,
"impersonate": impersonate_as,
}
with yt_dlp.YoutubeDL(ydl_opts) as ydl:
ydl.download(vimeo_urls)
except Exception as e:
# Print the traceback object
traceback.print_tb(e.__traceback__)
print(f"Error downloading {vimeo_url}: {str(e)}")
download_vimeo_video("https://vimeo.com/697201324")
Example trace:
Downloading https://vimeo.com/697201324 as safari-15.3:macos-11
[debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version stable@2024.07.25 from yt-dlp/yt-dlp [f0993391e] (pip) API
[debug] params: {'format': <function format_selector at 0x7f5c99b581f0>, 'outtmpl': '%(title)s', 'simulate': False, 'verbose': True, 'continue': True, 'impersonate': ImpersonateTarget(client='safari', version='15.3', os='macos', os_version='11'), 'progress_hooks': [<function download_vimeo_video.<locals>.dl_finished_hook at 0x7f5c98d9c5e0>], 'paths': {'temp': '/mnt/plex2/yoga', 'home': '/mnt/plex2/yoga'}, 'compat_opts': set(), 'http_headers': {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/93.0.4556.0 Safari/537.36', 'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-us,en;q=0.5', 'Sec-Fetch-Mode': 'navigate'}}
[debug] Python 3.10.12 (CPython x86_64 64bit) - Linux-5.15.0-117-generic-x86_64-with-glibc2.35 (OpenSSL 3.0.2 15 Mar 2022, glibc 2.35)
[debug] exe versions: ffmpeg 4.4.2 (setts), ffprobe 4.4.2
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.45.1, requests-2.32.3, secretstorage-3.3.1, sqlite3-3.37.2, urllib3-2.2.2, websockets-12.0
[debug] Proxy map: {}
[debug] Request Handlers: urllib, requests, websockets, curl_cffi
[debug] Loaded 1829 extractors
[vimeo] Extracting URL: https://vimeo.com/697201324
[vimeo] 697201324: Downloading webpage
ERROR: [vimeo] 697201324: Got HTTP Error 429 when using impersonate target "chrome-110:windows-10". If you are using a data center IP or VPN/proxy, your IP may be blocked; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/extractor/common.py", line 740, in extract
ie_result = self._real_extract(url)
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/extractor/vimeo.py", line 886, in _real_extract
raise ExtractorError(
File "/home/zeus/src/yoga_scraper/yoga_scraper.py", line 558, in download_vimeo_video
ydl.download(vimeo_urls)
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3607, in download
self.__download_wrapper(self.extract_info)(
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3582, in wrapper
res = func(*args, **kwargs)
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1615, in extract_info
return self.__extract_info(url, self.get_info_extractor(key), download, extra_info, process)
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1644, in wrapper
self.report_error(str(e), e.format_traceback())
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1092, in report_error
self.trouble(f'{self._format_err("ERROR:", self.Styles.ERROR)} {message}', *args, **kwargs)
File "/home/zeus/.local/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 1031, in trouble
raise DownloadError(message, exc_info)
Error downloading https://vimeo.com/697201324: ERROR: [vimeo] 697201324: Got HTTP Error 429 when using impersonate target "chrome-110:windows-10". If you are using a data center IP or VPN/proxy, your IP may be blocked; please report this issue on https://github.com/yt-dlp/yt-dlp/issues?q= , filling out the appropriate issue template. Confirm you are on the latest version using yt-dlp -U
Notice explicitly how yt-dlp is logging the impersonate target provided 'impersonate': ImpersonateTarget(client='safari', version='15.3', os='macos', os_version='11')
at the beginning, but later the logs show that client doesn't seem to be used - instead, it's always Got HTTP Error 429 when using impersonate target "chrome-110:windows-10".
or Got HTTP Error 403 when using impersonate target "chrome-110:windows-10".
This could just be a logging error - it looks like vimeo.py is explicitly setting impersonate: True
when https
is part of the vimeo URL and false otherwise https://github.com/yt-dlp/yt-dlp/blob/28d485714fef88937c82635438afba5db81f9089/yt_dlp/extractor/vimeo.py#L868
My bet* is that it's easy for Vimeo to identify the yt-dlp traffic because all of us end up using a default impersonate target. Shifting to use v4 temporarily works, but because the same impersonate is repeatedly used, again - easy to identify and block.
impersonate=True
is hardcoded in the extractor, which means use the request handler's default impersonate target.
You actually don't want to pass impersonate
in your params anyways; this would force curl_cffi as the request handler for downloading too, which is prone to segfaults/crashes/etc
FWIW, I've tried to repro this issue by running yt-dlp repeatedly with the default impersonate target and I never get 403/429
Ah - alright. I went into the vimeo.py code locally, commented out the impersonate flag and it respected the impersonate target I provided, but to your point - the impersonate target has further implications when provided than I realized. If we can't change the impersonate target on the initial metadata request, it would be nice if a random one was selected rather than the first one. (e.g. perhaps add an option of random_impersonate_target
?)
My contention is that using the same user agent repeatedly from the same IP is what gets us found and 429-ed. If multiple user agents come through, my bet is they respect that more - what if it's a coffee shop behind that IP for example.
Where I've been getting blocked is fetching the initial metadata - but if I get video metadata, then the download works, which makes sense. I would highly recommend running my code without the impersonate
flag), which will just loop over and over on the same video. Let it run the loop and if you run it over and over again, you should see the error. It's not the video download which seems to get the 429 - it's the metadata fetch. You only need to download the video once, repeated fetches of metadata for a video I already had is where they got me. I was using simulate
at that point when I was blocked.
I didn't realize that using curl_cffi as the downloader was an issue, really good to know. Let me know if running my code (perhaps a few times) allows you to reproduce the issue.
I just got the 429 locally with impersonation. I went to the browser and I got a captcha screen (which explains why yt-dlp couldn't bypass it)
However, once I solved the captcha, I re-ran the exact same yt-dlp command on the same IP (no cookies or anything needed from the browser) and it worked again.
Might be IP-reputation related?
OOO - interesting - let me try solving the captcha
JFYI, what @coletdjnz said has been the case for at least months. I had a personal project to scrape some vimeo metadata from webpages and have noticed that for a while.
You usually trigger the captcha by having too many threads (>10, or even fewer if your Internet is fast). After solving it your reputation would be raised back to normal until you did the same thing again. Repeatedly doing so would got your IP blocked "permanently" (not actually permanent usually).
Alright - so this is interesting. I have 2 computers on my home network - ubuntu server and a mac. When I run from my mac - not blocked (regardless of impersonate or not). When I run from Ubuntu - blocked unless my impersonate target is mac/safari (with my hack in place around allowing impersonation to be proxied through to the metadata request). On Ubuntu server - I can't open a web browser to do a Captcha and I don't get a captcha on the laptop - video just loads.
You should try to dump the HTML (--dump-pages
I think? not very familiar with this area) on your Ubuntu machine and see what exactly it says.
Originally, I didn't think there was harm in running in simulate
mode, so I just hit vimeo 400+ times and later once done with development, I turned simulate off and let yt-dlp determine that the file was there. When I started seeing the blocks, I added a caching layer, but I think I did it too late in the game.
I absolutely can give dump-pages a shot.
--write-pages
and --dump-pages
don't do anything if the request errors. You'd need to patch the Vimeo extractor like so:
diff --git a/yt_dlp/extractor/vimeo.py b/yt_dlp/extractor/vimeo.py
index d10689cd8..b0fb3d889 100644
--- a/yt_dlp/extractor/vimeo.py
+++ b/yt_dlp/extractor/vimeo.py
@@ -874,6 +874,7 @@ def _real_extract(self, url):
except ExtractorError as error:
if not isinstance(error.cause, HTTPError) or error.cause.status not in (403, 429):
raise
+ self._webpage_read_content(error.cause.response, url, video_id)
errmsg = error.cause.response.read()
if b'Because of its privacy settings, this video cannot be played here' in errmsg:
raise ExtractorError(
and --write-pages
might be more useful than --dump-pages
here
Indeed! After patching in that code (thanks @bashonly) I've got a Captcha challenge. It's interesting that both of these are using the same source IP, but there's different behavior.
<!DOCTYPE html>
<html lang="en">
<head>
<meta name="robots" content="noindex, nofollow" />
<title>Vimeo / CAPTCHA Challenge</title>
<style type="text/css">
body {
background: #141531;
font-family: helvetica, arial, san-serif;
font-size: 18px;
font-weight: normal;
color: #fff;
}
p {
margin: 0 0 25px 0;
line-height: 1.2;
margin: 0 0 25px 0;
color: #eee;
}
svg {
margin: 75px;
width: 250px;
}
div.content {
margin: 75px;
position: absolute;
bottom: 0px;
}
a,
a:visited {
color: #2196f3;
}
a:hover {
color: #64b5f6;
}
code {
color: #fff;
font-weight: bold;
font-family: monospace;
}
html,
body {
height: 100%;
margin: 0;
overflow: hidden;
}
body {
transform-origin: 0 0;
transform: scale(1);
}
body > * {
transform: scale(1);
}
div.content {
position: relative;
bottom: auto;
margin: 25px;
}
svg {
margin: 25px;
}
body {
display: flex;
flex-direction: column;
justify-content: space-between;
align-items: flex-start;
height: 100%;
margin: 0;
overflow: hidden;
}
svg {
margin: 75px;
}
.content {
margin: 75px;
}
#turnstile-wrapper {
margin-bottom: 25px;
}
#loading-indicator {
display: block;
color: #fff;
text-align: center;
}
</style>
</head>
<body>
<svg
xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 140 40"
preserveAspectRatio="xMinYMin none"
>
<path
fill="#fff"
d="M31.277 18.832c-.14 3.052-2.27 7.23-6.39 12.53-4.26 5.537-7.863 8.307-10.81 8.307-1.826 0-3.372-1.69-4.634-5.06-.843-3.1-1.686-6.19-2.53-9.28-.937-3.37-1.942-5.06-3.018-5.06-.234 0-1.054.49-2.458 1.473l-1.474-1.9c1.546-1.36 3.07-2.718 4.572-4.08 2.062-1.78 3.61-2.72 4.642-2.813 2.438-.234 3.938 1.433 4.502 5 .6 3.85 1.03 6.247 1.26 7.183.7 3.194 1.472 4.79 2.32 4.79.66 0 1.64-1.037 2.95-3.108 1.31-2.073 2.015-3.65 2.11-4.733.187-1.79-.517-2.684-2.11-2.684-.75 0-1.522.173-2.32.514 1.54-5.044 4.48-7.495 8.825-7.355 3.22.094 4.737 2.183 4.552 6.265z"
/>
<path
fill="#fff"
d="M50.613 28.713c-1.313 2.484-3.12 4.733-5.417 6.748-3.143 2.72-6.285 4.08-9.425 4.08-1.45 0-2.57-.47-3.34-1.4-.77-.937-1.13-2.152-1.09-3.652.05-1.547.53-3.937 1.44-7.172.92-3.23 1.38-4.966 1.38-5.2 0-1.22-.422-1.83-1.265-1.83-.282 0-1.08.496-2.393 1.48l-1.62-1.9 4.5-4.08c2.02-1.784 3.52-2.72 4.507-2.814 1.546-.14 2.684.314 3.41 1.367.726 1.05.996 2.415.81 4.096-.61 2.85-1.27 6.47-1.972 10.863-.046 2.01.68 3.014 2.182 3.014.655 0 1.826-.69 3.517-2.08 1.406-1.156 2.555-2.243 3.447-3.26l1.336 1.753zm-6.12-25.016c-.047 1.168-.633 2.288-1.76 3.36-1.266 1.213-2.767 1.82-4.5 1.82-2.673 0-3.964-1.165-3.87-3.498.045-1.22.76-2.39 2.144-3.51C37.89.75 39.427.19 41.117.19c.983 0 1.804.388 2.46 1.156.657.772.962 1.553.916 2.344z"
/>
<path
fill="#fff"
d="M94.543 28.713c-1.314 2.484-3.117 4.733-5.416 6.748-3.145 2.72-6.285 4.08-9.426 4.08-3.05 0-4.52-1.68-4.43-5.06.05-1.5.34-3.3.88-5.41.54-2.11.83-3.75.88-4.92.05-1.78-.49-2.673-1.62-2.673-1.222 0-2.68 1.456-4.374 4.36-1.79 3.05-2.754 6.005-2.894 8.86-.095 2.02.103 3.57.59 4.648-3.27.097-5.563-.443-6.87-1.616-1.17-1.033-1.71-2.743-1.615-5.136.045-1.5.276-3 .69-4.502.414-1.5.644-2.836.69-4.01.095-1.734-.54-2.604-1.9-2.604-1.177 0-2.444 1.34-3.806 4.01-1.36 2.675-2.113 5.467-2.253 8.373-.094 2.624.074 4.453.503 5.483-3.22.096-5.506-.582-6.858-2.035-1.12-1.214-1.632-3.06-1.54-5.54.047-1.214.26-2.91.648-5.084.39-2.175.607-3.87.65-5.087.094-.843-.12-1.265-.632-1.265-.28 0-1.08.476-2.393 1.425l-1.686-1.9c.232-.184 1.71-1.545 4.43-4.08 1.97-1.826 3.304-2.764 4.01-2.81 1.216-.095 2.2.41 2.95 1.51s1.13 2.38 1.13 3.835c0 .47-.05.915-.14 1.336.7-1.077 1.52-2.017 2.46-2.814 2.155-1.875 4.57-2.93 7.244-3.167 2.3-.19 3.94.35 4.926 1.615.795 1.033 1.17 2.51 1.125 4.433.33-.28.68-.586 1.057-.915 1.08-1.265 2.135-2.27 3.166-3.02 1.737-1.268 3.54-1.97 5.42-2.113 2.25-.188 3.87.35 4.854 1.61.843 1.03 1.22 2.5 1.126 4.415-.044 1.31-.36 3.213-.946 5.712-.588 2.5-.88 3.934-.88 4.31-.048.98.048 1.66.28 2.032.236.373.797.56 1.69.56.655 0 1.825-.694 3.517-2.084 1.403-1.156 2.552-2.243 3.444-3.262l1.336 1.757z"
/>
<path
fill="#fff"
d="M120.922 28.642c-1.36 2.25-4.033 4.495-8.02 6.743-4.97 2.856-10.012 4.284-15.125 4.284-3.797 0-6.52-1.27-8.16-3.8-1.172-1.74-1.734-3.8-1.688-6.19.04-3.8 1.73-7.41 5.06-10.83 3.66-3.75 7.97-5.63 12.94-5.63 4.595 0 7.032 1.87 7.313 5.61.19 2.383-1.125 4.84-3.938 7.367-3.005 2.76-6.78 4.515-11.33 5.263.843 1.17 2.11 1.753 3.8 1.753 3.375 0 7.06-.856 11.045-2.575 2.86-1.205 5.11-2.46 6.753-3.76l1.338 1.756zm-15.97-7.345c.046-1.26-.468-1.89-1.546-1.89-1.406 0-2.83.97-4.283 2.906-1.45 1.936-2.2 3.79-2.248 5.562-.025 0-.025.305 0 .91 2.295-.838 4.287-2.12 5.97-3.848 1.358-1.49 2.06-2.707 2.108-3.64z"
/>
<path
fill="#fff"
d="M140.018 23.926c-.19 4.31-1.78 8.03-4.783 11.17-3.002 3.136-6.73 4.705-11.186 4.705-3.71 0-6.52-1.19-8.45-3.58-1.405-1.772-2.18-4-2.32-6.67-.237-4.03 1.216-7.73 4.36-11.1 3.38-3.74 7.62-5.62 12.732-5.62 3.28 0 5.766 1.11 7.457 3.3 1.594 2.02 2.32 4.62 2.18 7.8zm-7.95-.264c.047-1.27-.13-2.434-.527-3.49-.4-1.057-.97-1.587-1.72-1.587-2.39 0-4.36 1.293-5.9 3.877-1.316 2.115-2.02 4.37-2.11 6.766-.05 1.176.163 2.21.632 3.104.514 1.032 1.24 1.55 2.18 1.55 2.11 0 3.915-1.245 5.417-3.736 1.267-2.068 1.945-4.23 2.04-6.484z"
/>
</svg>
<div class="content">
<h1>Verify to continue</h1>
<p>
To continue, please confirm that you're a human (and not a spambot).
</p>
<div class="main-wrapper" role="main">
<div class="main-content">
<h2 id="challenge-running">
Checking if the site connection is secure
</h2>
<div id="challenge-stage">
<div id="loading-indicator"></div>
<div id="turnstile-wrapper"></div>
</div>
<div id="challenge-body-text">
vimeo.com needs to review the security of your connection before
proceeding.
</div>
<div class="footer"></div>
</div>
</div>
<script>
function turnstileLoad() {
const turnstileWrapper = document.getElementById("turnstile-wrapper");
const loadingIndicator = document.getElementById("loading-indicator");
turnstile.render("#turnstile-wrapper", {
sitekey: "0x4AAAAAAAe0ufFllHGZPwDJ",
"error-callback": function () {
loadingIndicator.textContent = "";
},
callback: function (token) {
if (token) {
const url = new URL(window.location.href);
url.searchParams.set("turnstile", encodeURIComponent(token));
window.location.href = url.toString() + window.location.hash;
} else {
loadingIndicator.textContent = "";
}
},
});
}
window.addEventListener("load", turnstileLoad);
</script>
<script
src="https://challenges.cloudflare.com/turnstile/v0/api.js?onload=turnstileLoad"
async
defer
></script>
</div>
</body>
</html>
Since this isn't based entirely on source IP (given the different behavior of the machines on my network), I'm thinking that there's got to be something I can do to bust through this.
I'm mostly trying to be responsive since I'm able to repeatedly replicate the behavior - I definitely yield to y'all as the domain experts. Let me know if there's anything you'd like me to try.
What I'm almost wondering is - maybe I setup some weird tunnel from my program where I proxy my browser traffic through the affected node so I can click the captchas when they occur.
I proxied my computer through my server by opening a SOCK5 proxy.
/etc/ssh/sshd_config
, ensure:
AllowTcpForwarding yes
PermitTunnel yes
sudo systemctl restart sshd
ssh -D 8080 user@server
localhost
on port 8080
This doesn't scale obviously, but just if you're blocked from the CLI - try to get it open in a browser which is the same as your impersonation target.
I started getting the 429 error a few days ago, basically everytime I tried. However I found a fix: it appears that if you have the same URL open with active playback, in any browser, the error doesn't show up.
Beware you have to keep the video playing until the download if finished, otherwise it will start timing out.
Hope it helps someone else while the DEVs manage to fix the issue - if it's possible at all. BTW thanks for this awesome piece of sofwrare! Cheers!
I found a fix - for now!
I started getting the 429 error a few days ago, basically everytime I tried. However I found a fix: it appears that if you have the same URL open with active playback, in any browser, the error doesn't show up.
Beware you have to keep the video playing until the download if finished, otherwise it will start timing out.
Hope it helps someone else while the DEVs manage to fix the issue - if it's possible at all. BTW thanks for this awesome piece of sofwrare! Cheers!
Not working for me. I try to download private video which is embedded to a website, where I have paid access to the content. Even when the video is running same ID gets 429. What I can say: my list of non working IDs gets shorter and shorter day by day. Each day I try to download them. Most get 429 (I use imersonate=safari) and fault says target "chrome-110:windows-10". But some start to download. no parallel downloads, 1Sec delay between attempts
I just got the 429 locally with impersonation. I went to the browser and I got a captcha screen (which explains why yt-dlp couldn't bypass it)
However, once I solved the captcha, I re-ran the exact same yt-dlp command on the same IP (no cookies or anything needed from the browser) and it worked again.
Might be IP-reputation related?
I wanted to chime in to say, I just got this same "broken" captcha screen on 1fichier after being advised by Safari with that security screen saying it was running through an insecure connection and might be trying to impersonate the original site and so on. Then, on https://vimeo.com/522536765 I'm getting the error everyone else is getting. Something Cloudfare related?
My arguments were: --extractor-args "youtube:player_client=ios" --cookies-from-browser safari "
not sure if would have changed had I not added that youtube part which I did some days ago to get around another issue with YouTube not downloading certain videos. But now it fails without any arguments too so I guess I've been blocked more broadly.
edit: make this of what you will but a bunch of other services have stopped working for me as well (a Deezer downloader, an IUP image host)
I found a fix - for now!
I started getting the 429 error a few days ago, basically everytime I tried. However I found a fix: it appears that if you have the same URL open with active playback, in any browser, the error doesn't show up. Beware you have to keep the video playing until the download if finished, otherwise it will start timing out. Hope it helps someone else while the DEVs manage to fix the issue - if it's possible at all. BTW thanks for this awesome piece of sofwrare! Cheers!
Not working for me. I try to download private video which is embedded to a website, where I have paid access to the content. Even when the video is running same ID gets 429. What I can say: my list of non working IDs gets shorter and shorter day by day. Each day I try to download them. Most get 429 (I use imersonate=safari) and fault says target "chrome-110:windows-10". But some start to download. no parallel downloads, 1Sec delay between attempts
I did it by right-clicking on the Vimeo Frame, than from the menu > view Frame source, than in the source code you find the player embed URL. If you open that link in another tab, it should play the video. While this video is playing, it works for me.
I found a fix - for now!
I started getting the 429 error a few days ago, basically everytime I tried. However I found a fix: it appears that if you have the same URL open with active playback, in any browser, the error doesn't show up.
Beware you have to keep the video playing until the download if finished, otherwise it will start timing out.
Hope it helps someone else while the DEVs manage to fix the issue - if it's possible at all. BTW thanks for this awesome piece of sofwrare! Cheers!
That did indeed work for me! It was my first ever attempt to use yt-dlp, and I didn't know if I was doing anything wrong, but seems like Vimeo started detecting the yt-dlp traffic. But parallel to playing in the browser works!
I found a solution for my missing private vimeo IDs. What I get: if I use "player.vimeo.com/video/VIMEOID1" I get HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>) if I use "vimeo.com/VIMEOID1" on the same ID, I get Got HTTP Error 429 when using impersonate target "chrome-110:windows-10". If you are using a data center IP or VPN/proxy, your IP may be blocked; After that I open vimeo.com/VIMEOID1 in a incognito chrome browser and I see the capture. the forwarding leads to nowhere, because the video ID is private. After that I can download from "player.vimeo.com/video/VIMEOID1" For the next ID I need to close and reopen incognito Chrome. After I browsed to vimeo.com/VIMEOID2 it works with yt-dlp downloading from "player.vimeo.com/video/VIMEOID2"
--impersonate="chrome"
didn't change anything for me, still 429
Yes, you are right, it worked only for a wile also for me. Apparently vimeo accepts (for the moment) this browser identification (as extracted with httpTracker from Chrome where I could download the m3u playlist for the video), it would be nice if the dev could allow in the future some combination of flags (I read that it is useless to use --impersonate="chrome" since it is invalidated by an internal flag that can not be set when using the compiled yt-dlp.exe (BTW, it is extremely cumbersome to tweak each time the .py files...):
sec-ch-ua | "Not)A;Brand";v="99", "Google Chrome";v="127", "Chromium";v="127" sec-ch-ua-mobile ?0 sec-ch-ua-platform "Windows"
so it would be nice to have two flags for the compiled version, one --impersonate=True and second --impersonateVersion="xxxx" where xxx could be entered manually according to the current chrome/chromium version that is accepted by vimeo, in this case it would be --impersonateVersion="chrome-127:windows-10" BTW in the http Tracker log I see sec-ch-ua-platform "Windows" not "Windows-10"...
Thanks
Based on reports like https://github.com/yt-dlp/yt-dlp/issues/10422#issuecomment-2256124424, it seems that browser impersonation is only half the battle (not to mention that our available impersonation targets will almost never include the latest browser versions). There may be some sort of required heartbeat or traffic analysis going on as well.
Also, it seems that only the webpage requests are impacted by these 429s/403s; API requests seem to be working just fine. If we could find a way to get embed-only videos (the player.vimeo.com
URLs which require a referer header) via API calls only, then I think only using the API and not bothering with the webpages would be the ideal solution.
You're right, getting the embed-only videos via api calls could solve individual videos retrieval but imagine that one of the most used features of yt-dlp on vimeo channels with many videos is the --dump-json feature on the channel with extraction of almost all the useful features (including subtitles) and this feature is broken now. The 429 error is what I get at almost all requests now. Not mentioning the error again on each individual video one at a time :(. So a comprehensive resolution that would restore the also previous functionality on non-embed videos / cchannels would be great. Thanks in advance.
Non-embed videos (vimeo.com/ID
URLs) can already be extracted only via API without any loss of metadata/subtitles/etc.
I haven't looked into full channel extraction yet, but is there any indication that impersonation isn't sufficient? It has not been reported as an issue.
Well I just got two different behaviours on using --dump-json on vimeo channels:
1) Piping the result of --dump-json on jq dies EXACTLY after 16 videos with 429 but in this case with "Too many requests" and also freezes the DOS window wheb using Ctrl+C
O:\cnn>yt-dlp -vU --dump-json https://vimeo.com/cnn | jq -r "[.id,.duration,.title,.original_url] + (.subtitles.\"en-US\"[2] | [.url]) |@csv" >> cnn.csv [debug] Command-line config: ['-vU', '--dump-json', 'https://vimeo.com/cnn'] [debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out cp1252 (No VT), error utf-8, screen utf-8 [debug] yt-dlp version stable@2024.08.01 from yt-dlp/yt-dlp [ffd7781d6] (win_exe) [debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k 25 Mar 2021) [debug] exe versions: ffmpeg 2024-06-09-git-94f2274a8b-full_build-www.gyan.dev (setts), ffprobe 2024-06-09-git-94f2274a8b-full_build-www.gyan.dev, phantomjs 2.1.1 [debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0 [debug] Proxy map: {} [debug] Request Handlers: urllib, requests, websockets, curl_cffi [debug] Loaded 1830 extractors [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest Latest version: stable@2024.08.01 from yt-dlp/yt-dlp yt-dlp is up to date (stable@2024.08.01 from yt-dlp/yt-dlp) [vimeo:user] Extracting URL: https://vimeo.com/cnn [vimeo:user] cnn: Downloading page 1 [download] Downloading playlist: CNN [vimeo:user] cnn: Downloading page 2 [vimeo:user] cnn: Downloading page 3 [vimeo:user] cnn: Downloading page 4 [vimeo:user] cnn: Downloading page 5 [vimeo:user] cnn: Downloading page 6 [vimeo:user] cnn: Downloading page 7 [vimeo:user] cnn: Downloading page 8 [vimeo:user] cnn: Downloading page 9 [vimeo:user] cnn: Downloading page 10 [vimeo:user] Playlist CNN: Downloading 111 items of 111 [download] Downloading item 1 of 111 [vimeo] Extracting URL: https://vimeo.com/137624463 [vimeo] 137624463: Downloading webpage [vimeo] 137624463: Downloading JSON metadata [vimeo] 137624463: Downloading JSON metadata [vimeo] 137624463: Downloading jwt token [vimeo] 137624463: Downloading JSON metadata [vimeo] 137624463: Downloading akfire_interconnect_quic m3u8 information [vimeo] 137624463: Downloading fastly_skyfire m3u8 information [vimeo] 137624463: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 137624463: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 137624463: Downloading fastly_skyfire MPD information WARNING: [vimeo] 137624463: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 137624463: Downloading 1 format(s): http-1080p [download] Downloading item 2 of 111 [vimeo] Extracting URL: https://vimeo.com/137623605 [vimeo] 137623605: Downloading webpage [vimeo] 137623605: Downloading JSON metadata [vimeo] 137623605: Downloading JSON metadata [vimeo] 137623605: Downloading jwt token [vimeo] 137623605: Downloading JSON metadata [vimeo] 137623605: Downloading akfire_interconnect_quic m3u8 information [vimeo] 137623605: Downloading fastly_skyfire m3u8 information [vimeo] 137623605: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 137623605: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 137623605: Downloading fastly_skyfire MPD information WARNING: [vimeo] 137623605: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 137623605: Downloading 1 format(s): http-1080p [download] Downloading item 3 of 111 [vimeo] Extracting URL: https://vimeo.com/137154842 [vimeo] 137154842: Downloading webpage [vimeo] 137154842: Downloading JSON metadata [vimeo] 137154842: Downloading JSON metadata [vimeo] 137154842: Downloading jwt token [vimeo] 137154842: Downloading JSON metadata [vimeo] 137154842: Downloading akfire_interconnect_quic m3u8 information [vimeo] 137154842: Downloading fastly_skyfire m3u8 information [vimeo] 137154842: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 137154842: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 137154842: Downloading fastly_skyfire MPD information WARNING: [vimeo] 137154842: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 137154842: Downloading 1 format(s): http-1080p [download] Downloading item 4 of 111 [vimeo] Extracting URL: https://vimeo.com/137151693 [vimeo] 137151693: Downloading webpage [vimeo] 137151693: Downloading JSON metadata [vimeo] 137151693: Downloading JSON metadata [vimeo] 137151693: Downloading jwt token [vimeo] 137151693: Downloading JSON metadata [vimeo] 137151693: Downloading akfire_interconnect_quic m3u8 information [vimeo] 137151693: Downloading fastly_skyfire m3u8 information [vimeo] 137151693: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 137151693: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 137151693: Downloading fastly_skyfire MPD information WARNING: [vimeo] 137151693: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 137151693: Downloading 1 format(s): http-1080p [download] Downloading item 5 of 111 [vimeo] Extracting URL: https://vimeo.com/108930850 [vimeo] 108930850: Downloading webpage [vimeo] 108930850: Downloading JSON metadata [vimeo] 108930850: Downloading JSON metadata [vimeo] 108930850: Downloading jwt token [vimeo] 108930850: Downloading JSON metadata [vimeo] 108930850: Downloading akfire_interconnect_quic m3u8 information [vimeo] 108930850: Downloading fastly_skyfire m3u8 information [vimeo] 108930850: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 108930850: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 108930850: Downloading fastly_skyfire MPD information WARNING: [vimeo] 108930850: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 108930850: Downloading 1 format(s): http-1080p [download] Downloading item 6 of 111 [vimeo] Extracting URL: https://vimeo.com/100808861 [vimeo] 100808861: Downloading webpage [vimeo] 100808861: Downloading JSON metadata [vimeo] 100808861: Downloading JSON metadata [vimeo] 100808861: Downloading jwt token [vimeo] 100808861: Downloading JSON metadata [vimeo] 100808861: Downloading akfire_interconnect_quic m3u8 information [vimeo] 100808861: Downloading fastly_skyfire m3u8 information [vimeo] 100808861: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 100808861: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 100808861: Downloading fastly_skyfire MPD information WARNING: [vimeo] 100808861: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 100808861: Downloading 1 format(s): http-720p [download] Downloading item 7 of 111 [vimeo] Extracting URL: https://vimeo.com/91613782 [vimeo] 91613782: Downloading webpage [vimeo] 91613782: Downloading JSON metadata [vimeo] 91613782: Downloading JSON metadata [vimeo] 91613782: Downloading jwt token [vimeo] 91613782: Downloading JSON metadata [vimeo] 91613782: Downloading akfire_interconnect_quic m3u8 information [vimeo] 91613782: Downloading fastly_skyfire m3u8 information [vimeo] 91613782: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 91613782: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 91613782: Downloading fastly_skyfire MPD information WARNING: [vimeo] 91613782: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 91613782: Downloading 1 format(s): http-720p [download] Downloading item 8 of 111 [vimeo] Extracting URL: https://vimeo.com/91575687 [vimeo] 91575687: Downloading webpage [vimeo] 91575687: Downloading JSON metadata [vimeo] 91575687: Downloading JSON metadata [vimeo] 91575687: Downloading jwt token [vimeo] 91575687: Downloading JSON metadata [vimeo] 91575687: Downloading akfire_interconnect_quic m3u8 information [vimeo] 91575687: Downloading fastly_skyfire m3u8 information [vimeo] 91575687: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 91575687: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 91575687: Downloading fastly_skyfire MPD information WARNING: [vimeo] 91575687: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 91575687: Downloading 1 format(s): http-540p [download] Downloading item 9 of 111 [vimeo] Extracting URL: https://vimeo.com/90026742 [vimeo] 90026742: Downloading webpage [vimeo] 90026742: Downloading JSON metadata [vimeo] 90026742: Downloading JSON metadata [vimeo] 90026742: Downloading jwt token [vimeo] 90026742: Downloading JSON metadata [vimeo] 90026742: Downloading akfire_interconnect_quic m3u8 information [vimeo] 90026742: Downloading fastly_skyfire m3u8 information [vimeo] 90026742: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 90026742: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 90026742: Downloading fastly_skyfire MPD information WARNING: [vimeo] 90026742: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 90026742: Downloading 1 format(s): http-720p [download] Downloading item 10 of 111 [vimeo] Extracting URL: https://vimeo.com/89404004 [vimeo] 89404004: Downloading webpage [vimeo] 89404004: Downloading JSON metadata [vimeo] 89404004: Downloading JSON metadata [vimeo] 89404004: Downloading jwt token [vimeo] 89404004: Downloading JSON metadata [vimeo] 89404004: Downloading akfire_interconnect_quic m3u8 information [vimeo] 89404004: Downloading fastly_skyfire m3u8 information [vimeo] 89404004: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 89404004: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 89404004: Downloading fastly_skyfire MPD information WARNING: [vimeo] 89404004: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 89404004: Downloading 1 format(s): http-720p [download] Downloading item 11 of 111 [vimeo] Extracting URL: https://vimeo.com/84641059 [vimeo] 84641059: Downloading webpage [vimeo] 84641059: Downloading JSON metadata [vimeo] 84641059: Downloading JSON metadata [vimeo] 84641059: Downloading jwt token [vimeo] 84641059: Downloading JSON metadata [vimeo] 84641059: Downloading akfire_interconnect_quic m3u8 information [vimeo] 84641059: Downloading fastly_skyfire m3u8 information [vimeo] 84641059: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 84641059: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 84641059: Downloading fastly_skyfire MPD information WARNING: [vimeo] 84641059: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 84641059: Downloading 1 format(s): http-720p [download] Downloading item 12 of 111 [vimeo] Extracting URL: https://vimeo.com/84640489 [vimeo] 84640489: Downloading webpage [vimeo] 84640489: Downloading JSON metadata [vimeo] 84640489: Downloading JSON metadata [vimeo] 84640489: Downloading jwt token [vimeo] 84640489: Downloading JSON metadata [vimeo] 84640489: Downloading akfire_interconnect_quic m3u8 information [vimeo] 84640489: Downloading fastly_skyfire m3u8 information [vimeo] 84640489: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 84640489: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 84640489: Downloading fastly_skyfire MPD information WARNING: [vimeo] 84640489: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [debug] Sort order given by extractor: quality, res, fps, hdr:12, source [debug] Formats sorted by: hasvid, ie_pref, quality, res, fps, hdr:12(7), source, lang, vcodec:vp9.2(10), channels, acodec, size, br, asr, proto, vext, aext, hasaud, id [debug] Default format spec: bestvideo+bestaudio/best [info] 84640489: Downloading 1 format(s): http-720p [download] Downloading item 13 of 111 [vimeo] Extracting URL: https://vimeo.com/84561593 [vimeo] 84561593: Downloading webpage [vimeo] 84561593: Downloading JSON metadata ERROR: [vimeo] 84561593: Unable to download JSON metadata: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: Too Many Requests>) File "yt_dlp\extractor\common.py", line 740, in extract File "yt_dlp\extractor\vimeo.py", line 940, in _real_extract File "yt_dlp\extractor\common.py", line 1139, in download_content File "yt_dlp\extractor\common.py", line 1099, in download_handle File "yt_dlp\extractor\common.py", line 960, in _download_webpage_handle File "yt_dlp\extractor\common.py", line 909, in _request_webpage
File "yt_dlp\extractor\common.py", line 896, in _request_webpage File "yt_dlp\YoutubeDL.py", line 4165, in urlopen File "yt_dlp\networking\common.py", line 117, in send File "yt_dlp\networking_helper.py", line 208, in wrapper File "yt_dlp\networking\common.py", line 340, in send File "yt_dlp\networking_requests.py", line 365, in _send yt_dlp.networking.exceptions.HTTPError: HTTP Error 429: Too Many Requests
[download] Downloading item 14 of 111 [vimeo] Extracting URL: https://vimeo.com/84555972 [vimeo] 84555972: Downloading webpage [vimeo] 84555972: Downloading JSON metadata ERROR: [vimeo] 84555972: Unable to download JSON metadata: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: Too Many Requests>) File "yt_dlp\extractor\common.py", line 740, in extract File "yt_dlp\extractor\vimeo.py", line 940, in _real_extract File "yt_dlp\extractor\common.py", line 1139, in download_content File "yt_dlp\extractor\common.py", line 1099, in download_handle File "yt_dlp\extractor\common.py", line 960, in _download_webpage_handle File "yt_dlp\extractor\common.py", line 909, in _request_webpage
File "yt_dlp\extractor\common.py", line 896, in _request_webpage File "yt_dlp\YoutubeDL.py", line 4165, in urlopen File "yt_dlp\networking\common.py", line 117, in send File "yt_dlp\networking_helper.py", line 208, in wrapper File "yt_dlp\networking\common.py", line 340, in send File "yt_dlp\networking_requests.py", line 365, in _send yt_dlp.networking.exceptions.HTTPError: HTTP Error 429: Too Many Requests
[download] Downloading item 15 of 111 [vimeo] Extracting URL: https://vimeo.com/84554995 [vimeo] 84554995: Downloading webpage [vimeo] 84554995: Downloading JSON metadata ERROR: [vimeo] 84554995: Unable to download JSON metadata: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: Too Many Requests>) File "yt_dlp\extractor\common.py", line 740, in extract File "yt_dlp\extractor\vimeo.py", line 940, in _real_extract File "yt_dlp\extractor\common.py", line 1139, in download_content File "yt_dlp\extractor\common.py", line 1099, in download_handle File "yt_dlp\extractor\common.py", line 960, in _download_webpage_handle File "yt_dlp\extractor\common.py", line 909, in _request_webpage
File "yt_dlp\extractor\common.py", line 896, in _request_webpage File "yt_dlp\YoutubeDL.py", line 4165, in urlopen File "yt_dlp\networking\common.py", line 117, in send File "yt_dlp\networking_helper.py", line 208, in wrapper File "yt_dlp\networking\common.py", line 340, in send File "yt_dlp\networking_requests.py", line 365, in _send yt_dlp.networking.exceptions.HTTPError: HTTP Error 429: Too Many Requests
[download] Downloading item 16 of 111 [vimeo] Extracting URL: https://vimeo.com/69480404 [vimeo] 69480404: Downloading webpage [vimeo] 69480404: Downloading JSON metadata ERROR: [vimeo] 69480404: Unable to download JSON metadata: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: Too Many Requests>) File "yt_dlp\extractor\common.py", line 740, in extract File "yt_dlp\extractor\vimeo.py", line 940, in _real_extract File "yt_dlp\extractor\common.py", line 1139, in download_content File "yt_dlp\extractor\common.py", line 1099, in download_handle File "yt_dlp\extractor\common.py", line 960, in _download_webpage_handle File "yt_dlp\extractor\common.py", line 909, in _request_webpage
File "yt_dlp\extractor\common.py", line 896, in _request_webpage File "yt_dlp\YoutubeDL.py", line 4165, in urlopen File "yt_dlp\networking\common.py", line 117, in send File "yt_dlp\networking_helper.py", line 208, in wrapper File "yt_dlp\networking\common.py", line 340, in send File "yt_dlp\networking_requests.py", line 365, in _send yt_dlp.networking.exceptions.HTTPError: HTTP Error 429: Too Many Requests
[download] Downloading item 17 of 111 [vimeo] Extracting URL: https://vimeo.com/69480102 [vimeo] 69480102: Downloading webpage
ERROR: Interrupted by user
2) On other user channels it dies with 429 impersonation from the first request
So it is very probable as you said that vimeo has implemented some kind of traffic monitoring on top of the 429 impersonation trap
BTW is it any way to set a timeout for the --dump-json requests to avoid thefast firing that trggers the 429 Too many requets? Whe n I set a manual delay of 5 secs in a batch file that does the --dump-json on each videos in part there is no 429 Too many requests triggering
[vimeo] 69480404: Downloading JSON metadata ERROR: [vimeo] 69480404: Unable to download JSON metadata: HTTP Error 429: Too Many Requests (caused by <HTTPError 429: Too Many Requests>)
This just seems like you're exceeding the rate-limit, plain and simple. This is coming from the API, too, so the webpage-bypass is irrelevant. And although this is the same HTTP error as what's in this issue's title, it is a distinctly different issue IMO.
BTW is it any way to set a timeout for the --dump-json requests to avoid thefast firing that trggers the 429 Too many requets?
--sleep-requests
, --min-sleep-interval
, --max-sleep-interval
see the README for how to use them
Thanks, sometimes works with --sleep-requests greater than 5 secs, sometimes doesn't. BTW, coming back to somebody's presumption of captcha blocking, look what I found in a footer of a curl request to a embeded vimeo video that when used with yt-dlp AND passing the referer gave a "Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>)"
<script src="https://challenges.cloudflare.com/turnstile/v0/api.js?onload=turnstileLoad" async defer
Is this turnstile feature new or I haven't spotted it in the past?
This is the whole code:
curl --referer https://escuelaplayterapia.com https://player.vimeo.com/video/516390862 <!DOCTYPE html>
And this is the yt-dlp denied request (apparently dies in a curl request as I can see...)
yt-dlp -vU --referer https://escuelaplayterapia.com --sleep-requests 5 -F https://player.vimeo.com/video/516390862 [debug] Command-line config: ['-vU', '--referer', 'https://escuelaplayterapia.com', '--sleep-requests', '5', '-F', 'https://player.vimeo.com/video/516390862'] [debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version stable@2024.08.01 from yt-dlp/yt-dlp [ffd7781d6] (win_exe) [debug] Python 3.8.10 (CPython AMD64 64bit) - Windows-10-10.0.19045-SP0 (OpenSSL 1.1.1k 25 Mar 2021) [debug] exe versions: ffmpeg 2024-06-09-git-94f2274a8b-full_build-www.gyan.dev (setts), ffprobe 2024-06-09-git-94f2274a8b-full_build-www.gyan.dev, phantomjs 2.1.1 [debug] Optional libraries: Cryptodome-3.20.0, brotli-1.1.0, certifi-2024.07.04, curl_cffi-0.5.10, mutagen-1.47.0, requests-2.32.3, sqlite3-3.35.5, urllib3-2.2.2, websockets-12.0 [debug] Proxy map: {} [debug] Request Handlers: urllib, requests, websockets, curl_cffi [debug] Loaded 1830 extractors [debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest Latest version: stable@2024.08.01 from yt-dlp/yt-dlp yt-dlp is up to date (stable@2024.08.01 from yt-dlp/yt-dlp) [vimeo] Extracting URL: https://player.vimeo.com/video/516390862 [vimeo] 516390862: Downloading webpage ERROR: [vimeo] 516390862: Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>) File "yt_dlp\extractor\common.py", line 740, in extract File "yt_dlp\extractor\vimeo.py", line 871, in _real_extract File "yt_dlp\extractor\common.py", line 960, in _download_webpage_handle File "yt_dlp\extractor\common.py", line 909, in _request_webpage
File "yt_dlp\extractor\common.py", line 896, in _request_webpage File "yt_dlp\YoutubeDL.py", line 4165, in urlopen File "yt_dlp\networking\common.py", line 117, in send File "yt_dlp\networking_curlcffi.py", line 156, in send File "yt_dlp\networking_helper.py", line 208, in wrapper File "yt_dlp\networking\common.py", line 340, in send File "yt_dlp\networking_curlcffi.py", line 252, in _send yt_dlp.networking.exceptions.HTTPError: HTTP Error 401: Unauthorized
It's curious, but the turnstile feature appears also in vimeo videos that DO work in the browser but do not work in the yt-dlp request with the same 401 error (the above mentioned does not work in browser, the usual "Because of the privacy.bla bla..." that means it needs an refferer.
look what I found in a footer of a curl request to a embeded vimeo video that when used with yt-dlp AND passing the referer gave a "Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>)" Is this turnstile feature new or I haven't spotted it in the past?
Thanks, this means that by switching to the Python version of yt-dlp combined with the the following steps: 1) pip uninstall curl_cffi 2)python3 -m pip install -U "yt-dlp[default,curl-cffi]"
would prevent the error until a new compiled version of yt-dlp appears?
If you're getting 403/429/401 errors with curl-cffi/impersonation, there is no fix for that yet. There is no known solution yet
Yes I still got 401, but I managed to download with curl some pages and indeed there is a challenge block inside the response, I suppose you already got the code...if not I can post it here
just some info in case it helps:
This is from the command line on a mac yt-dlp --version 2024.08.06
yt-dlp https://vimeo.com/506237132
[vimeo] Extracting URL: https://vimeo.com/506237132
[vimeo] 506237132: Downloading webpage
WARNING: [vimeo] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see https://github.com/yt-dlp/yt-dlp#impersonation for information on installing the required dependencies
ERROR: [vimeo] 506237132: This request has been blocked due to its TLS fingerprint. Install a required impersonation dependency if possible, or else if you are okay with compromising your security/cookies, try replacing "https:" with "http:" in the input URL. If you are using a data center IP or VPN/proxy, your IP may be blocked.
tried it with http:
yt-dlp http://vimeo.com/506237132
[vimeo] Extracting URL: http://vimeo.com/506237132
[vimeo] 506237132: Downloading webpage
ERROR: [vimeo] 506237132: Got HTTP Error 429. If you are using a data center IP or VPN/proxy, your IP may be blocked
grabbed the url from the embed code and tried that:
yt-dlp https://player.vimeo.com/video/506237132?h=30f720a9e2
[vimeo] Extracting URL: https://player.vimeo.com/video/506237132?h=30f720a9e2
[vimeo] 506237132: Downloading webpage
WARNING: [vimeo] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see https://github.com/yt-dlp/yt-dlp#impersonation for information on installing the required dependencies
ERROR: [vimeo] 506237132: Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>)
changing the embed url to http
finally worked:
yt-dlp http://player.vimeo.com/video/506237132?h=30f720a9e2
[vimeo] Extracting URL: http://player.vimeo.com/video/506237132?h=30f720a9e2
[vimeo] 506237132: Downloading webpage
[vimeo] 506237132: Downloading akfire_interconnect_quic m3u8 information
[vimeo] 506237132: Downloading fastly_skyfire m3u8 information
[vimeo] 506237132: Downloading akfire_interconnect_quic MPD information
WARNING: [vimeo] 506237132: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[vimeo] 506237132: Downloading fastly_skyfire MPD information
WARNING: [vimeo] 506237132: Failed to parse XML: not well-formed (invalid token): line 1, column 0
[info] 506237132: Downloading 1 format(s): hls-fastly_skyfire-2604+hls-fastly_skyfire-audio-low-Original
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 36
[download] Destination: Adriano Celentano - Prisencolinensinainciusol [506237132].fhls-fastly_skyfire-2604.mp4
just some info in case it helps:
This is from the command line on a mac yt-dlp --version 2024.08.06
yt-dlp https://vimeo.com/506237132 [vimeo] Extracting URL: https://vimeo.com/506237132 [vimeo] 506237132: Downloading webpage WARNING: [vimeo] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see https://github.com/yt-dlp/yt-dlp#impersonation for information on installing the required dependencies ERROR: [vimeo] 506237132: This request has been blocked due to its TLS fingerprint. Install a required impersonation dependency if possible, or else if you are okay with compromising your security/cookies, try replacing "https:" with "http:" in the input URL. If you are using a data center IP or VPN/proxy, your IP may be blocked.
tried it with http:
yt-dlp http://vimeo.com/506237132 [vimeo] Extracting URL: http://vimeo.com/506237132 [vimeo] 506237132: Downloading webpage ERROR: [vimeo] 506237132: Got HTTP Error 429. If you are using a data center IP or VPN/proxy, your IP may be blocked
grabbed the url from the embed code and tried that:
yt-dlp https://player.vimeo.com/video/506237132?h=30f720a9e2 [vimeo] Extracting URL: https://player.vimeo.com/video/506237132?h=30f720a9e2 [vimeo] 506237132: Downloading webpage WARNING: [vimeo] The extractor is attempting impersonation, but no impersonate target is available; if you encounter errors, then see https://github.com/yt-dlp/yt-dlp#impersonation for information on installing the required dependencies ERROR: [vimeo] 506237132: Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>)
changing the embed url to
http
finally worked:yt-dlp http://player.vimeo.com/video/506237132?h=30f720a9e2 [vimeo] Extracting URL: http://player.vimeo.com/video/506237132?h=30f720a9e2 [vimeo] 506237132: Downloading webpage [vimeo] 506237132: Downloading akfire_interconnect_quic m3u8 information [vimeo] 506237132: Downloading fastly_skyfire m3u8 information [vimeo] 506237132: Downloading akfire_interconnect_quic MPD information WARNING: [vimeo] 506237132: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [vimeo] 506237132: Downloading fastly_skyfire MPD information WARNING: [vimeo] 506237132: Failed to parse XML: not well-formed (invalid token): line 1, column 0 [info] 506237132: Downloading 1 format(s): hls-fastly_skyfire-2604+hls-fastly_skyfire-audio-low-Original [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 36 [download] Destination: Adriano Celentano - Prisencolinensinainciusol [506237132].fhls-fastly_skyfire-2604.mp4
Doesn't work here =/
yt-dlp http://player.vimeo.com/video/882423874?app_id=58479
[vimeo] Extracting URL: http://player.vimeo.com/video/882423874?app_id=58479
[vimeo] 882423874: Downloading webpage
ERROR: [vimeo] 882423874: Unable to download webpage: HTTP Error 401: Unauthorized (caused by <HTTPError 401: Unauthorized>)
yt-dlp --version
2024.08.06
DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE
Checklist
Region
Australia
Provide a description that is worded well enough to be understood
I am downloading videos from a page that embeds Vimeo videos. I was able to download successfully until a few minutes ago. Now, when I use the same command that I have been using all along, I get the error:
Has my IP been blocked by Vimeo? I have downloaded about 10-15 videos in the last 48 hours. I am able to normally access the webpage that embeds these videos on Chrome, and play these videos.
I would really appreciate any help or advice as I really want to download these videos. Thank you!
Provide verbose output that clearly demonstrates the problem
yt-dlp -vU <your command line>
)'verbose': True
toYoutubeDL
params instead[debug] Command-line config
) and insert it belowComplete Verbose Output