Some Instagram reels giving an HTTPError()

Esoppant commented 2 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2021.06.06
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.instagram.com/reel/CTckaQvFNsT/']
[debug] Encodings: locale cp1254, fs mbcs, out cp857, pref cp1254
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041
[debug] exe versions: ffmpeg 3.4.1
[debug] Proxy map: {}
[Instagram] CTckaQvFNsT: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpkqxnwl31\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 502, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 685, in http_error_302
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

Description

Certain instagram reels seem to be giving the above error. It happens on videos from various public channels and on URLs with no special characters. It doesn't seem to be channel-specific either as some reels from the same channel can download fine. Edit: I should point out that I'm using the youtube-dl.exe for Windows on the command prompt.

u-spec-png commented 2 years ago

Try passing a cookies with --cookies cookies.txt

Esoppant commented 2 years ago

Try passing a cookies with --cookies cookies.txt

Like youtube-dl --cookies cookies.txt URL ? That gives the same " HTTP Error 404: Not Found (caused by HTTPError()) " error.

dirkf commented 2 years ago

Try opening the page in your browser, if you can stomach Instagram:

Sorry, this page isn't available. The link you followed may be broken, or the page may have been removed. Go back to Instagram.

So that would explain why yt-dl can't load the page.

Esoppant commented 2 years ago

Wait, it doesn't open for you? That's odd because the link works fine in my browser (Firefox)... Alright, I tried on Opera (and later on Firefox after logging out) and it doesn't load for some reason when I'm not logged in. Even though the account and the thumbnail of the video are perfectly visible.

So, Instagram apparently works in mysterious ways and the bug is not at the youtube-dl end. I might try doing a logged in download then.

bozoputer commented 2 years ago

@Esoppant i'm still getting the error from instagram (for this video) even when doing a logged-in download 🤷🏽‍♂️

if you've attempted it yet, were able to get your vid to download?

Esoppant commented 2 years ago

if you've attempted it yet, were able to get your vid to download?

Nope, this time it's giving an "HTTP 506" error. But for some reason, another video with the same error I had tried before now works normally (and is visible without logging in). Maybe Instagram is moving around servers or something.

tucomel commented 2 years ago

http error 405 while trying to download reels from instragram (already logged using cookies) latest version verbose log below

arthur@blackpower ~ % yt-dlp "https://www.instagram.com/marvelskies.fc/reel/CWqAgUZgCku/" --cookies "~/instagram.com_cookies.txt" --verbose
[debug] Command-line config: ['https://www.instagram.com/marvelskies.fc/reel/CWqAgUZgCku/', '--cookies', '~/instagram.com_cookies.txt', '--verbose']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, err utf-8, pref UTF-8
[debug] yt-dlp version 2021.12.01 [91f071af6]
[debug] Python version 3.10.0 (CPython 64bit) - macOS-11.3-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome, mutagen, sqlite, websockets
[debug] Proxy map: {}
[debug] [generic] Extracting URL: https://www.instagram.com/marvelskies.fc/reel/CWqAgUZgCku/
[generic] CWqAgUZgCku: Requesting header
WARNING: [generic] Could not send HEAD request to https://www.instagram.com/marvelskies.fc/reel/CWqAgUZgCku/: HTTP Error 405: Method Not Allowed
[generic] CWqAgUZgCku: Downloading webpage
WARNING: [generic] Falling back on generic information extractor.
[generic] CWqAgUZgCku: Extracting information
[debug] Looking for video embeds
[debug] Identified a JW Player JS loader
[debug] Default format spec: bestvideo*+bestaudio/best
[info] 259945183_634763724195677_1967405584202529129_n.mp4?_nc_ht=instagram.fgyn3-1.fna.fbcdn: Downloading 1 format(s): 0
[debug] Invoking downloader on "https://instagram.fgyn3-1.fna.fbcdn.net/v/t50.2886-16/259945183_634763724195677_1967405584202529129_n.mp4?_nc_ht=instagram.fgyn3-1.fna.fbcdn.net\u0026_nc_cat=105\u0026_nc_ohc=pIfmkouO64cAX-mzjZh\u0026edm=APfKNqwBAAAA\u0026ccb=7-4\u0026oe=61B1C916\u0026oh=9c99e1bc5cbb455c9b5a2ac528d429de\u0026_nc_sid=74f7ba"
ERROR: unable to download video data: HTTP Error 403: Forbidden
Traceback (most recent call last):
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 2933, in process_info
    success, real_download = self.dl(temp_filename, info_dict)
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 2654, in dl
    return fd.download(name, new_info, subtitle)
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/downloader/common.py", line 421, in download
    ret = self.real_download(filename, info_dict)
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/downloader/http.py", line 372, in real_download
    establish_connection()
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/downloader/http.py", line 120, in establish_connection
    raise err
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/downloader/http.py", line 114, in establish_connection
    ctx.data = self.ydl.urlopen(request)
  File "/usr/local/Cellar/yt-dlp/2021.12.1/libexec/lib/python3.10/site-packages/yt_dlp/YoutubeDL.py", line 3385, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 557, in error
    result = self._call_chain(*args)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 749, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 525, in open
    response = meth(req, response)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 634, in http_response
    response = self.parent.error(
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 563, in error
    return self._call_chain(*args)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 496, in _call_chain
    result = func(*args)
  File "/usr/local/Cellar/python@3.10/3.10.0_2/Frameworks/Python.framework/Versions/3.10/lib/python3.10/urllib/request.py", line 643, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

tucomel commented 2 years ago

but the same error occurs on youtube-dl too

ghost commented 2 years ago

@tucomel Please do not post irrelevant log here.

I have updated PR #29751 to accept this type of URL https://www.instagram.com/marvelskies.fc/reel/CWqAgUZgCku/ as Instagram URL. Unfortunately, you won't be able to get new youtube-dl release any time soon, so you have to modify the code by yourself if you need this.

Edit: It seems that yt-dlp will fix this problem in the next release. You may want to wait for it.

ytdl-org / youtube-dl