ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.77k stars 10.08k forks source link

Facebook video will throws 500 if append "__fns" parameter #22011

Closed limkokhole closed 5 years ago

limkokhole commented 5 years ago

Checklist

Verbose log

xb@dnxb:~/Downloads$ /usr/local/bin/youtube-dl --verbose 'https://www.facebook.com/VT/videos/1814453785363122/?v=1814453785363122&__fns'
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--verbose', u'https://www.facebook.com/VT/videos/1814453785363122/?v=1814453785363122&__fns']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2019.08.02
[debug] Python version 2.7.15+ (CPython) - Linux-4.15.0-56-generic-x86_64-with-Ubuntu-18.04-bionic
[debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6, rtmpdump 2.4
[debug] Proxy map: {}
[facebook] 1814453785363122: Downloading webpage
ERROR: Unable to download webpage: HTTP Error 500: Internal Server Error (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage
    return self._downloader.urlopen(url_or_request)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2227, in urlopen
    return self._opener.open(req, timeout=self._socket_timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 473, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

xb@dnxb:~/Downloads$ 

Description

I've noticed youtube-dl will get 500 HTTP error if appended "&fns" parameter (or "?fns", depends on the url already has '?' or not). I get this parameter from a shared post of my friend wall, the video url is 'https://www.facebook.com/1886799411623042/videos/2387358644830449/UzpfSTEwMDAzMDI4NTE2Njk4NDoxODI0NzI0MzI3NzIyNjE/?story_fbid=182472432772261&id=100030285166984&__fns&hash=Ac2A00t7U5_-GkV4' , which not able to download. I narrow down and figure out the critical parameter which causes 500 is "__fns".

I can't reproduce to get the parameters "hash" and "__fns " after back to my friend wall and re-click, and also I tried to share the post. So the parameter only appeared one time.

I tried with other video such as https://www.facebook.com/weilamanner/videos/487811265309174/ also same, "https://www.facebook.com/weilamanner/videos/487811265309174/?dummy" is works but not "https://www.facebook.com/weilamanner/videos/487811265309174/?__fns" , it's pretty obvious __fns causes the issue.

dstftw commented 5 years ago

500 in browser either.

limkokhole commented 5 years ago

Are you guy saying that I invented this magic parameter by copy-paste from url bar ? I doubt other users will know need to narrow down and remove __fn and figure out the issue.

1565003199_2019-08-05_gWzmTOwyoo

dstftw commented 5 years ago

I'm saying that URL you've provided without additional context gives the same 500 in browser that clearly matches the behaviour you get with youtube-dl.