search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.4k stars 10.04k forks source link

[nbc.com] bug with SOME geo-restricted videos watchable via VPN #9578

Closed kissoq closed 8 years ago

kissoq commented 8 years ago

Please follow the guide below

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2016.05.21.2. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:

Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

C:\Users\New\Downloads>youtube-dl.exe --verbose http://www.nbc.com/Kings/vid
eo/goliath/n1806
[debug] System config: []
[debug] User config: []
[debug] Command-line args: [u'--verbose', u'http://www.nbc.com/Kings/video/golia
th/n1806']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2016.05.21.2
[debug] Python version 2.7.10 - Windows-8-6.2.9200
[debug] exe versions: ffmpeg N-80052-g64fe1ee
[debug] Proxy map: {}
[NBC] n1806: Downloading webpage
[ThePlatform] 101528f5a9e8127b107e98c5e6ce4638: Downloading webpage
[ThePlatform] 101528f5a9e8127b107e98c5e6ce4638: Downloading SMIL data
[ThePlatform] 101528f5a9e8127b107e98c5e6ce4638: Downloading m3u8 information
[ThePlatform] 101528f5a9e8127b107e98c5e6ce4638: Downloading JSON metadata
[debug] Invoking downloader on u'https://nbcvodenc-i.akamaihd.net/i/NBCdotCOM/4c
722cb6039c20872d14d18f81172816_,1696,1296,0896,0696,0496,0306,0240,0150,0064,.mp
4.csmil/index_0_av.m3u8?e=7985ffe09f218020&__s__=&__a__=on&ads='
[download] Destination: Goliath-n1806.mp4
[debug] ffmpeg command line: ffmpeg -y -headers 'Accept-Language: en-us,en;q=0.5

Accept-Encoding: gzip, deflate
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/44.0
 (Chrome)
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Cookie: hdnea2=expires=1464021261~access=/*~md5=c67cf42cff5ecf798f63ac3b8f709c3b

' -i 'https://nbcvodenc-i.akamaihd.net/i/NBCdotCOM/4c722cb6039c20872d14d18f81172
816_,1696,1296,0896,0696,0496,0306,0240,0150,0064,.mp4.csmil/index_0_av.m3u8?e=7
985ffe09f218020&__s__=&__a__=on&ads=' -c copy -f mp4 -bsf:a aac_adtstoasc file:G
oliath-n1806.mp4.part
ffmpeg version N-80052-g64fe1ee Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.0 (GCC)
  configuration: --enable-gpl --enable-version3 --disable-w32threads --enable-av
isynth --enable-bzlib --enable-fontconfig --enable-frei0r --enable-gnutls --enab
le-iconv --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --
enable-libfreetype --enable-libgme --enable-libgsm --enable-libilbc --enable-lib
modplug --enable-libmfx --enable-libmp3lame --enable-libopencore-amrnb --enable-
libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librtmp --enabl
e-libschroedinger --enable-libsnappy --enable-libsoxr --enable-libspeex --enable
-libtheora --enable-libtwolame --enable-libvidstab --enable-libvo-amrwbenc --ena
ble-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx
264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable
-lzma --enable-decklink --enable-zlib
  libavutil      55. 24.100 / 55. 24.100
  libavcodec     57. 42.100 / 57. 42.100
  libavformat    57. 36.100 / 57. 36.100
  libavdevice    57.  0.101 / 57.  0.101
  libavfilter     6. 46.100 /  6. 46.100
  libswscale      4.  1.100 /  4.  1.100
  libswresample   2.  0.101 /  2.  0.101
  libpostproc    54.  0.100 / 54.  0.100
[https @ 0000000001d23ce0] HTTP error 403 Forbidden
https://nbcvodenc-i.akamaihd.net/i/NBCdotCOM/4c722cb6039c20872d14d18f81172816_,1
696,1296,0896,0696,0496,0306,0240,0150,0064,.mp4.csmil/index_0_av.m3u8?e=7985ffe
09f218020&__s__=&__a__=on&ads=: Server returned 403 Forbidden (access denied)

ERROR: ffmpeg exited with code 1
  File "__main__.py", line 19, in <module>
  File "youtube_dl\__init__.pyo", line 421, in main
  File "youtube_dl\__init__.pyo", line 411, in _real_main
  File "youtube_dl\YoutubeDL.pyo", line 1736, in download
  File "youtube_dl\YoutubeDL.pyo", line 687, in extract_info
  File "youtube_dl\YoutubeDL.pyo", line 758, in process_ie_result
  File "youtube_dl\YoutubeDL.pyo", line 733, in process_ie_result
  File "youtube_dl\YoutubeDL.pyo", line 1382, in process_video_result
  File "youtube_dl\YoutubeDL.pyo", line 1644, in process_info
  File "youtube_dl\YoutubeDL.pyo", line 1586, in dl
  File "youtube_dl\downloader\common.pyo", line 350, in download
  File "youtube_dl\downloader\external.pyo", line 43, in real_download
  File "youtube_dl\downloader\common.pyo", line 161, in report_error
  File "youtube_dl\YoutubeDL.pyo", line 550, in report_error
  File "youtube_dl\YoutubeDL.pyo", line 512, in trouble

Description of your issue, suggested solution and other information

Latest versions, as of date of posting, of both youtube-dl and ffmpeg. Content is geo-restricted, so using a VPN to simulate location in NYC. VPN active system-wide and not app-wide, so in my eyes it should be passing all traffic through it. URL reporting the aforementioned behaviour: http://www.nbc.com/Kings/video/goliath/n1806 On a browser, url plays fine after accessing it with the vpn enabled. URL of the same nbc.com site and even the same show, working perfectly fine with youtube-dl (and also playing fine in browser with vpn enabled): http://www.nbc.com/classic-tv/kings/video/prosperity/2880912 Note the different url structure, so my guess is this 2nd link is using an 'old' nbc site structure or system, while the 1st one has been updated as is the part of nbc.com currently not worki with youtube-dl.exe

I have previously searched the forum for similar issues with nbc.com and have found some, but none was seeing the exact same behaviour as this one. Thanks for having a look at this!

yan12125 commented 8 years ago

Which VPN software are you using?

kissoq commented 8 years ago

I am using Windows 7's own VPN support, configuring the vpn manually and not going through the vpn company's software. I assumed that would be the safest way to go? VPN protocol being L2TP, in case that makes a difference. Thanks for your help.

On Sun, May 29, 2016 at 2:23 PM, Yen Chi Hsuan notifications@github.com wrote:

Which VPN software are you using?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rg3/youtube-dl/issues/9578#issuecomment-222357800, or mute the thread https://github.com/notifications/unsubscribe/AJo_uE9mYFxc92-zhXwAVbWW8N06lwUGks5qGYUtgaJpZM4IkBnu .

yan12125 commented 8 years ago

Strange. Maybe this VPN does not allow traffic to https://nbcvodenc-i.akamaihd.net/i/NBCdotCOM/4c722cb6039c20872d14d18f81172816_,1696,1296,0896,0696,0496,0306,0240,0150,0064,.mp4.csmil/index_0_av.m3u8?e=7985ffe09f218020&__s__=&__a__=on&ads=

kissoq commented 8 years ago

This is what I received from the VPN provider after sending them your response, hope it helps in some way. Any wya to download using a different protocol as they suggest? Thanks!

We have no traffic restrictions, once the vpn connected to server, the traffic made via that tunnel that has been created goes without our control or any technical possibility of setting restrictions. Also, we could not possibly set restrictions for only part of your activity there, if there were any restrictions, on speed or anything, you would have noticed it in general, it can not be done selectively for only something specifically that you download via our service :) What you could check with the source of that link if they have any restrictions in report to the protocols used by the vpn. I would suggest trying to make those downloads using a different protocol and see what the outcome is :) We would love to have it in our power and to be as simple as us lifting a limitations, but unfortunately it is not us that have control over this issue.

On Wed, Jun 1, 2016 at 7:26 AM, Yen Chi Hsuan notifications@github.com wrote:

Strange. Maybe this VPN does not allow traffic to https://nbcvodenc-i.akamaihd.net/i/NBCdotCOM/4c722cb6039c20872d14d18f81172816_,1696,1296,0896,0696,0496,0306,0240,0150,0064,.mp4.csmil/index_0_av.m3u8?e=7985ffe09f218020&__s__=&__a__=on&ads=

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/rg3/youtube-dl/issues/9578#issuecomment-222896024, or mute the thread https://github.com/notifications/unsubscribe/AJo_uI80z3a9BY9urgLM4-MfILMrgVx9ks5qHRf7gaJpZM4IkBnu .

yan12125 commented 8 years ago

OK I got 403 from an US proxy, too.

Note to myself: now ThePlatform m3u8 manifest requires an additional hdnea3 field.

yan12125 commented 8 years ago

Thanks for the report and sorry for the late response. NBC via proxies should be fixed in the next version. At least this video works fine with my US proxy.

kissoq commented 8 years ago

Looking forward to testing the new version then. Thanks for the hard work!