ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
131.96k stars 10.01k forks source link

Pluralsight Issue #24895

Closed jhondbv closed 4 years ago

jhondbv commented 4 years ago

Hello , I´ve been working with youtube-dl the las two weeks , the last two days i have the next error from pluralsight , the program can´t connecting the login page as it did before

--verbose log

youtube-dl --verbose [debug] System config: [] [debug] User config: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-i', '-c', '--no-warnings', '--console-title', '--batch-file=batch-file.txt', '--max-sleep-interval', '80', '--min-sleep-interval', '60', '-o', '%(playlist_title)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s-%(title)s.%(ext)s', '-f', 'best[height<=720]/worst[height>720]'] [debug] Custom config: [] [debug] Command-line args: ['--verbose'] [debug] Batch file urls: ['https://app.pluralsight.com/library/courses/kafka-build-deploy-monitor-real-world-application'] [debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252 [debug] youtube-dl version 2020.03.24 [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362 [debug] exe versions: none [debug] Proxy map: {} [pluralsight:course] kafka-build-deploy-monitor-real-world-application: Downloading JSON metadata [pluralsight:course] kafka-build-deploy-monitor-real-world-application: Downloading JSON metadata ERROR: Unable to download JSON metadata: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 627, in _request_webpage File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\YoutubeDL.py", line 2238, in urlopen File "C:\Python\Python34\lib\urllib\request.py", line 470, in open File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response File "C:\Python\Python34\lib\urllib\request.py", line 508, in error File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

jaimebl commented 4 years ago

Are you able to watch it using your browser? You might have been banned

jhondbv commented 4 years ago

Are you able to watch it using your browser? You might have been banned

Yes , i was banned , but after send a few emails to pluralsight they unbloked my account , the last few hours I've figured out a few things that i would like share with you :

My account was banned and also my IP was flagged for using automated scripting to view content. I can't use any diferent account from my IP because they allow a captcha validation for all request from my PC

I solved this 2 things using a VPN to hidde my ip , the problem is that almost all good VPN are free with a limit of GB .

On the ohter hand this issue has appeared when i tried to download subtittles of the videos that i had already donwloaded , --sleep-interval doesn't work with --skip-donwload I was discovered for too many request to the server in a short time.

The real problem is ,How can i download only subtittles without be banned because of the too many request to the server ? --sleep-interval doesn't work when i don't invoke the Donwloader (when i use --skip-donwload)

rali-gh commented 4 years ago

Related, so using the same issue report - My user/pass validates, I'm able to watch using browser and youyoutube-dl is up-to-date (2020.03.24). I'm NOT banned or blocked and I'm getting the 403 Forbidden on the video download. Verbose output:

% youtube-dl --username --password \ -o "./%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s - %(title)s.%(ext)s" \ --min-sleep-interval 30 --max-sleep-interval 60 \ https://app.pluralsight.com/library/courses/palo-alto-firewalls-deploying-administering-securing \ --playlist-start 1 --verbose [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'--username', u'PRIVATE', u'--password', u'PRIVATE', u'-o', u'./%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s - %(title)s.%(ext)s', u'--min-sleep-interval', u'30', u'--max-sleep-interval', u'60', u'https://app.pluralsight.com/library/courses/palo-alto-firewalls-deploying-administering-securing', u'--playlist-start', u'1', u'--verbose'] [debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2020.03.24 [debug] Python version 2.7.17 (CPython) - Linux-4.15.0-54-generic-x86_64-with-Ubuntu-18.04-bionic [debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6 [debug] Proxy map: {} [pluralsight:course] palo-alto-firewalls-deploying-administering-securing: Downloading JSON metadata [download] Downloading playlist: Deploying, Administering, and Securing Palo Alto Firewalls [pluralsight:course] playlist Deploying, Administering, and Securing Palo Alto Firewalls: Collected 33 video ids (downloading 33 of them) [download] Downloading video 1 of 33 [pluralsight] Downloading login page ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/home/rali/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage return self._downloader.urlopen(url_or_request) File "/home/rali/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib/python2.7/urllib2.py", line 435, in open response = meth(req, response) File "/usr/lib/python2.7/urllib2.py", line 548, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python2.7/urllib2.py", line 473, in error return self._call_chain(args) File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(args) File "/usr/lib/python2.7/urllib2.py", line 556, in http_error_default raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)

Thanks

jhondbv commented 4 years ago

Hello @rali-gh , some times i have the same issue , it's can be a wall from the server to avoid this type of connections and request , but I was able to download after several attempts, try open two terminals and execute youtube-dl command in both , try do it at the same time for a few minutes , almost all the times it works for me , you don't have the same issue that I posted before but I am glad to help you .

jaimebl commented 4 years ago

Yes @rali-gh it's weird that issue only happens with Python 2.7, if you use Python 3 it will work fine

rali-gh commented 4 years ago

Hi @jaimebl -- Interesting. I tried:

% python3 ~/bin/youtube-dl [All the same arguments I used in my first post]

and got the same error (except, of course, that the diagnostics noew report offsets into Python 3.6 libraries):

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--username', 'PRIVATE', '--password', 'PRIVATE', '-o', './%(playlist)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s - %(title)s.%(ext)s', '--min-sleep-interval', '30', '--max-sleep-interval', '60', 'https://app.pluralsight.com/library/courses/palo-alto-firewalls-deploying-administering-securing', '--playlist-start', '1', '--verbose'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2020.03.24 [debug] Python version 3.6.9 (CPython) - Linux-4.15.0-54-generic-x86_64-with-Ubuntu-18.04-bionic [debug] exe versions: ffmpeg 3.4.6, ffprobe 3.4.6 [debug] Proxy map: {} [pluralsight:course] palo-alto-firewalls-deploying-administering-securing: Downloading JSON metadata [download] Downloading playlist: Deploying, Administering, and Securing Palo Alto Firewalls [pluralsight:course] playlist Deploying, Administering, and Securing Palo Alto Firewalls: Collected 33 video ids (downloading 33 of them) [download] Downloading video 1 of 33 [pluralsight] Downloading login page ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/home/rali/bin/youtube-dl/youtube_dl/extractor/common.py", line 627, in _request_webpage return self._downloader.urlopen(url_or_request) File "/home/rali/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 2238, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/usr/lib/python3.6/urllib/request.py", line 532, in open response = meth(req, response) File "/usr/lib/python3.6/urllib/request.py", line 642, in http_response 'http', request, response, code, msg, hdrs) File "/usr/lib/python3.6/urllib/request.py", line 570, in error return self._call_chain(args) File "/usr/lib/python3.6/urllib/request.py", line 504, in _call_chain result = func(args) File "/usr/lib/python3.6/urllib/request.py", line 65

Thanks again!

jaimebl commented 4 years ago

@rali-gh You're right, running with Python 3.6.8 fails as well, however with versions 3.7.7 and 3.8.1 it works fine

rali-gh commented 4 years ago

@jaimebl -- bingo!

I pulled down python3.8 and the course is downloading now. Thank you for the assist,

R

Lezette commented 4 years ago

I'm also facing a 403 error trying to download from Pluralsight I was not banned from pluralsight and youtube-dl is up to date (2020.03.24) Here's the verbose output

[debug] System config: [] [debug] User config: ['-u', 'PRIVATE', '-p', 'PRIVATE', '-i', '-c', '--no-warnings', '--console-title', '--batch-file=batch-file.txt', '--max-sleep-interval', '80', '--min-sleep-interval', '60', '-o', '%(playlist_title)s/%(chapter_number)s - %(chapter)s/%(playlist_index)s-%(title)s.%(ext)s', '-f', 'best[height<=720]/worst[height>720]'] [debug] Custom config: [] [debug] Command-line args: ['--verbose'] [debug] Batch file urls: ['https://app.pluralsight.com/library/courses/python-data-science/'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2020.03.24 [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.17763 [debug] exe versions: none [debug] Proxy map: {} [pluralsight:course] python-data-science: Downloading JSON metadata [pluralsight:course] python-data-science: Downloading JSON metadata ERROR: python-data-science: Failed to parse JSON (caused by ValueError('Expecting value: line 1 column 1 (char 0)',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 899, in _parse_json File "C:\Python\Python34\lib\json__init.py", line 318, in loads File "C:\Python\Python34\lib\json\decoder.py", line 343, in decode File "C:\Python\Python34\lib\json\decoder.py", line 361, in raw_decode ValueError: Expecting value: line 1 column 1 (char 0) Traceback (most recent call last): File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 899, in _parse_json File "C:\Python\Python34\lib\json\init__.py", line 318, in loads File "C:\Python\Python34\lib\json\decoder.py", line 343, in decode File "C:\Python\Python34\lib\json\decoder.py", line 361, in raw_decode ValueError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\YoutubeDL.py", line 797, in extract_info File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 530, in extract File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\pluralsight.py", line 467, in _real_extract File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\pluralsight.py", line 93, in _download_course File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 892, in _download_json File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 876, in _download_json_handle File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpjwbwqymm\build\youtube_dl\extractor\common.py", line 903, in _parse_json youtube_dl.utils.ExtractorError: python-data-science: Failed to parse JSON (caused by ValueError('Expecting value: line 1 column 1 (char 0)',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

jaimebl commented 4 years ago

@Lezette Have you tried using python 3.7 or 3.8?

xxbbcc commented 4 years ago

I'm using the latest Windows executable (2020.03.24) and I'm also getting the same error. I had the same issue with build 2020.03.08, I just haven't used it for a while so I noticed both breaking on the same day. (My account is not banned and I'm able to watch in browser.) I assume (based on the download instructions) that an older version of Python is embedded in the .exe and that is why it fails. Is there anything I can do to resume correct operation?

jaimebl commented 4 years ago

@xxbbcc yep I think it has python 3.4.4 embedded

There's another thing you can do:

Edited: You may need have to modify one line of your cookies file to make it look like this

app.pluralsight.com FALSE   /   FALSE   1650564064  _sdsat_Target Subscription Info InTerm:true|FullLibrary:true|Free:true|Trial:false|UserType:individual
jhondbv commented 4 years ago

@xxbbcc , yes the .exe embebed Phyton 3.4 , but you can say youtube-dl which version is going to use , try download the last versions of Phyton for your OS and install it , and then execute the next commands :

(on windows )

this is the reference [https://www.reddit.com/r/youtubedl/comments/dzj0mr/youtubedl_the_complete_installation_guide_for/ ] works for me .

xxbbcc commented 4 years ago

@jaimebl @jhondbv Thank you for the suggestions, I'll give them a try.

Lezette commented 4 years ago

@Lezette Have you tried using python 3.7 or 3.8?

I just needed to reinstall Microsoft Visual C++ 2010 Service Pack 1 Redistributable Package MFC Security Update and now it's working

jhondbv commented 4 years ago

the issue has been done .