search

rojter-tech / pluradl.py

Automated download of Pluralsight courses
MIT License
2 stars 0 forks source link

plurdl.py quits / crashes when downloading #24

Closed lonegeek closed 4 years ago

lonegeek commented 4 years ago

robc@Slicer-PC:~/pluradl.py$ python3 pluradl.py --subtitle Enter you Pluralsight credentials Enter username: robc@jhu.edu Enter password (will not be displayed) : Setting username to: robc@jhu.edu Subtitles will not be appended to videoclips Downloading subtitles is set to: off

Downloading video indicies up to 10 to [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] plura-dl version 1.0.0b4 [debug] Git HEAD: 49cb809 [debug] Python version 3.7.3 (CPython) - Linux-4.4.0-18362-Microsoft-x86_64-with-debian-10.3 [debug] exe versions: none [debug] Proxy map: {} [pluralsight:course] comptia-project-plus-pt1: Downloading JSON metadata [download] Downloading playlist: CompTIA Project+: Part 1 [pluralsight:course] playlist CompTIA Project+: Part 1: Collected 109 video ids (downloading 10 of them) [download] Downloading video 1 of 10 [pluralsight] Downloading login page [pluralsight] Logging in [pluralsight] comptia-project-plus-pt1-m01-0: Downloading JSON metadata [pluralsight] comptia-project-plus-pt1-m01-0: Downloading mp4-high viewclip graphql [pluralsight] comptia-project-plus-pt1-m01-0: Waiting for 3 seconds to avoid throttling [debug] Invoking downloader on 'https://vid5.pluralsight.com/expiretime=1586202073/3e8743e24d074a46d2421f6291ed2536/clips/resolution/fa4e6475-6c58-4bfc-8e0f-c23d5a9faef1/current/mp4/1024x768.mp4' [download] Sleeping 235.25 seconds... [download] Destination: 01-1-Getting_Started-1024x768.mp4 [download] 0.0% of 11.07MiB at 43.67KiB/s ETA 04:20

[download] 99.8% of 11.07MiB at 45.56KiB/s ETA 00:00 [download] 100.0% of 11.07MiB at 45.56KiB/s ETA 00:00 [download] 100% of 11.07MiB in 04:08 [download] Downloading video 2 of 10 [pluralsight] comptia-project-plus-pt1-m02-0: Downloading JSON metadata [pluralsight] comptia-project-plus-pt1-m02-0: Downloading mp4-high viewclip graphql [pluralsight] comptia-project-plus-pt1-m02-0: Waiting for 2 seconds to avoid throttling [debug] Invoking downloader on 'https://vid5.pluralsight.com/expiretime=1586202561/900118a275bd25f280f7446c2f121aa0/clips/resolution/8b1a0636-33a0-41bf-a67d-7f8abe1f1135/current/mp4/1024x768.mp4' [download] Sleeping 121.31 seconds... [download] Destination: 02-2-Introduction-1024x768.mp4 [download] 0.1% of 1.36MiB at 43.84KiB/s ETA 00:31

[download] 99.0% of 1.36MiB at 45.56KiB/s ETA 00:00 [download] 100.0% of 1.36MiB at 45.56KiB/s ETA 00:00 [download] 100% of 1.36MiB in 00:30 [download] Downloading video 3 of 10 [pluralsight] comptia-project-plus-pt1-m02-1: Downloading JSON metadata [pluralsight] comptia-project-plus-pt1-m02-1: Downloading mp4-high viewclip graphql [pluralsight] comptia-project-plus-pt1-m02-1: Waiting for 2 seconds to avoid throttling [debug] Invoking downloader on 'https://vid.pluralsight.com/clips/resolution/76b4a9b0-056c-4ee4-8fff-946f7c6913ba/current/mp4/1024x768.mp4?bronzVQ1uZH2XB-HHL0BmX2nYcuV-D4yDQNX0VYU1ypAENQFJ1lNxQw7Se9nCkkuBkS4avhcw4IPboKLb4v_IXxDg2Ru9ha2NbVn0SDNR0e1tMiBzyJq2m6Aya2EdhYS4pYkhsXcLUIa9lWY9FjW47weY3NII6vCrh7KgJvd028lU7mEi1iX5Qv-3PXz' [download] Sleeping 170.30 seconds... [download] Destination: 03-2-The_Software_I_Used-1024x768.mp4 [download] 0.0% of 4.11MiB at 44.87KiB/s ETA 01:33 [download] 0.1% of 4.11MiB at 44.61KiB/s ETA 01:34 [download] 0.2% of 4.11MiB at 45.10KiB/s ETA 01:33 [download] 0.4% of 4.11MiB at 45.31KiB/s ETA 01:32 [download] 0.7% of 4.11MiB at 45.46KiB/s ETA 01:31 [download] 1.5% of 4.11MiB at 45.49KiB/s ETA 01:31 [download] 2.6% of 4.11MiB at 45.52KiB/s ETA 01:30 [download] 3.7% of 4.11MiB at 45.53KiB/s ETA 01:28 [download] 4.7% of 4.11MiB at 45.54KiB/s ETA 01:27 [download] 5.8% of 4.11MiB at 45.55KiB/s ETA 01:26 [download] 6.9% of 4.11MiB at 45.55KiB/s ETA 01:25 [download] 8.0% of 4.11MiB at 45.55KiB/s ETA 01:24 [download] 9.1% of 4.11MiB at 45.55KiB/s ETA 01:23 [download] 10.2% of 4.11MiB at 45.55KiB/s ETA 01:22 [download] 11.2% of 4.11MiB at 45.56KiB/s ETA 01:21 [download] 12.3% of 4.11MiB at 45.55KiB/s ETA 01:20 [download] 13.4% of 4.11MiB at 45.55KiB/s ETA 01:19 [download] 14.5% of 4.11MiB at 45.56KiB/s ETA 01:18 [download] 15.6% of 4.11MiB at 45.56KiB/s ETA 01:17 [download] 16.7% of 4.11MiB at 45.56KiB/s ETA 01:16 [download] 17.7% of 4.11MiB at 45.56KiB/s ETA 01:15 [download] 18.8% of 4.11MiB at 45.56KiB/s ETA 01:14 [download] 19.9% of 4.11MiB at 45.56KiB/s ETA 01:13 [download] 21.0% of 4.11MiB at 45.56KiB/s ETA 01:12 [download] 22.1% of 4.11MiB at 45.56KiB/s ETA 01:11 [download] 23.2% of 4.11MiB at 45.56KiB/s ETA 01:10 [download] 24.2% of 4.11MiB at 45.56KiB/s ETA 01:09 [download] 25.3% of 4.11MiB at 45.56KiB/s ETA 01:08 [download] 26.4% of 4.11MiB at 45.56KiB/s ETA 01:07 [download] 27.5% of 4.11MiB at 45.56KiB/s ETA 01:06 [download] 28.6% of 4.11MiB at 45.56KiB/s ETA 01:05 [download] 29.7% of 4.11MiB at 45.56KiB/s ETA 01:04 [download] 30.7% of 4.11MiB at 45.56KiB/s ETA 01:03 [download] 31.8% of 4.11MiB at 45.56KiB/s ETA 01:02 [download] 32.9% of 4.11MiB at 45.56KiB/s ETA 01:01 [download] 34.0% of 4.11MiB at 45.56KiB/s ETA 01:00 [download] 35.1% of 4.11MiB at 45.56KiB/s ETA 00:59 [download] 36.2% of 4.11MiB at 45.56KiB/s ETA 00:58 [download] 37.2% of 4.11MiB at 45.56KiB/s ETA 00:57 [download] 38.3% of 4.11MiB at 45.56KiB/s ETA 00:56 [download] 39.4% of 4.11MiB at 45.56KiB/s ETA 00:55 [download] 40.5% of 4.11MiB at 45.56KiB/s ETA 00:54 [download] 41.6% of 4.11MiB at 45.56KiB/s ETA 00:53 [download] 42.7% of 4.11MiB at 45.56KiB/s ETA 00:52 [download] 43.7% of 4.11MiB at 45.56KiB/s ETA 00:51 [download] 44.8% of 4.11MiB at 45.56KiB/s ETA 00:50 [download] 45.9% of 4.11MiB at 45.56KiB/s ETA 00:49 [download] 47.0% of 4.11MiB at 45.56KiB/s ETA 00:48 [download] 48.1% of 4.11MiB at 45.56KiB/s ETA 00:47 [download] 49.2% of 4.11MiB at 45.56KiB/s ETA 00:46 [download] 50.2% of 4.11MiB at 45.56KiB/s ETA 00:45 [download] 51.3% of 4.11MiB at 45.56KiB/s ETA 00:44 [download] 52.4% of 4.11MiB at 45.56KiB/s ETA 00:43 [download] 53.5% of 4.11MiB at 45.56KiB/s ETA 00:42 [download] 54.6% of 4.11MiB at 45.56KiB/s ETA 00:41 [download] 55.7% of 4.11MiB at 45.56KiB/s ETA 00:40 [download] 56.7% of 4.11MiB at 45.56KiB/s ETA 00:39 [download] 57.8% of 4.11MiB at 45.56KiB/s ETA 00:38 [download] 58.9% of 4.11MiB at 45.56KiB/s ETA 00:37 [download] 60.0% of 4.11MiB at 45.56KiB/s ETA 00:36 [download] 61.1% of 4.11MiB at 45.56KiB/s ETA 00:35 [download] 62.2% of 4.11MiB at 45.56KiB/s ETA 00:34 [download] 63.2% of 4.11MiB at 45.56KiB/s ETA 00:33 [download] 64.3% of 4.11MiB at 45.56KiB/s ETA 00:32 [download] 65.4% of 4.11MiB at 45.56KiB/s ETA 00:31 [download] 66.5% of 4.11MiB at 45.56KiB/s ETA 00:30 [download] 67.6% of 4.11MiB at 45.56KiB/s ETA 00:29 [download] 68.7% of 4.11MiB at 45.56KiB/s ETA 00:28 [download] 69.7% of 4.11MiB at 45.56KiB/s ETA 00:27 [download] 70.8% of 4.11MiB at 45.56KiB/s ETA 00:26 [download] 71.9% of 4.11MiB at 45.56KiB/s ETA 00:25 [download] 73.0% of 4.11MiB at 45.56KiB/s ETA 00:24 [download] 74.1% of 4.11MiB at 45.56KiB/s ETA 00:23 [download] 75.2% of 4.11MiB at 45.56KiB/s ETA 00:22 [download] 76.2% of 4.11MiB at 45.56KiB/s ETA 00:21 [download] 77.3% of 4.11MiB at 45.56KiB/s ETA 00:20 [download] 78.4% of 4.11MiB at 45.56KiB/s ETA 00:19 [download] 79.5% of 4.11MiB at 45.56KiB/s ETA 00:18 [download] 80.6% of 4.11MiB at 45.56KiB/s ETA 00:17 [download] 81.7% of 4.11MiB at 45.56KiB/s ETA 00:16 [download] 82.7% of 4.11MiB at 45.56KiB/s ETA 00:15 [download] 83.8% of 4.11MiB at 45.56KiB/s ETA 00:14 [download] 84.9% of 4.11MiB at 45.56KiB/s ETA 00:13 [download] 86.0% of 4.11MiB at 45.56KiB/s ETA 00:12 [download] 87.1% of 4.11MiB at 45.56KiB/s ETA 00:11 [download] 88.2% of 4.11MiB at 45.56KiB/s ETA 00:10 [download] 89.2% of 4.11MiB at 45.56KiB/s ETA 00:09 [download] 90.3% of 4.11MiB at 45.56KiB/s ETA 00:08 [download] 91.4% of 4.11MiB at 45.56KiB/s ETA 00:07 [download] 92.5% of 4.11MiB at 45.56KiB/s ETA 00:06 [download] 93.6% of 4.11MiB at 45.56KiB/s ETA 00:05 [download] 94.7% of 4.11MiB at 45.56KiB/s ETA 00:04 [download] 95.7% of 4.11MiB at 45.56KiB/s ETA 00:03 [download] 96.8% of 4.11MiB at 45.56KiB/s ETA 00:02 [download] 97.0% of 4.11MiB at 45.56KiB/s ETA 00:02 ERROR: giving up after 0 retries File "pluradl.py", line 444, in main() File "pluradl.py", line 440, in main download_courses(courses) File "pluradl.py", line 390, in download_courses if pluradl(course): File "pluradl.py", line 181, in pluradl return invoke_download(course_id, course_url, coursepath) File "pluradl.py", line 117, in invoke_download pdl.download([course_url]) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 2019, in download url, force_generic_extractor=self.params.get('force_generic_extractor', False)) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 808, in extract_info return self.process_ie_result(ie_result, download, extra_info) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 1008, in process_ie_result extra_info=extra) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 901, in process_ie_result new_result, download=download, extra_info=extra_info) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 863, in process_ie_result return self.process_video_result(ie_result, download=download) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 1644, in process_video_result self.process_info(new_info) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 1926, in process_info success = dl(filename, info_dict) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 1865, in dl return fd.download(name, info) File "/home/robc/pluradl.py/plura_dl/downloader/common.py", line 366, in download return self.real_download(filename, info_dict) File "/home/robc/pluradl.py/plura_dl/downloader/http.py", line 353, in real_download self.report_error('giving up after %s retries' % retries) File "/home/robc/pluradl.py/plura_dl/downloader/common.py", line 165, in report_error self.ydl.report_error(*args, **kargs) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 625, in report_error self.trouble(error_message, tb) File "/home/robc/pluradl.py/plura_dl/PluraDL.py", line 587, in trouble tb_data = traceback.format_list(traceback.extract_stack())

Something went wrong. The download request for 'comptia-project-plus-pt1' was forced to terminate. Double check that https://app.pluralsight.com/library/courses/comptia-project-plus-pt1 exists or that your subscription is valid for accessing its content.

lonegeek commented 4 years ago

Running debian via windows linux subsystem

rojter-tech commented 4 years ago

@lonegeek hmm, somehow your rate limit add upp to 45.56KiB, this is my output for the first video download:

[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] plura-dl version 1.0.0b4
[debug] Git HEAD: f6df2a3
[debug] Python version 3.7.6 (CPython) - Linux-4.19.114-1-surface-lts-x86_64-with-arch-Arch-Linux
[debug] exe versions: ffmpeg 4.2.2, ffprobe 4.2.2, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[pluralsight:course] comptia-project-plus-pt1: Downloading JSON metadata
[download] Downloading playlist: CompTIA Project+: Part 1
[pluralsight:course] playlist CompTIA Project+: Part 1: Collected 109 video ids (downloading 109 of them)
[download] Downloading video 1 of 109
[pluralsight] Downloading login page
[pluralsight] Logging in
[pluralsight] comptia-project-plus-pt1-m01-0: Downloading JSON metadata
[pluralsight] comptia-project-plus-pt1-m01-0: Downloading mp4-high viewclip graphql
[pluralsight] comptia-project-plus-pt1-m01-0: Waiting for 3 seconds to avoid throttling
[pluralsight] comptia-project-plus-pt1-m01-0: Downloading captions JSON
[info] Writing video subtitles to: 001-1-Getting_Started-1024x768.en.srt
[debug] Invoking downloader on 'https://vid5.pluralsight.com/expiretime=1586206964/7b9dc3139a2b6ec2063abedd8b3b4cdd/clips/resolution/fa4e6475-6c58-4bfc-8e0f-c23d5a9faef1/current/mp4/1024x768.mp4'
[download] Sleeping 91.82 seconds...
[download] Destination: 001-1-Getting_Started-1024x768.mp4
[download]   0.0% of 11.07MiB at 857.03KiB/s ETA 00:14
[download]   0.0% of 11.07MiB at 784.96KiB/s ETA 00:14
.
.
.
[download] 100.0% of 11.07MiB at 976.16KiB/s ETA 00:00
[download] 100% of 11.07MiB in 00:11

Sorry for asking but are you on ADSL, mobile network, or something?

rojter-tech commented 4 years ago

I am still downloading videos from this playlist ...:

.
.
.
[download] Downloading video 7 of 109
[pluralsight] comptia-project-plus-pt1-m02-5: Downloading JSON metadata
[pluralsight] comptia-project-plus-pt1-m02-5: Downloading mp4-high viewclip graphql
[pluralsight] comptia-project-plus-pt1-m02-5: Waiting for 4 seconds to avoid throttling
[pluralsight] comptia-project-plus-pt1-m02-5: Downloading captions JSON
[info] Writing video subtitles to: 007-2-What_We_Covered-1024x768.en.srt
[debug] Invoking downloader on 'https://vid.pluralsight.com/clips/resolution/651b6115-c02d-49d1-a8f6-874b3fefb2ee/current/mp4/1024x768.mp4?1EDdPZA3ZfqSbHvXcnDyXPLSae2o6a4b6BJUBnGYONfHeIWMJ66TFk7U8Zrvqx0UzIB4Fc_rKohvs8ZhWMF86XBrs3CrEk_vYcFP3oVfVC-XULJKqg-RSEdkDAKNEK-ZoGZsiGTY2ZkGcy1LOO851UK5j7Xq6uaXo8MEku7ChMWbKXcocZoGXlXmyVBS'
[download] Sleeping 130.40 seconds...
[download] Destination: 007-2-What_We_Covered-1024x768.mp4
[download]   0.0% of 2.02MiB at 814.43KiB/s ETA 00:02
[download]   0.1% of 2.02MiB at 904.99KiB/s ETA 00:02
[download]   0.3% of 2.02MiB at 943.87KiB/s ETA 00:02
[download]   0.7% of 2.02MiB at 962.56KiB/s ETA 00:02
[download]   1.5% of 2.02MiB at 964.80KiB/s ETA 00:02
[download]   3.0% of 2.02MiB at 959.40KiB/s ETA 00:02
[download]   6.1% of 2.02MiB at 971.68KiB/s ETA 00:01
[download]  12.3% of 2.02MiB at 968.73KiB/s ETA 00:01
[download]  24.7% of 2.02MiB at 972.14KiB/s ETA 00:01
[download]  49.4% of 2.02MiB at 973.00KiB/s ETA 00:01
[download]  96.5% of 2.02MiB at 975.37KiB/s ETA 00:00
[download] 100.0% of 2.02MiB at 976.08KiB/s ETA 00:00
[download] 100% of 2.02MiB in 00:02
lonegeek commented 4 years ago

I'm on gig fiber. I throttled my connect back in the config file when the first crash happened.

lonegeek commented 4 years ago

I'm going to build on windows and see if it works. Thanks for developing this script

lonegeek commented 4 years ago

Building it on windows 10 worked great. Thanks!