search

r0oth3x49 / acloud-dl

A cross-platform python based utility to download courses from acloud.guru for personal offline use.
MIT License
351 stars 142 forks source link

Issue in downloading AWS CSAA, CSAP, VMware courses #47

Closed rakeshkanagaraj1990 closed 4 years ago

rakeshkanagaraj1990 commented 4 years ago

Couldn't download following courses getting below errors

I have a good internet connection able to download other courses

[] : Downloading chapter : (1 of 13) [+] : Chapter (01 Introduction) [] : Found (3) lectures ...

[] : Lecture(s) : (1 of 3) [] : Downloading (001 Introduction) [*] : 23.79MB/23.79MB 100.00% |##############################| 839.51kB/s [+] : Downloaded (001 Introduction)

[] : Lecture(s) : (2 of 3) [] : Downloading (002 The Exam Blueprint) [*] : Lecture : '002 The Exam Blueprint' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (3 of 3) [] : Downloading (003 Why Should I Learn and Get Certified with AWS) [*] : Lecture : '003 Why Should I Learn and Get Certified with AWS' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (2 of 13) [+] : Chapter (02 AWS - 10000-Foot Overview) [] : Found (4) lectures ...

r0oth3x49 commented 4 years ago

Couldn't download following courses getting below errors

* AWS Certified Solutions Architect Associate 2020

* AWS Certified Solutions Architect - Professional 2020

* VMware Cloud on AWS Management Exam (5VO-31.19)

I have a good internet connection able to download other courses

[] : Downloading chapter : (1 of 13) [+] : Chapter (01 Introduction) [] : Found (3) lectures ...

[] : Lecture(s) : (1 of 3) [] : Downloading (001 Introduction) [*] : 23.79MB/23.79MB 100.00% |##############################| 839.51kB/s [+] : Downloaded (001 Introduction)

[] : Lecture(s) : (2 of 3) [] : Downloading (002 The Exam Blueprint) [*] : Lecture : '002 The Exam Blueprint' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (3 of 3) [] : Downloading (003 Why Should I Learn and Get Certified with AWS) [*] : Lecture : '003 Why Should I Learn and Get Certified with AWS' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (2 of 13) [+] : Chapter (02 AWS - 10000-Foot Overview) [] : Found (4) lectures ...

i tried the this one AWS Certified Solutions Architect - Professional 2020 and worked fine for me, i was not able to reproduce the error.

salmanparacha commented 4 years ago

Very similar issues as reported by rakeshkanagaraj1990 $ python --version Python 3.7.4 $ git --version git version 2.23.0.windows.1

Here is the error log. Please note that few lectures were downloaded successfully. e.g. lecture "Downloading (001 Welcome to the Course) and the last lecture

[55] : AWS Certified Solutions Architect - Professional 2020 [56] : AWS Certified DevOps Engineer - Professional 2020 [?] : select course number or range (1/56/range): 51

[] : Course 'VMware Cloud on AWS Management Exam (5VO-31.19)'. [] : Downloaded course information .. (done) [+] : Chapter(s) (6). [*] : Lecture(s) (27).

[] : Downloading chapter : (1 of 6) [+] : Chapter (01 Introduction to the Course) [] : Found (2) lectures ...

[] : Lecture(s) : (1 of 2) [] : Downloading (001 Welcome to the Course) [*] : 54.57MB/54.57MB 100.00% |##############################| 7.71MB/s ta 00:00s [+] : Downloaded (001 Welcome to the Course)

[] : Lecture(s) : (2 of 2) [] : Downloading (002 VMware Cloud on AWS) [*] : Lecture : '002 VMware Cloud on AWS' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (2 of 6) [+] : Chapter (02 Getting Started) [] : Found (8) lectures ...

[] : Lecture(s) : (1 of 8) [] : Downloading (001 Introduction) [*] : Lecture : '001 Introduction' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (2 of 8) [] : Downloading (002 Setting up VMware Cloud on AWS) [*] : Lecture : '002 Setting up VMware Cloud on AWS' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (3 of 8) [] : Downloading (003 Software Defined Data Centers) [*] : Lecture : '003 Software Defined Data Centers' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (003 Configuration Maximums for VMware Cloud on AWS.html) [*] : Asset : '003 Configuration Maximums for VMware Cloud on AWS.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (4 of 8) [] : Downloading (004 Understanding VMware Cloud on AWS Compute) [*] : Lecture : '004 Understanding VMware Cloud on AWS Compute' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (5 of 8) [] : Downloading (005 Understanding VMware Cloud on AWS Storage) [*] : Lecture : '005 Understanding VMware Cloud on AWS Storage' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (005 vSAN Policies.html) [*] : Asset : '005 vSAN Policies.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (6 of 8) [] : Downloading (006 Understanding VMware Cloud on AWS Network) [*] : Lecture : '006 Understanding VMware Cloud on AWS Network' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (7 of 8) [] : Downloading (007 Optional Features) [*] : Lecture : '007 Optional Features' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (007 Deploy an SDDC from the VMC Console.html) [*] : Asset : '007 Deploy an SDDC from the VMC Console.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (8 of 8) [] : Downloading (008 Chapter Summary) [*] : Lecture : '008 Chapter Summary' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (3 of 6) [+] : Chapter (03 Connecting Services with VMware Cloud on AWS) [] : Found (5) lectures ...

[] : Lecture(s) : (1 of 5) [] : Downloading (001 Introduction) [*] : Lecture : '001 Introduction' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (2 of 5) [] : Downloading (002 Understanding VMware Hybrid Linked Mode) [*] : Lecture : '002 Understanding VMware Hybrid Linked Mode' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (3 of 5) [] : Downloading (003 Install Configure and Manage Hybrid Linked Mode) [*] : Lecture : '003 Install Configure and Manage Hybrid Linked Mode' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (003 Hybrid Linked Mode Prerequisites.html) [*] : Asset : '003 Hybrid Linked Mode Prerequisites.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Downloading asset(s) [] : Downloading (003 Add an Identity Source to the SDDC LDAP Domain.html) [*] : Asset : '003 Add an Identity Source to the SDDC LDAP Domain.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Downloading asset(s) [] : Downloading (003 Install the vCenter Cloud Gateway Appliance from the Graphical Installer.html) [*] : Asset : '003 Install the vCenter Cloud Gateway Appliance from the Graphical Installer.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Downloading asset(s) [] : Downloading (003 Install the vCenter Cloud Gateway Appliance Using the Command-Line Installer.html) [*] : Asset : '003 Install the vCenter Cloud Gateway Appliance Using the Command-Line Installer.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (4 of 5) [] : Downloading (004 AWS Service Connections) [*] : Lecture : '004 AWS Service Connections' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (004 Access an S3 Bucket Using an S3 Endpoint.html) [*] : Asset : '004 Access an S3 Bucket Using an S3 Endpoint.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Downloading asset(s) [] : Downloading (004 Adding Elastic File System to your VMware Virtual Machine.html) [*] : Asset : '004 Adding Elastic File System to your VMware Virtual Machine.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Downloading asset(s) [] : Downloading (004 VMware and RDS.html) [*] : Asset : '004 VMware and RDS.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (5 of 5) [] : Downloading (005 Chapter Summary) [*] : Lecture : '005 Chapter Summary' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (4 of 6) [+] : Chapter (04 Configuring Managing and Maintaining) [] : Found (5) lectures ...

[] : Lecture(s) : (1 of 5) [] : Downloading (001 Introduction) [*] : Lecture : '001 Introduction' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (2 of 5) [] : Downloading (002 Managing Identity and Roles) [*] : Lecture : '002 Managing Identity and Roles' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (002 CloudAdmin Privileges.html) [*] : Asset : '002 CloudAdmin Privileges.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (3 of 5) [] : Downloading (003 Cluster Management) [*] : Lecture : '003 Cluster Management' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (4 of 5) [] : Downloading (004 Troubleshooting and Recovery) [*] : Lecture : '004 Troubleshooting and Recovery' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (004 Shared Responsibility.html) [*] : Asset : '004 Shared Responsibility.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (5 of 5) [] : Downloading (005 Chapter Summary) [*] : Lecture : '005 Chapter Summary' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (5 of 6) [+] : Chapter (05 Disaster Recovery and High-Availability Strategies) [] : Found (5) lectures ...

[] : Lecture(s) : (1 of 5) [] : Downloading (001 Introduction) [*] : Lecture : '001 Introduction' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (2 of 5) [] : Downloading (002 Understanding VMware Site Recovery Manager) [*] : Lecture : '002 Understanding VMware Site Recovery Manager' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading asset(s) [] : Downloading (002 VMware Site Recovery Manager Documentation.html) [*] : Asset : '002 VMware Site Recovery Manager Documentation.html' (download skipped). [-] : Exception : 'NoneType' object has no attribute 'strip'

[] : Lecture(s) : (3 of 5) [] : Downloading (003 Understanding VMware Hybrid Cloud Extension) [*] : Lecture : '003 Understanding VMware Hybrid Cloud Extension' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (4 of 5) [] : Downloading (004 Understanding VMware Stretched Clusters) [*] : Lecture : '004 Understanding VMware Stretched Clusters' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (5 of 5) [] : Downloading (005 Chapter Summary) [*] : Lecture : '005 Chapter Summary' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Downloading chapter : (6 of 6) [+] : Chapter (06 Wrapping Up) [] : Found (2) lectures ...

[] : Lecture(s) : (1 of 2) [] : Downloading (001 Course Summary) [*] : Lecture : '001 Course Summary' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (2 of 2) [] : Downloading (002 Thanks and Good Luck) [*] : 36.22MB/36.22MB 100.00% |##############################| 9.35MB/s eta 00:00s [+] : Downloaded (002 Thanks and Good Luck)

JonMerlevede commented 4 years ago

I am having the same issue when trying to obtain the courses mentioned above, as well as the "Google Certified Professional Data Engineer"-course:

[*] : Course 'Google Certified Professional Data Engineer'.
[*] : Downloaded course information .. (done)
[+] : Chapter(s) (20).
[*] : Lecture(s) (125).

[*] : Downloading chapter : (1 of 20)
[+] : Chapter (01 Introduction)
[*] : Found (3) lectures ...

[*] : Lecture(s) : (1 of 3)
[*] : Downloading (001 Welcome)
[*] : Lecture : '001 Welcome' (download skipped).
[-] : URLError : either your internet connection is not working or server aborted the request

[*] : Lecture(s) : (2 of 3)
[*] : Downloading (002 About This Course)
[*] : Lecture : '002 About This Course' (download skipped).
[-] : URLError : either your internet connection is not working or server aborted the request
...

I am running this from within a clean venv and a recent version of Python (3.8.4). Downloading some other courses worked just fine.

JonMerlevede commented 4 years ago

I added a simple print statement to the download function in _shared.py in attempt to figure out what URL is that I can't download. The extraction process looks fairly complicated; I can't really do more unfortunately.

Anyway, for courses that do not work, self.url appears to contain a JSON string or object, not an URL:

[*] : Lecture(s) : (8 of 9)
[*] : Downloading (008 Thank You and Good Luck)
{'bucket': 'acloud-content', 'filePath': 'gcp-certified-professional-data-engineer/video/86907026-0c4e-7331-da62-...'}
[*] : Lecture : '008 Thank You and Good Luck' (download skipped).
[-] : URLError : either your internet connection is not working or server aborted the request

For other courses that do work, self.url contains an URL:

[*] : Lecture(s) : (2 of 3)
[*] : Downloading (002 Goodbye)
https://media.acloud.guru/kubernetes-deep-dive/...
[*] : 3.35MB/3.35MB 100.00% |##############################| 5.42MB/s
[+] : Downloaded  (002 Goodbye)

I'm not sure what makes this error hard to reproduce. Maybe what sort of link you get depends on when you signed up for the course, or from where you're accessing ACG?

ducna96 commented 4 years ago

i get the same issue, only AWS Certified Solutions Architect Associate 2020 AWS Certified Solutions Architect - Professional 2020 AWS Certified Solutions Architect Associate SAA-CO2 ....

[] : Downloading chapter : (13 of 13) [+] : Chapter (13 Good Luck) [] : Found (5) lectures ...

[] : Lecture(s) : (1 of 5) [] : Downloading (001 Good Luck and How to Book Your Exam) [*] : Lecture : '001 Good Luck and How to Book Your Exam' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

[] : Lecture(s) : (2 of 5) [] : Downloading (002 Thank You and Next Steps) [*] : Lecture : '002 Thank You and Next Steps' (download skipped). [-] : URLError : either your internet connection is not working or server aborted the request

ducna96 commented 4 years ago

I added a simple print statement to the download function in _shared.py in attempt to figure out what URL is that I can't download. The extraction process looks fairly complicated; I can't really do more unfortunately.

Anyway, for courses that do not work, self.url appears to contain a JSON string or object, not an URL:

[*] : Lecture(s) : (8 of 9)
[*] : Downloading (008 Thank You and Good Luck)
{'bucket': 'acloud-content', 'filePath': 'gcp-certified-professional-data-engineer/video/86907026-0c4e-7331-da62-...'}
[*] : Lecture : '008 Thank You and Good Luck' (download skipped).
[-] : URLError : either your internet connection is not working or server aborted the request

For other courses that do work, self.url contains an URL:

[*] : Lecture(s) : (2 of 3)
[*] : Downloading (002 Goodbye)
https://media.acloud.guru/kubernetes-deep-dive/...
[*] : 3.35MB/3.35MB 100.00% |##############################| 5.42MB/s
[+] : Downloaded  (002 Goodbye)

I'm not sure what makes this error hard to reproduce. Maybe what sort of link you get depends on when you signed up for the course, or from where you're accessing ACG?

I replaced the self.url to download with this: download_url = self.url if self.url['filePath'] == None else "https://media.acloud.guru/" + self.url['filePath']

It opened the right url to the mp4 file, but missing some parameters to access.

For example, the normal url would be: https://media.acloud.guru/aws-csa-2019/video/0d16c6b1-a88f-ecfe-157c-1bf203fdd9cb_b11e8356-0df1-e8ad-b36a-b6bcb0431b4b/720p/aws-csa-2019-710a4207-cde9-46e5-a1ee-2983bd6025f0-720p.mp4?Expires=1596004685&Signature=g7L6qLXVC8qphQzdqVsLUDQ362ZaNZM146ObRZRzlij5C2l+EfMXo5SLjgGFEnqd41nhjmg5JzE2DMdD/5ao1C8MPAZEOYVAHuQSG1OMqq6xw3EJazQOuW+sc2m333WA3BQw6MxdO1s1c4SDtlCuZmKNc8BU5Igbz5td6JkErwUaIGGOhGC2TfQyxGwMhSx82mgW9kNEYlOXA0Ea8foVWjAwwEAbeTLIVRHrFevvMEDDlF3yGVFdUVciljdlNOrtVJHai9tB4tNBBFy6JJQ8USle/VhXK+/fMkmC3ZE5Y7pGIiQlz6kbDI/AdCpCuLjeTxz587Qjg1wO/LnnrdFEaA==&Key-Pair-Id=APKAISLU6JPYU7SF6EUA

But i only got: https://media.acloud.guru/aws-csa-2019/video/0d16c6b1-a88f-ecfe-157c-1bf203fdd9cb_b11e8356-0df1-e8ad-b36a-b6bcb0431b4b/720p/aws-csa-2019-710a4207-cde9-46e5-a1ee-2983bd6025f0-720p.mp4

Any ideas how to solve this?

r0oth3x49 commented 4 years ago

I added a simple print statement to the download function in _shared.py in attempt to figure out what URL is that I can't download. The extraction process looks fairly complicated; I can't really do more unfortunately. Anyway, for courses that do not work, self.url appears to contain a JSON string or object, not an URL:

[*] : Lecture(s) : (8 of 9)
[*] : Downloading (008 Thank You and Good Luck)
{'bucket': 'acloud-content', 'filePath': 'gcp-certified-professional-data-engineer/video/86907026-0c4e-7331-da62-...'}
[*] : Lecture : '008 Thank You and Good Luck' (download skipped).
[-] : URLError : either your internet connection is not working or server aborted the request

For other courses that do work, self.url contains an URL:

[*] : Lecture(s) : (2 of 3)
[*] : Downloading (002 Goodbye)
https://media.acloud.guru/kubernetes-deep-dive/...
[*] : 3.35MB/3.35MB 100.00% |##############################| 5.42MB/s
[+] : Downloaded  (002 Goodbye)

I'm not sure what makes this error hard to reproduce. Maybe what sort of link you get depends on when you signed up for the course, or from where you're accessing ACG?

I replaced the self.url to download with this: download_url = self.url if self.url['filePath'] == None else "https://media.acloud.guru/" + self.url['filePath']

It opened the right url to the mp4 file, but missing some parameters to access.

For example, the normal url would be: https://media.acloud.guru/aws-csa-2019/video/0d16c6b1-a88f-ecfe-157c-1bf203fdd9cb_b11e8356-0df1-e8ad-b36a-b6bcb0431b4b/720p/aws-csa-2019-710a4207-cde9-46e5-a1ee-2983bd6025f0-720p.mp4?Expires=1596004685&Signature=g7L6qLXVC8qphQzdqVsLUDQ362ZaNZM146ObRZRzlij5C2l+EfMXo5SLjgGFEnqd41nhjmg5JzE2DMdD/5ao1C8MPAZEOYVAHuQSG1OMqq6xw3EJazQOuW+sc2m333WA3BQw6MxdO1s1c4SDtlCuZmKNc8BU5Igbz5td6JkErwUaIGGOhGC2TfQyxGwMhSx82mgW9kNEYlOXA0Ea8foVWjAwwEAbeTLIVRHrFevvMEDDlF3yGVFdUVciljdlNOrtVJHai9tB4tNBBFy6JJQ8USle/VhXK+/fMkmC3ZE5Y7pGIiQlz6kbDI/AdCpCuLjeTxz587Qjg1wO/LnnrdFEaA==&Key-Pair-Id=APKAISLU6JPYU7SF6EUA

But i only got: https://media.acloud.guru/aws-csa-2019/video/0d16c6b1-a88f-ecfe-157c-1bf203fdd9cb_b11e8356-0df1-e8ad-b36a-b6bcb0431b4b/720p/aws-csa-2019-710a4207-cde9-46e5-a1ee-2983bd6025f0-720p.mp4

Any ideas how to solve this?

it's not like that bro, try reading the code instead if you want to implement a fix around otherwise share your credentials on my email nasir251292@gmail.com to reproduce and fix.

ducna96 commented 4 years ago

it's not like that bro, try reading the code instead if you want to implement a fix around otherwise share your credentials on my email nasir251292@gmail.com to reproduce and fix.

check your email :D Many Thanks.

r0oth3x49 commented 4 years ago

it's not like that bro, try reading the code instead if you want to implement a fix around otherwise share your credentials on my email nasir251292@gmail.com to reproduce and fix.

check your email :D Many Thanks.

thanks, i will check tonight after office work and try to reproduce and fix the issue ASAP. currently in office. thank you for sharing the creds.

DoguD commented 4 years ago

@r0oth3x49 Is there any progress regarding the issue? I am having the same issue for most of the lectures on AWS Certified Developer Associate and for a few lectures on other courses. I tried to generate the URL from the JSON @JonMerlevede has mentioned but I couldn't find the signature query parameter anywhere.

ducna96 commented 4 years ago

Hi Khan, thanks for fix #47 :D, but i see some sub file missing download [image: image.png] Is that IMPOSSIBLE to download the sub files having the different name to the original file?

I can just download the same file names like below [image: image.png] please look over and reproduct and fix this. :D

thanks and best regard !

Nguyen Anh Duc Mobile : 0965940725 email : duc.na1006@gmail.com

Vào CN, 2 thg 8, 2020 vào lúc 15:39 Nasir Khan notifications@github.com đã viết:

Closed #47 https://github.com/r0oth3x49/acloud-dl/issues/47 via 044dce5 https://github.com/r0oth3x49/acloud-dl/commit/044dce598129c26148a0f9588ee83870e07b933f .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/r0oth3x49/acloud-dl/issues/47#event-3613161289, or unsubscribe https://github.com/notifications/unsubscribe-auth/AILFTFVMJF3O2PTYXUCGUFTR6UQ4DANCNFSM4O7ZVMFQ .