When Can Support for MITx be added?

kunal16890 commented 6 years ago

🚨Please review the Troubleshooting section before reporting any issue. Don't forget also to check the current issues to avoid duplicates.

Subject of the issue

When Can Support for MITx be added? Need to download this course: https://mitxpro.mit.edu/courses/course-v1:MITxPRO+DSx+3T2017/course/

Your environment

Operating System (name/version): Windows 7.
Python version: 3.6
youtube-dl version: Latest as on June 15th
edx-dl version: Latest as on June 15th

Steps to reproduce

https://mitxpro.mit.edu/courses/course-v1:MITxPRO+DSx+3T2017/course/

Expected behaviour

My courses on EDX got downloaded

Actual behaviour

Tell us what happens instead. If the script fails, please copy the entire output of the command or the stacktrace (don't forget to obfuscate your username and password). If you cannot copy the exception, attach a screenshot.

rbrito commented 6 years ago

The problem with supporting mitxpro is that no current developer of edx-dl has access to it to log in and to see what works and what doesn't (these courses are paid, right?).

Furthermore, it seems that edx (the organization) isn't favorable to the use of edx-dl... :cry:

We would need to either drop support for mitxpro or have some contributor with that access that can test/develop edx-dl...

kunal16890 commented 6 years ago

Hi Rogerio,

Thanks for reverting. Though I am not a developer, I can share my credentials for a limited period (say 3 months) for someone to develop and test edx-dl package for mitxpro. Yes, its a paid account but my issues is you can see it for only a limited amount of time 6 months after course ended. Information is well condensed, so I would like to revisit the videos especially after paying $1100.

Can I get blacklisted from MITxPRO, by sharing my credentials? if not, let me know if I can help!

Regards, Kunal

On Tue, 19 Jun 2018 at 13:55, Rogério Brito notifications@github.com wrote:

The problem with supporting mitxpro is that no current developer of edx-dl has access to it to log in and to see what works and what doesn't (these courses are paid, right?).

Furthermore, it seems that edx (the organization) isn't favorable to the use of edx-dl... 😢

We would need to either drop support for mitxpro or have some contributor with that access that can test/develop edx-dl...

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/coursera-dl/edx-dl/issues/509#issuecomment-398317529, or mute the thread https://github.com/notifications/unsubscribe-auth/AHHzMQ31ARzS-EYt7fcwTUdgGEFdl760ks5t-LWRgaJpZM4Uolt3 .

kyilmaz80 commented 6 years ago

I managed to download the mitx pro videos and file with changing parsing.py and edx_dl.py. In parsing.py file I changed the line at 415

if (url.startswith('https://courses.edx.org')

with

if (url.startswith('https://courses.edx.org') or
    url.startswith('https://mitxpro.mit.edu')
):

and the line at 391

name=s.a.div.span.string.strip())

with

name=s.a.div.div.string.strip())

I changed the edx_dl.py at line 718

urlretrieve(url, filename)

with

import requests
if 'zip' in url:
    urlretrieve(url, filename)
else:
    r = requests.get(url, headers=headers)
    with open(filename, 'wb') as fp:
        fp.write(r.content)

It's a little dirty but it works. The script also downloads the pdf and zip files. I tested with my own MITx Pro account.

Regards, -Koray

kunal16890 commented 6 years ago

Thanks Koray for reverting! :)

I did all the changes, and double checked the formatting/indentation. Issue is I am being directed to edx webpage and not https://mitxpro.mit.edu. Screenshots attached. Let me know if you can help.

Also, my course is in the archived state, will it have an impact?

kyilmaz80 commented 6 years ago

Can you use -x mitxpro arguement please? Because the script logins to edx by default. We have to change it so that it will login to mitxpro site. Also please run the local changed edx_dl.py file via .\edx_dl.py. Because the changes did not commited to the repository and have to be tested. So you have to do git clone the repository and change the lines.

Regards, -Koray

kunal16890 commented 6 years ago

Hi, Thanks a lot. I was actually able to access the course. BUT, the command returned No downloadable videos found.

kyilmaz80 commented 6 years ago

The sections could not be parsed with the current version. Just download the source, then patch the source with mitxpro_patch.zip

#unzip master.zip
cd edx-dl
#unzip mitxpro_patch.zip to edx-dl
./patch.py mitxpro.patch

Regards, -Koray

LookyYami commented 6 years ago

@kyilmaz80 thank you! Your fix worked perfectly =)

rbrito commented 6 years ago

@kyilmaz80, can you submit a pull request with your changes, so that we can integrate them and every user benefit from your work?

Thanks,

Rogério.

kyilmaz80 commented 6 years ago

I submitted the pull request. Sorry for late response. It is my first pull request :)

Regards, -Koray

Arshad83 commented 5 years ago

I managed to download the mitx pro videos and file with changing parsing.py and edx_dl.py. In parsing.py file I changed the line at 415
if (url.startswith('https://courses.edx.org')
with
if (url.startswith('https://courses.edx.org') or
    url.startswith('https://mitxpro.mit.edu')
):
and the line at 391
name=s.a.div.span.string.strip())
with
name=s.a.div.div.string.strip())
I changed the edx_dl.py at line 718
urlretrieve(url, filename)
with
import requests
if 'zip' in url:
    urlretrieve(url, filename)
else:
    r = requests.get(url, headers=headers)
    with open(filename, 'wb') as fp:
        fp.write(r.content)
It's a little dirty but it works. The script also downloads the pdf and zip files. I tested with my own MITx Pro account.

Regards, -Koray

Mitxpro courses are paid one, But how to get those courses. you can please provide the courses. I am unable to pay for courses becoz the cost of courses is too high.

MATRIX30 commented 4 years ago

@kunal16890 @rbrito @kyilmaz80 @LookyYami Please can you guys help us with a drive link to

MITxpro Data science course.

that you have downloaded It's pretty expensive some of us can't afford

rbrito commented 4 years ago

Since I was mentioned, I don't really have the money right now to experiment with MITxpro.

ichit commented 4 years ago

pls sir help us fix edx-dl downloader. Thanks

MATRIX30 commented 4 years ago

Since I was mentioned, I don't really have the money right now to experiment with MITxpro.

Ok Sir, @rbrito your contributions to edx-dl is so amazing would take this opportunity to say thanks immensely 🙏🙏

coursera-dl / edx-dl