search

coursera-dl / edx-dl

A simple tool to download video lectures from edx.org (and other openedx sites)
GNU Lesser General Public License v3.0
1.93k stars 640 forks source link

edX Terms of Service #392

Open e0d opened 8 years ago

e0d commented 8 years ago

🚨Please review the Troubleshooting section before reporting any issue. Don't forget also to check the current issues to avoid duplicates.

Subject of the issue

edX Terms of Service

Your environment

i wanted to point out that use of this tool violates the documented edX Terms of Service that must be accepted when registering. Please see the section entitled, RULES FOR ONLINE CONDUCT in there TOS.

Please reach out to me discuss next steps.

Expected behaviour

Tell us what should happen.

Actual behaviour

Tell us what happens instead. If the script fails, please copy the entire output of the command or the stacktrace (don't forget to obfuscate your username and password). If you cannot copy the exception, attach a screenshot.

e0d commented 8 years ago

@balta2ar @iemejia you seem to be the most recent commiters to this project, can either of you reply to this or point me in the right direction?

rbrito commented 8 years ago

@e0d, what kind of reply do you want from us?

I had a brief conversation with @balta2ar (it seems that @iemejia is short on time) regarding this bug and we are willing to have a solution that is as amicable as possible, because, well, we are all MOOC fans here.

We (@balta2ar and I) already discussed a possible technical solution/workaround to the "problem", but we would like to hear from you what we should do.

I am going to bed right now (it's almost 4am here), but I will read my emails here as soon as I wake up.

Regards,

Rogério Brito.

P.S.: Please, take into account that none of the contributors to this project (that I know of) is a native English speaker. By that, I mean that we may use phrases or constructions that may sound less polite than we want to, but I can assure you that we are all friendly here and the language barrier should be no problem, I hope, if you keep that in mind.

iemejia commented 8 years ago

Hello @e0d,

Sorry if I did not answer before. We had already some previous interactions with some members of edx.org and already discussed about various issues, including the scraping of the website, we even discussed the possible use of the mobile REST API to access the course contents (and like this avoid scraping the whole html of the site), and it is probably the next thing we intend to do, however we have not done this because this is a something we do in our free time and nobody seems to have had the time to contribute the required changes.

Notice from the previous discussions that we have always tried to be good edx citizens and we identify our request agent as well as try to put considerate rate limits to the script. The goal of this tool is not to promote abuse, but to help people with limited access to follow the courses (e.g. users who don't have a permanent internet connection).

https://github.com/coursera-dl/edx-dl/issues/148 https://github.com/coursera-dl/edx-dl/issues/377

I read the 'rules for online conduct document' section you mention' and it seems that users of this tool might be violating the rules: "...Furthermore, you agree not to scrape, or otherwise download in bulk, any Site content..."

If you want we can put a warning so people become aware of the fact that using this tool can be considered a violation of the terms of service of the website, and let them decide if they want to continue.

Other idea we can do is to try to make the scrapping sequential by default, this would be slower for the users of the script but better for you considerating the write contention issues discussed before).

Do you have other ideas in mind ?

Ismaël