dgorissen / coursera-dl

A script for downloading course material (video's, pdfs, quizzes, etc) from coursera.org
http://dirkgorissen.com/2012/09/07/coursera-dl-a-coursera-download-script/
GNU General Public License v3.0
1.74k stars 299 forks source link

Fails going through proxy server that has an expired certificate #136

Open rtalexander opened 10 years ago

rtalexander commented 10 years ago

coursera-dl, when executed with the following command line:

coursera-dl -x https://wall.ad.abc.com:8080 -u XXXXXXXXX@XXX.com -p guess  pgm

fails with the following error:

Coursera-dl v2.0.1 (html.parser)
Logging in as 'XXXXXXXXX@XXX.com'...
Traceback (most recent call last):
  File "/usr/local/bin/coursera-dl", line 9, in <module>
    load_entry_point('coursera-dl==2.0.1', 'console_scripts', 'coursera-dl')()
  File "/usr/local/lib/python2.7/site-packages/courseradownloader/courseradownloader.py", line 642, in main
    d.login(args.course_names[0])
  File "/usr/local/lib/python2.7/site-packages/courseradownloader/courseradownloader.py", line 90, in login
    res = s.get(url, timeout=self.TIMEOUT, verify=None)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 397, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 385, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 488, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 390, in send
    raise SSLError(e)
requests.exceptions.SSLError: hostname 'wall.ad.abc.com' doesn't match either of '*.coursera.org', 'coursera.org'

I eventually tracked the problem down to to the fact that the proxy server that I use has an expired certificate. The correct thing to do, of course, is to fix the certificate problem. But for various reasons beyond my control, that's not going to happen.

The solution I came up with is to simply have the method that actually sends the request to the proxy server (i.e., requests.Session.request()) to not verify the certificate. This is accomplished by simply passing False as the value of the verify keyword argument.

I edited coursera-dl.py and added that argument to the call to request.Session.get(), which is somewhere around line 396 (the line number may be off as I've been monkeying around with the code). The following should be enough context:

res = s.get(url, timeout=self.TIMEOUT, verify=False)
        if res.status_code == 404:
            raise Exception("Unknown class %s" % className)
        res.close()

Please consider adding a command line argument that would allow the user to specify that certificate checking should not be done.

Thanks!

dgorissen commented 10 years ago

Thanks. Can you please issue a pull request so this can be merged.