search

Mattschillinger / wikiteam

Automatically exported from code.google.com/p/wikiteam
0 stars 0 forks source link

dumpgenerator should follow redirects #46

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
http://controls.engin.umich.edu/wiki/api.php (taken from the new list of 7000 
wikis) redirects to HTTPS and I get

Checking api.php...
api.php is OK
Traceback (most recent call last):
  File "dumpgenerator.py", line 1029, in <module>
    main()
  File "dumpgenerator.py", line 838, in main
    config, other = getParameters(params=params)
  File "dumpgenerator.py", line 795, in getParameters
    if checkIndexphp(config['index']):
  File "dumpgenerator.py", line 818, in checkIndexphp
    f = urllib2.urlopen(req)
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 513, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 432, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 372, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 598, in http_error_302
    new = self.redirect_request(req, fp, code, msg, headers, newurl)
  File "/usr/lib/python2.7/urllib2.py", line 559, in redirect_request
    raise HTTPError(req.get_full_url(), code, msg, headers, fp)
urllib2.HTTPError: HTTP Error 307: Temporary Redirect

Original issue reported on code.google.com by nemow...@gmail.com on 8 Apr 2012 at 7:22

GoogleCodeExporter commented 8 years ago 
Still true:

Checking api.php... http://controls.engin.umich.edu/wiki/api.php
api.php is OK
Traceback (most recent call last):
  File "dumpgenerator.py", line 1196, in <module>
    main()
  File "dumpgenerator.py", line 1153, in main
    config, other = getParameters(params=params)
  File "dumpgenerator.py", line 917, in getParameters
    if checkIndexphp(config['index']):
  File "dumpgenerator.py", line 943, in checkIndexphp
    f = urllib2.urlopen(req)
  File "/usr/lib/python2.6/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.6/urllib2.py", line 397, in open
    response = meth(req, response)
  File "/usr/lib/python2.6/urllib2.py", line 510, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.6/urllib2.py", line 429, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.6/urllib2.py", line 369, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.6/urllib2.py", line 584, in http_error_302
    new = self.redirect_request(req, fp, code, msg, headers, newurl)
  File "/usr/lib/python2.6/urllib2.py", line 556, in redirect_request
    raise HTTPError(req.get_full_url(), code, msg, headers, fp)
urllib2.HTTPError: HTTP Error 307: Temporary Redirect

Original comment by nemow...@gmail.com on 5 Nov 2013 at 8:35

GoogleCodeExporter commented 8 years ago 
Should get the true URL with .geturl() 
http://docs.python-requests.org/en/latest/user/quickstart/#redirection-and-histo
ry
Or just use Requests instead... 
http://docs.python-requests.org/en/latest/user/quickstart/#redirection-and-histo
ry

Original comment by nemow...@gmail.com on 5 Nov 2013 at 9:48

GoogleCodeExporter commented 8 years ago

Original comment by nemow...@gmail.com on 27 Feb 2014 at 12:11