search

mikeshultz / whatthediff

Application to track the changes between pages. Used for whatthediff.com
GNU General Public License v2.0
1 stars 0 forks source link

403 Error When Fetching some documents #22

Closed mikeshultz closed 9 years ago

mikeshultz commented 9 years ago 
HTTPError at /document/new
HTTP Error 403: Forbidden
Request Method: POST
Request URL:    http://localhost:8000/document/new
Django Version: 1.8.3
Exception Type: HTTPError
Exception Value:    
HTTP Error 403: Forbidden
Exception Location: /usr/lib64/python3.4/urllib/request.py in http_error_default, line 587
Python Executable:  /home/mike/.virtualenvs/whatthediff/bin/python
Python Version: 3.4.3
Python Path:    
['/home/mike/dev/whatthediff',
 '/home/mike/.virtualenvs/whatthediff/lib/python3.4/site-packages/django_tenants-1.0.3-py3.4.egg',
 '/home/mike/.virtualenvs/whatthediff/lib/python34.zip',
 '/home/mike/.virtualenvs/whatthediff/lib/python3.4',
 '/home/mike/.virtualenvs/whatthediff/lib/python3.4/plat-linux',
 '/home/mike/.virtualenvs/whatthediff/lib/python3.4/lib-dynload',
 '/usr/lib64/python3.4',
 '/usr/lib/python3.4',
 '/usr/lib/python3.4/plat-linux',
 '/home/mike/.virtualenvs/whatthediff/lib/python3.4/site-packages']
Server time:    Mon, 26 Oct 2015 06:24:16 +0000
mikeshultz commented 9 years ago

Possibly related: http://stackoverflow.com/questions/3336549/pythons-urllib2-why-do-i-get-error-403-when-i-urlopen-a-wikipedia-page

wtd should probably have a custom user agent, at least.

mikeshultz commented 9 years ago

Appears that the default python user agent is blocked(probably by popular CMS software), probably to prevent careless malicious users. Setup a custom user agent, so at least we're properly identifying, but that's the best I can do for this issue.