search

econchick / new-coder

New Coder tutorials
zlib License
597 stars 394 forks source link

Giant Bomb Response 403 #206

Open blogscot opened 8 years ago

blogscot commented 8 years ago

I'm running the API example, api.py, from the repository, using Python 3 on a Windows 10 machine with minor software mods. The response from Giant Bomb was quite enlightening when I viewed it in the browser:

Wordpress RSS Reader, Anonymous Bot or Scraper Blocked

Sorry we do not allow WordPress plugins to scrape our site. They tend to be used maliciously to steal our content. We do not allow scraping of any kind. You can load our RSS feeds using any other reader but you may not download our content. Click here more information on our feeds

Or you're running a bot that does not provide a unique user agent. Please provide a UNIQUE user agent that describes you. Do not use a default user agent like "PHP", "Java", "Ruby", "wget", "curl" etc. You MUST provide a UNIQUE user agent. ...and for God's sake don't impersonate another bot like Google Bot that will for sure get you permanently banned.

Or.... maybe you're running an LG Podcast player written by a 10 year old. Either way, Please stop doing that.

The request headers were:

{'User-Agent': 'python-requests/2.10.0', 'Accept-Encoding': 'gzip, deflate', 'Connection': 'keep-alive', 'Accept': '/'}

I verified that my API key works okay by manually copying the request url into the browser.

Any suggestions how I might provide a unique user agent to make the app work?

joshuamsmith commented 7 years ago

I ran into that also but adding user-agent into the headers resolved it for me: result=requests.get(self.base_url + '/platforms/', params=params, headers={'user-agent':'newcoder'})