Missing something obvious: urllib 400 error on simple requests

[pkedrosky]

When I try to run the scraper using a simple test and request as follows, it consistently fails as show with this error traceback. Any idea where I am going awry? Apologies for my idiocy here, but not obvious to me why urllib is throwing a 400 error.

from amazon_scraper import AmazonScraper
amzn = AmazonScraper("XXXX", "XXXX", "XXXX")
import itertools
for p in itertools.islice(amzn.search(Keywords='python', SearchIndex='Books'), 5):
   print p.title

And here is the result:

Traceback (most recent call last):
  File "amazon-scraper.py", line 4, in <module>
    for p in itertools.islice(amzn.search(Keywords='python', SearchIndex='Books'), 5):
  File "/Users/pk/anaconda/lib/python2.7/site-packages/amazon_scraper/__init__.py", line 188, in search
    for p in self.api.search(**kwargs):
  File "/Users/pk/anaconda/lib/python2.7/site-packages/amazon/api.py", line 519, in __iter__
    for page in self.iterate_pages():
  File "/Users/pk/anaconda/lib/python2.7/site-packages/amazon/api.py", line 535, in iterate_pages
    yield self._query(ItemPage=self.current_page, **self.kwargs)
  File "/Users/pk/anaconda/lib/python2.7/site-packages/amazon/api.py", line 548, in _query
    response = self.api.ItemSearch(ResponseGroup=ResponseGroup, **kwargs)
  File "/Users/pk/anaconda/lib/python2.7/site-packages/bottlenose/api.py", line 242, in __call__
    {'api_url': api_url, 'cache_url': cache_url})
  File "/Users/pk/anaconda/lib/python2.7/site-packages/bottlenose/api.py", line 203, in _call_api
    return urllib2.urlopen(api_request, timeout=self.Timeout)
  File "/Users/pk/anaconda/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/Users/pk/anaconda/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/Users/pk/anaconda/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Users/pk/anaconda/lib/python2.7/urllib2.py", line 475, in error
    return self._call_chain(*args)
  File "/Users/pk/anaconda/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/Users/pk/anaconda/lib/python2.7/urllib2.py", line 558, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error 400: Bad Request

[adamlwgriffiths]

Strange. The function you're calling is a thin wrapper around Amazon Simple API to provide the same functionality. It literally redirects to that API.

Are you sure you're registered for ad revenue and have an associate tag?

Can you try using the Amazon Simple API directly and seeing if you get similar results? https://pypi.python.org/pypi/python-amazon-simple-product-api/

[pkedrosky]

Yes, that gives the same error. And yes, I do have a valid Amazon associate tag. Perhaps it's not properly connected to the AWS account somehow?

[adamlwgriffiths]

There might be a delay between creation and activation if it's new, not sure. I wouldn't count on that being the issue, the error is 400, not 403 which I would expect in that case.

Perhaps try (if you haven't) both git HEAD and the latest Amazon Simple API from PyPi. Also check your version of bottlenose, there may be an update if you're using an old version. Failing that, file a bug on Amazon Simple API. I'm at a loss as to what is wrong.

[apapillon]

With Scratchpad (http://webservices.amazon.fr/scratchpad/index.html), you receive the same error ?

[hermanschaaf]

For anyone finding this thread through Google: I had the same issue, and it was resolved by the extra step of authorising my account for API access (as suggested here). This is separate from joining the Associates Program.

To do so, you can go to https://affiliate-program.amazon.com/gp/advertising/api/registration/pipeline.html

Hope this helps someone!

[chaitra-kamath]

How did you 'authorize' the account? I have joined the associate program and am on the step 3: Success. I have joined the program, but do not find any link to 'authorize' my account for API access? I am getting the same 400 error.

Appreciate your comments and help!

Best, Chai

[adamlwgriffiths]

I think you should take these questions to StackOverflow.