search

GeneralMills / pytrends

Pseudo API for Google Trends
Other
3.28k stars 821 forks source link

HTTP Error 400: Bad Request #24

Closed smtoycom closed 8 years ago

smtoycom commented 8 years ago

Getting HTTP Error 400, Please Help

C:\Windows\System32>python C:\Users\SGG\Desktop\pytrends-master\examples\example .py Traceback (most recent call last): File "C:\Users\SGG\Desktop\pytrends-master\examples\example.py", line 10, in < module> connector = pyGTrends(google_username, google_password) File "C:\Python34\lib\site-packages\pytrends-1.1.2-py3.4.egg\pytrends\pyGTrend s.py", line 51, in init File "C:\Python34\lib\site-packages\pytrends-1.1.2-py3.4.egg\pytrends\pyGTrend s.py", line 71, in _connect File "C:\Python34\lib\urllib\request.py", line 469, in open response = meth(req, response) File "C:\Python34\lib\urllib\request.py", line 579, in http_response 'http', request, response, code, msg, hdrs) File "C:\Python34\lib\urllib\request.py", line 501, in error result = self._call_chain(_args) File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain result = func(_args) File "C:\Python34\lib\urllib\request.py", line 684, in http_error_302 return self.parent.open(new, timeout=req.timeout) File "C:\Python34\lib\urllib\request.py", line 469, in open response = meth(req, response) File "C:\Python34\lib\urllib\request.py", line 579, in http_response 'http', request, response, code, msg, hdrs) File "C:\Python34\lib\urllib\request.py", line 507, in error return self._call_chain(_args) File "C:\Python34\lib\urllib\request.py", line 441, in _call_chain result = func(_args) File "C:\Python34\lib\urllib\request.py", line 587, in http_error_default raise HTTPError(req.full_url, code, msg, hdrs, fp) urllib.error.HTTPError: HTTP Error 400: Bad Request

adstastic commented 8 years ago

Hi,

I fixed this by commenting out the self.opener.open(self.url_CookieCheck) line in _connect in pytrends.py.

However, now my CSV for the example is a bad request page:

<html>
<head>
<title>Redirecting</title>

<script type="text/javascript" language="javascript">
      // Accessing window.external members can cause IE to throw exceptions.
      // Any code that acesses window.external members must be try/catch wrapped
      /** @preserveTry */
      try {
        if (top == self) {
          if (window.gtbExternal) {
            window.gtbExternal.setM();
          } else {
            window.external.setM();
          }
        }
      }
      catch(err) {
      }
    </script>

<meta http-equiv="refresh" content="0; url=&#39;http://www.google.com/trends#cmpt=q&amp;q=pizza&amp;hl=en-US&amp;content=1&#39;"></head>
<body bgcolor="#ffffff" text="#000000" link="#0000cc" vlink="#551a8b" alink="#ff0000">
<script type="text/javascript" language="javascript">
    location.replace("http://www.google.com/trends#cmpt\x3dq\x26q\x3dpizza\x26hl\x3den-US\x26content\x3d1")
</script>
</body>
</html>

I'm not sure if these the CSV response is affected by removing the cookie check but with the cookie check in place, I get the same HTTP Error 400: Bad Request as you.

However, the get_suggestions is working fine and is returning JSON of related keywords.

Edit: When I load the URL printed to stdout with the params in a browser (and am logged in with a Google account) it automatically downloads report.csv.

dreyco676 commented 8 years ago

I'm unable to replicate this.

Please reopen if you have this issue again otherwise I'll assume one of two things happened 1) the recent pull request solved it or 2) that Google sent you an email saying you were logging in from another place and denied your login.

burakuyar commented 8 years ago

Hi,

Yesterday I started using pytrends and it was working just fine. Today, I've just ran the same code and received this error, as described in the first message. Then I tried what @adstastic did and saw the same results.

I think it's time to reopen the issue :)

rdahis commented 8 years ago

Same issue here. Any help would be greatly appreciated.

ericross2 commented 8 years ago

Please re-open: I have the same exact issue.

Pytrends worked perfectly for several weeks with zero glitches. Last night and today I get the same 400 error. I too tried what @adstastic did and saw the exact same results (same csv file dump s/he saw).

(On a side note, I greatly appreciate the work done on pytrends. Thank you for providing this.)

A bit more info (perhaps relevant): If I instead manually type the exact same url produced by pytrends in my browser (I am using Opera) it gives me a "Bad Request 400" in the browser. If I replace some of the encoding (such as using actual double quotes and commas, etc. instead of the encoded %22, etc.) it redirects me to the Google Trends main page. If I take this same url into another computer browser (already logged into the same Google account for email/etc.) it gives me a good csv.

I then set up a fresh Google account and tried it with pytrends on the first computer, but unfortunately with the same results (bad request). I cleared the Opera cache, and don't show any cookies listed.

I assume these imply that something on that PC has registered cookies or something which has decided that my account shouldn't work with Google Trends. To be clear: I am only downloading between 20-50 trends TOTAL per day (each a list of up to five terms to be compared as a single trend/url); this is not a mass-trend downloader for me.

Is there a way I can simply reset the cache/browser/cookies to allow pytrends to work again?

Any help is appreciated.

charleswm commented 8 years ago

I'm also having this issue. I haven't been able to get the package to work for even a single trend. It dies on the first line.

google_username = "" google_password = "" connector = pyGTrends(google_username, google_password)

I'm entering my username and password correctly, but I'm getting the 400 Bad Request Error.

Also, is there a way to do this without logging into a gmail account? That would be very useful.

jpforny commented 8 years ago

I also had the same issue this Tuesday. As it seems that currently Google Trends doesn't require authentication, I cloned and installed pytrends locally with authentication code commented and get_suggestions is working fine.

On Thu, Aug 25, 2016 at 4:56 PM charleswm notifications@github.com wrote:

I'm also having this issue. I haven't been able to get the package to work for even a single trend. It dies on the first line.

google_username = "" google_password = "" connector = pyGTrends(google_username, google_password)

I'm entering my username and password correctly, but I'm getting the 400 Bad Request Error.

Also, is there a way to do this without logging into a gmail account? That would be very useful.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/GeneralMills/pytrends/issues/24#issuecomment-242515974, or mute the thread https://github.com/notifications/unsubscribe-auth/ABq1UdD2_zIwrpg-Pmx1yl7BXvNnhFEsks5qjfOFgaJpZM4GoM89 .

rdahis commented 8 years ago

@jpforny Could you please give more detail on what you did? In particular, what lines did you comment out?

amagnosousa commented 8 years ago

I checked the Google Trends page and I see that the Google made changes into page and their behavior was modified. Unfortunately I think pytrends needs upgrade to work again.