Different results between Google API and "real" Google

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Just do a simple search in Google
2. The same search with the Google API
3. Different results between both of them, whatever the request is or the
language is (the hl parameter)

What is the expected output? 
The same first results

What do you see instead?
Totally different results

What version of the product are you using? On what operating system?
Google AJAX Search API 1.0 vs Google. All operating systems.

ex :
http://www.google.fr/search?hl=fr&q=livre&btnG=Recherche+Google&meta=
http://ajax.googleapis.com/ajax/services/search/web?v=1.0&hl=fr&q=livre

I've read the documentation and I don't see any other parameter that would
change the results, so I guess it's an issue, and if it's not an issue then
you should probably mention in the documentation that we're not supposed
the get the same results between the two of them.

Thanks a lot, 

Ugo PARSI

Original issue reported on code.google.com by ugo.pa...@gmail.com on 29 May 2008 at 1:33

[GoogleCodeExporter]

I'm working to automated the data collection about our web site indexing using 
Perl
and I'm very disapointed to see this difference between the 2 sources.
It's a pity to see a such difference in the results, especially when there is no
other choice to use the AJAX API.
Did someone know an other solution ?

Original comment by GSan...@CMRemotes.com on 6 Nov 2009 at 1:46

[GoogleCodeExporter]

GSangla, there's no other legal way. Screen scraping www.google.com is against 
the 
terms of use.

Original comment by howb...@gmail.com on 6 Nov 2009 at 2:26

[GoogleCodeExporter]

Anyone know when it will be possible to add hl=pl parameter  ?

I'm not thinking about gl=pl.

I'm still getting very wired result against http://ajax.googleapis.com and 
normal
google...

I think api, use google.com without localisation...

Any idea ?

Original comment by Admin...@gmail.com on 12 Nov 2009 at 7:11

[GoogleCodeExporter]

AdminOPZ, the API and google.com do not use the same indexes for results. From 
the lack 
of attention to this ticket from development, I assume that was intentional and 
they 
aren't going to fix it. So more than likely, you cannot correct the "weird" 
results you 
get from the API (rendering it useless, in my opinion). 

Original comment by howb...@gmail.com on 12 Nov 2009 at 8:16

[GoogleCodeExporter]

A fast and first action to show how critical it is, I would like to suggest to 
"Vote"
this issue to increase the visibility to everybody who haven't done yet.
It should be the first priority for Google Ajax Search API

Original comment by JordiSu...@gmail.com on 17 Nov 2009 at 9:52

[GoogleCodeExporter]

@JordiSuner we should blog & twit about this issue in order to get even more 
attention. I think that many people are not aware of this difference and still 
trust 
the API.

Original comment by metallourlante on 18 Nov 2009 at 11:48

[GoogleCodeExporter]

this is serious bug..

Original comment by suhas.ar...@gmail.com on 2 Dec 2009 at 10:33

[GoogleCodeExporter]

Hi,

reviewing issues related I've found issue 78, 
http://code.google.com/p/google-ajax-
apis/issues/detail?id=78.
However I've tested and still I don't get the same results. Has anyone verified?

Original comment by JordiSu...@gmail.com on 6 Dec 2009 at 10:06

[GoogleCodeExporter]

Issues still ongoing:

I recently started using the google search api for a little project
I'm working on. However it becomes very unuseable for me since a lot
of queries don't seem to return almost no results while on google.com
I get lots of results.

An example:
 - query: intitle:"joost roelandts"
 - api: http://ajax.googleapis.com/ajax/services/search/web?
v=1.0&q=intitle:"joost%20roelandts"
 - google: http://www.google.com/search?hl=en&q=intitle:"joost+roelandts"

In google I get 600 results, while api only returns 1 result.

Original comment by folke.le...@gmail.com on 6 Dec 2009 at 12:49

[GoogleCodeExporter]

JordiSuner, this ticket has been verified to be unique and it does not appear 
Google 
has any interest in fixing it, or even acknowledging that it is a bug and not 
an 
undocumented feature.

Original comment by howb...@gmail.com on 7 Dec 2009 at 2:40

[GoogleCodeExporter]

I've spent countless hours going around in circles on this but then discovered 
this
thread and at least am glad that I'm not completely imagining what's going on 
here.

When adding "site:somesite.com" to the query, I'm repeatedly seeing the most
important top 1 or 2 results not being shown when using the API which makes it
virtually useless. Very frustrating.

Original comment by mbake...@gmail.com on 8 Dec 2009 at 5:09

[GoogleCodeExporter]

Issue 359 has been merged into this issue.

Original comment by jscud.w...@gmail.com on 9 Dec 2009 at 10:48

[GoogleCodeExporter]

Issue 355 has been merged into this issue.

Original comment by jscud.w...@gmail.com on 9 Dec 2009 at 10:50

[GoogleCodeExporter]

I wonder if this is why the search feature in Google Wave is so pants

Original comment by bluesk...@gmail.com on 10 Dec 2009 at 10:17

[GoogleCodeExporter]

Same here... No results from API, lots of results from www.google.com search.
Sight... Will have to move to Yahoo API for now.

Original comment by konstant...@gmail.com on 10 Dec 2009 at 3:10

[GoogleCodeExporter]

I would like to suggest my experience about this issue.
1 month ago I build a new website, now I can see it in google.com but not in 
the API. 
Is it possible that is only a matter of time for the update regarding the API? 
any 
cache system?

Original comment by guero....@gmail.com on 11 Dec 2009 at 1:04

[GoogleCodeExporter]

same problem. No solution in 19 months.

Original comment by jorgeja...@gmail.com on 3 Jan 2010 at 12:04

[GoogleCodeExporter]

Issue 362 has been merged into this issue.

Original comment by jscud.w...@gmail.com on 6 Jan 2010 at 6:05

[GoogleCodeExporter]

I have the same issue

http://groups.google.com/group/google-ajax-search-api/browse_thread/thread/bdd4c
47256ec3f55

Is there an alternative way to get the actual results from Google to determine 
page
rank, etc?

Original comment by drew.mic...@gmail.com on 15 Jan 2010 at 12:01

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Michael - 

>Is there an alternative way to get the actual results from Google to determine 
page 
rank, etc?

Not legally, unless you want to do something by hand. The API is the only way 
Google 
(through my interpretation of their Terms of Service) wants bots to access 
their 
content.

Original comment by howb...@gmail.com on 15 Jan 2010 at 12:11

[GoogleCodeExporter]

Actually, to be clear, the TOS prohibit the use of robots, spiders, and other
applications which make automated requests, even to the API.

Original comment by jrgeer...@gmail.com on 15 Jan 2010 at 12:29

[GoogleCodeExporter]

I do like  submit form with keys
http://www.google.com/search?q=test&sitesearch=yoursite.com

Original comment by Romankon...@gmail.com on 15 Jan 2010 at 1:45

[GoogleCodeExporter]

Clearly there is no way to stop people from just scraping results from Google 
pages,
why do they not just allow direct access via the API with automation? They 
already
have limits in place for daily searches, etc. Seems very strange to spend
time/money/energy on this API when it is so limited. I see plenty of online 
sites and
software offering to do google rank checks, etc, they must be using scraping 
techniques.

Original comment by drew.mic...@gmail.com on 15 Jan 2010 at 4:59

[GoogleCodeExporter]

In reality, I'm sure Google doesn't prosecute every offender they find in a 
court of 
law or even with a cease and desist letter. And on the contrary, I'm sure many 
offenders didn't even read the TOS to realize they were even doing anything 
wrong. 

ALL THE SAME, that doesn't make it right, via benign neglect or pure ignorance. 
But I 
agree, the service as a whole is frustrating in its current incarnation. But 
that's 
what you get with free - free doesn't mean high quality, doesn't mean constant 
availability, doesn't mean it's even guaranteed to be provided in the future, 
and it 
doesn't mean supported. 

But the internet community does hold Google to a high standard, and I think 
Google 
holds itself to a high standard, so I still feel disappointed. 

Original comment by howb...@gmail.com on 15 Jan 2010 at 6:41

[GoogleCodeExporter]

I think this is a critical error.... It's very stupid if are different result, 
What
use is it?

Original comment by udelunar@gmail.com on 21 Jan 2010 at 9:46

[GoogleCodeExporter]

Agreed. I understand that Google's servers in general are updated at different 
times
(amongst other mystery reasons) so a user might see different results in one 
area of
the world/time than another user somewhere else. BUT, why not use allow an API 
user
to specify an incoming IP (or geo co-ordinates) along with the search terms, 
etc so
results match perfectly with what a user would find doing a Google search on 
their own?

Original comment by drew.mic...@gmail.com on 21 Jan 2010 at 4:09

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

I can't understand that this bug is still there almost 2 years after have been 
reported.
For me, it's a high priority bug, it's a google search api that doesn't give 
the same
result at all that a regular search on google ,so what is the purpose of this 
api if
it's unusable?

Original comment by weblayou...@gmail.com on 2 Mar 2010 at 1:54

[GoogleCodeExporter]

Dear Google-Team,
I have also checked some search results and have found other issues.

The google maps results and also the google products are not displayed..

So it is a realy high priority bug.

PLEASE FIX!

Original comment by reac...@gmail.com on 2 Mar 2010 at 10:16

[GoogleCodeExporter]

... is this even real?

Original comment by obl...@gmail.com on 21 Mar 2010 at 9:08

[GoogleCodeExporter]

I suspect Google is neglecting this on purpose. They don't make any money if 
you use 
the API, in fact, it only generates more load on their servers. There is no 
business 
reason for them to fix this. Sucks for us...

Original comment by csgyuri...@gmail.com on 22 Mar 2010 at 2:35

[GoogleCodeExporter]

Please, please fix this...

Original comment by david.po...@gmail.com on 15 May 2010 at 10:13

[GoogleCodeExporter]

so?

Original comment by garciaj...@gmail.com on 17 May 2010 at 9:27

[GoogleCodeExporter]

Very, very annoying.
I'm maintaining an iPhone SERPS App ("rankometer") which works quite accurate 
in most cases. Yesterday one 
of my customers reported exactly this problem, and I had to confirm. 
Fortunately only very few search terms 
are affected, but if this is a larger problem it would mean the end for my App. 
One has to make a living...

- Has anyone come across the theory that the API just uses older data?

- Are there any other theories about how Google distorts the results?

Please, People of Google, tell us at least weather it's worth waiting for a 
solutiuon!

Original comment by hitthing...@gmail.com on 22 May 2010 at 12:48

[GoogleCodeExporter]

To the last poster... I wouldn't expect it will get fixed. As for the variance, 
their 
searching algorithm is always changing, so I wouldn't expect there's any linear 
way to 
predict the variance without doing something illegal.

Original comment by howb...@gmail.com on 24 May 2010 at 6:15

[GoogleCodeExporter]

I would add that, given that the TOS specifically prohibit automated queries 
and the
API will return no more than 64 results across 8 separate requests, presumably 
as a
deterrent to SEO applications, it's not at all unreasonable to assume that they 
are
intentionally introducing some variance as an additional discouragement for SEO
operators.

Original comment by jrgeer...@gmail.com on 24 May 2010 at 6:31

[GoogleCodeExporter]

Issue 468 has been merged into this issue.

Original comment by jrgeer...@gmail.com on 9 Jun 2010 at 11:37

[GoogleCodeExporter]

Hi there!
I tried use country-language search, like in example 
http://code.google.com/apis/ajax/playground/?exp=search#set_google_locale_using_
hostname

but it shows different results.

Maybe you found this info useful, that in Ukraine almost all users are using 
Russian language interface, but google.com.ua domain.

I can't get adequate results by ajax, same as in plain www.google.com.ua with 
Russian interface, russian search-language, and with "search only in Ukraine".

Would be glad if someday this issue will be solved.

Thanks.

Original comment by yeren...@gmail.com on 6 Jul 2010 at 8:32

[GoogleCodeExporter]

This really sucks. It is already more than two years a bug! Please google take 
a look to this and give and anwser if this is going to be fixed or not?

Original comment by i...@poihandler.com on 10 Jul 2010 at 6:52

[GoogleCodeExporter]

Issue 414 has been merged into this issue.

Original comment by jscud.w...@gmail.com on 13 Jul 2010 at 8:23

[GoogleCodeExporter]

Hi guys are you aware of this? Does it work? 
http://googleajaxsearchapi.blogspot.com/2009/10/web-search-in-your-country.html

Original comment by webcircl...@gmail.com on 23 Jul 2010 at 6:44

[GoogleCodeExporter]

Issue 114 has been merged into this issue.

Original comment by jscud.w...@gmail.com on 4 Aug 2010 at 2:20

[GoogleCodeExporter]

I am experiencing this issue as well.

Case & point - search maps.google.com for "Where is MIT" - then do the same 
search using the api (on local search of course) and you get totally different 
results.

It is incredibly frustrating!

Original comment by iamanopt...@gmail.com on 16 Aug 2010 at 9:30

[GoogleCodeExporter]

For what it's worth, I've been experiencing the same problem between regular 
search and the api, while attempting to use site:_______

I've noticed a very strange correlation:
If I do a regular search and the results page lists the "Similar" link, it 
works fine with the api.

If I do a regular search and it does NOT have the "Similar" link, I will get 
something along the lines of this:

{"responseData": {"results":[], .......

Original comment by laet...@gmail.com on 26 Aug 2010 at 7:34

[GoogleCodeExporter]

weird, you know guys, the Bing search API doesn't cheap out like this... though 
I suspect if they knew they could probably get away with not giving away all of 
their result content like Google *seems* to be doing, they probably would and 
I'm sure their stockholders would appreciate it as well. 

Just a thought.

Yeah its a shame to hear that this VERY SAME issue that I opened up like what 2 
years ago now is still only merely speculated-- I've had at least 2 people tell 
me I'm a crackpot I'm glad some light has been shed on this. 

Original comment by paigead...@gmail.com on 27 Aug 2010 at 10:27

[GoogleCodeExporter]

This definitely looks like a political decision. Google is aware of the issue 
and they could have fixed it easily in the last two years if they wanted.

Original comment by wellnho...@aevum.de on 27 Aug 2010 at 10:46

[GoogleCodeExporter]

Hypothesis: AJAX API queries are executed against a smaller index, perhaps the 
"top 10% of the web", thus using separate and smaller resources than the 
primary index.  It is probably updated more slowly too.

Expected effects: AJAX API works fine with common all-time top queries but 
falls down with more obscure or specific queries, eg. looking things in an 
unusual language or located in a particular town, or small news that happened 
yesterday.

Original comment by graham.poulter on 30 Aug 2010 at 9:46

[GoogleCodeExporter]

Issue 509 has been merged into this issue.

Original comment by jrgeer...@gmail.com on 1 Sep 2010 at 2:02

[GoogleCodeExporter]

I as well am getting similar results as everyone else.  When searching for 
highly specific "low volume" keywords the results are VERY different.

Original comment by d...@pringle.com on 16 Sep 2010 at 6:19