This Java package enables to scrape and parse Google, Bing and Baidu search results using SerpApi. Feel free to fork this repository to add more backends.
This project is an implementation of SerpApi in Java 7. This code depends on GSON for efficient JSON processing. The HTTP response are converted to JSON Object.
An example is provided in the test. @see src/test/java/GoogleSearchImplementationTest.java
The full documentation is available here.
Runtime:
For development:
Edit your build.gradle file
repositories {
maven { url "https://jitpack.io" }
}
dependencies {
implementation 'com.github.serpapi:google-search-results-java:2.0.2'
}
To list all the version available. https://jitpack.io/api/builds/com.github.serpapi/google-search-results-java
Note: jitpack.io enables to download maven package directly from github release.
To get started with this project in Java. We provided a fully working example.
git clone https://github.com/serpapi/google_search_results_java.git
cd google_search_results_java/demo
make run api_key=<your private key>
Note: You need an account with SerpApi to obtain this key from: https://serpapi.com/dashboard
file: demo/src/main/java/demo/App.java
public class App {
public static void main(String[] args) throws SerpApiSearchException {
if(args.length != 1) {
System.out.println("Usage: app <secret api key>");
System.exit(1);
}
String location = "Austin,Texas";
System.out.println("find the first Coffee in " + location);
// parameters
Map<String, String> parameter = new HashMap<>();
parameter.put("q", "Coffee");
parameter.put("location", location);
parameter.put(GoogleSearch.SERP_API_KEY_NAME, args[0]);
// Create search
GoogleSearch search = new GoogleSearch(parameter);
try {
// Execute search
JsonObject data = search.getJson();
// Decode response
JsonArray results = (JsonArray) data.get("local_results");
JsonObject first_result = results.get(0).getAsJsonObject();
System.out.println("first coffee: " + first_result.get("title").getAsString() + " in " + location);
} catch (SerpApiSearchException e) {
System.out.println("oops exception detected!");
e.printStackTrace();
}
}
}
This example runs a search about "coffee" using your secret api key.
The Serp API service (backend)
Alternatively, you can search:
See the playground to generate your code. https://serpapi.com/playground
The Serp API key can be set globally using a singleton pattern.
GoogleSearch.serp_api_key_default = "Your Private Key"
search = GoogleSearch(parameter)
Or the Serp API key can be provided for each query.
search = GoogleSearch(parameter, "Your Private Key")
query_parameter = {
"q": "query",
"google_domain": "Google Domain",
"location": "Location Requested",
"device": device,
"hl": "Google UI Language",
"gl": "Google Country",
"safe": "Safe Search Flag",
"num": "Number of Results",
"start": "Pagination Offset",
"serp_api_key": "Your SERP API Key",
"tbm": "nws|isch|shop",
"tbs": "custom to be search criteria",
"async": true|false, // allow async request - non-blocker
"output": "json|html", // output format
}
query = GoogleSearch.new(query_parameter)
query.parameter.put("location", "Austin,Texas")
String html_results = query.getHtml()
JsonObject json_results = query.getJson()
We love true open source, continuous integration and Test Drive Development (TDD). We are using RSpec to test our infrastructure around the clock to achieve the best QoS (Quality Of Service).
The directory test/ includes specification/examples.
To run the test:
gradle test
GoogleSearch search = new GoogleSearch(new HashMap<String, String());
JsonArray locationList = search.getLocation("Austin", 3);
System.out.println(locationList.toString());
it prints the first 3 location matching Austin (Texas, Texas, Rochester)
Let's run a search to get a search_id.
Map<String, String> parameter = new HashMap<>();
parameter.put("q", "Coffee");
parameter.put("location", "Austin,Texas");
GoogleSearch search = new GoogleSearch(parameter);
JsonObject result = search.getJson();
int search_id = result.get("search_metadata").getAsJsonObject().get("id").getAsInt();
Now let retrieve the previous search from the archive.
JsonObject archived_result = search.getSearchArchive(search_id);
System.out.println(archived_result.toString());
it prints the search from the archive.
Get account API
GoogleSearch.serp_api_key_default = "Your Private Key"
GoogleSearch search = new GoogleSearch();
JsonObject info = search.getAccount();
System.out.println(info.toString());
it prints your account information.
You must clone this repository.
git clone https://github.com/serpapi/google_search_results_java.git
Build the jar file.
gradle build
Copy the jar to your project lib/ directory.
cp build/libs/google_search_results_java.jar path/to/yourproject/lib
make test
This service supports Google Images, News, Shopping. To enable a type of search, the field tbm (to be matched) must be set to:
The full documentation is available here.
javax.net.ssl.SSLHandshakeException
SerpApi is using HTTPS / SSLv3. Older JVM version do not support this protocol because it's more recent.
Upgrade java to 1.8_201+ (which is recommended by Oracle).
On OSX you can switch versino of Java.
export JAVA_HOME=`/usr/libexec/java_home -v 1.8.0_201`
java -version
On Windows manually upgrade your JDK / JVM to the latest.
On Linux, Oracle JDK 8 (1.8_151+) seems to work fine. see: https://travis-ci.org/serpapi/google-search-results-java
Victor Benarbia - victor@serpapi.com