CfScrape

A java library to bypass Cloudflare's anti-bot and get the page contents for webscraping app. This is the java implementation of the CfScrape api.

How to import?

Gradle

Add repository :

repositories {
    maven {
        url 'https://jitpack.io'
    }
}

Add dependency :

dependencies {
    compile 'com.github.iambluedev1:cfscrape-java:1.0.0'
}

Maven

Add repository :

<repository>
    <id>jitpack.io</id>
    <url>https://jitpack.io</url>
</repository>

Add dependency :

<dependency>
    <groupId>com.github.iambluedev1</groupId>
    <artifactId>cfscrape-java</artifactId>
    <version>1.0.0</version>
</dependency>

How to use it ?

Firstly you need to get the CfScrape object :

CfScrape cfScrape = CfScrape.get();

Set website (with a cloudflare protection) url

cfScrape.setUrl("an url");

Set api url

By default, the url is http://127.0.0.1:8888

cfScrape.setApiHost("an url");

Scraping

There are two ways to scrape a website : asynchrone and synchrone ways.

Async

Return website contents

cfScrape.getAsyncHtml(new Callback<String>() {
    public void call(String html) {
        System.out.println(html);
    }
});

Example

Return api response

cfScrape.getAsyncResponse(new Callback<ApiResponse>() {
    @Override
    public void call(ApiResponse v) {
        String html = v.getHtml();
        String token = v.getToken();
        String url = v.getUrl();
    }
});

Example

Sync

Return website contents

String html = cfScrape.getSyncHtml();

Example

Return api response

ApiDirectResponse response = cfScrape.getSyncResponse();
String html = response.getHtml();
String url = response.getUrl();

Example

These examples can be viewed in the examples package