yasserg / crawler4j

Open Source Web Crawler for Java
Apache License 2.0
4.56k stars 1.93k forks source link

Fix Maven dependencies in build.gradle #468

Open ytxmobile98 opened 2 years ago

ytxmobile98 commented 2 years ago

I am trying to learn from the crawler examples you have provided here to write my own crawling program.

But when I opened the repo in Visual Studio Code, it complains "Unresolved dependency: com.palantir.docker.compose:docker-compose-rule-junit4:0.33.0".

I found this dependency specification in crawler4j-examples/crawler4j-examples-postgres/build.gradle, with the following lines causing the problem:

repositories {
    mavenCentral()
    // Needed for 'com.palantir.docker.compose'
    maven { url "https://palantir.bintray.com/releases" }
}

This URL no longer works. If I access it directly from my browser, it shows "502 Bad Gateway".

I checked on MVN Repository that the newest working version is 1.7.0. (.jar file download link)

So after checking the .jar file URL I changed these lines to:

repositories {
    mavenCentral()
    // Needed for 'com.palantir.docker.compose'
    maven { url "https://repo1.maven.org/maven2/" }
}

Now I can successfully run these examples.

Please fix the broken dependencies in the repo.