larsga / Duke

Duke is a fast and flexible deduplication engine written in Java
Apache License 2.0
613 stars 194 forks source link

java.lang.ClassNotFoundException for Postgres Driver #250

Closed bino013 closed 6 years ago

bino013 commented 6 years ago

Hi,

I'm encountering java.lang.ClassNotFoundException when I'm trying to use postgres as my jdbc driver.

StackTrace:

    at no.priv.garshol.duke.utils.ObjectUtils.instantiate(ObjectUtils.java:145)
    at no.priv.garshol.duke.utils.JDBCUtils.open(JDBCUtils.java:47)
    at no.priv.garshol.duke.datasources.JDBCDataSource.getRecords(JDBCDataSource.java:82)
    at no.priv.garshol.duke.Processor.deduplicate(Processor.java:198)
    at no.priv.garshol.duke.Processor.deduplicate(Processor.java:174)
    at DukeRunner.main(DukeRunner.java:27)
Caused by: java.lang.ClassNotFoundException: org.postgresql.Driver 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at no.priv.garshol.duke.utils.ObjectUtils.instantiate(ObjectUtils.java:142)
    ... 5 more

POM:


<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.core.samples</groupId>
    <artifactId>sample-duke-project</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>

        <dependency>
            <groupId>no.priv.garshol.duke</groupId>
            <artifactId>duke</artifactId>
            <version>1.2</version>
        </dependency>
        <dependency>
            <groupId>org.postgresql</groupId>
            <artifactId>postgresql</artifactId>
            <version>9.4.1212</version>
        </dependency>
    </dependencies>

</project>```
larsga commented 6 years ago

You seem to have set everything up correctly, so I guess it depends what your actual classpath is like during deployment. For some reason the Postgres driver isn't there when Duke tries to load it. I'm not sure where you're running this, but you may have to extend the classpath to explicitly include the Postgres driver.

bino013 commented 6 years ago

/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/bin/java "-javaagent:/Applications/IntelliJ IDEA.app/Contents/lib/idea_rt.jar=63947:/Applications/IntelliJ IDEA.app/Contents/bin" -Dfile.encoding=UTF-8 -classpath "/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/charsets.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/deploy.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/cldrdata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/dnsns.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/jfxrt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/localedata.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/nashorn.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/sunec.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/sunjce_provider.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/sunpkcs11.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/ext/zipfs.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/javaws.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/jce.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/jfr.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/jfxswt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/jsse.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/management-agent.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/plugin.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/resources.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/jre/lib/rt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/ant-javafx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/dt.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/javafx-mx.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/jconsole.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/packager.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/sa-jdi.jar:/Library/Java/JavaVirtualMachines/jdk1.8.0_45.jdk/Contents/Home/lib/tools.jar:/Users/arcaleon/Documents/Work Codes/sample-duke-project/target/classes:/Users/arcaleon/.m2/repository/no/priv/garshol/duke/duke/1.2/duke-1.2.jar:/Users/arcaleon/.m2/repository/org/apache/lucene/lucene-core/4.0.0/lucene-core-4.0.0.jar:/Users/arcaleon/.m2/repository/org/apache/lucene/lucene-analyzers-common/4.0.0/lucene-analyzers-common-4.0.0.jar:/Users/arcaleon/.m2/repository/org/apache/lucene/lucene-spatial/4.0.0/lucene-spatial-4.0.0.jar:/Users/arcaleon/.m2/repository/com/spatial4j/spatial4j/0.3/spatial4j-0.3.jar:/Users/arcaleon/.m2/repository/org/apache/lucene/lucene-queries/4.0.0/lucene-queries-4.0.0.jar:/Users/arcaleon/.m2/repository/org/mapdb/mapdb/0.9.9/mapdb-0.9.9.jar:/Users/arcaleon/.m2/repository/org/postgresql/postgresql/9.4.1212/postgresql-9.4.1212.jar" DukeRunner

Base on the classpath that intellij uses when running the program and it seems that postresql jar is included. I'm just wondering if you have tried using the duke in maven project which uses postgres as its jdbc?

larsga commented 6 years ago

That command should work, assuming /Users/arcaleon/.m2/repository/org/postgresql/postgresql/9.4.1212/postgresql-9.4.1212.jaris actually there.

I do the same thing with MySQL all the time, and it works just fine. So the problem isn't Duke.

bino013 commented 6 years ago

I think Duke has a problem with Postgres.

I tried printing out if Class.forName("org.postgresql.Driver"); on my main class, and it did not throw any ClassNotFoundException. See that Class.forName() method is the same method that causes the ClassNotFoundException.

pass: class org.postgresql.Driver
Exception in thread "main" java.lang.RuntimeException: java.lang.ClassNotFoundException: org.postgresql.Driver 
    at no.priv.garshol.duke.utils.ObjectUtils.instantiate(ObjectUtils.java:145)
    at no.priv.garshol.duke.utils.JDBCUtils.open(JDBCUtils.java:47)
    at no.priv.garshol.duke.datasources.JDBCDataSource.getRecords(JDBCDataSource.java:82)
    at no.priv.garshol.duke.Processor.deduplicate(Processor.java:198)
    at no.priv.garshol.duke.Processor.deduplicate(Processor.java:174)
    at DukeRunner.main(DukeRunner.java:27)
Caused by: java.lang.ClassNotFoundException: org.postgresql.Driver 
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:264)
    at no.priv.garshol.duke.utils.ObjectUtils.instantiate(ObjectUtils.java:142)
    ... 5 more
public class DukeRunner {
    static {
        try {
            Class<?> aClass = Class.forName("org.postgresql.Driver");
            System.out.println("pass: " + aClass);
        } catch (ClassNotFoundException e) {
            e.printStackTrace();
        }
    }

    private static final String DUKE_CONFIG_XML_FILENAME = "classpath:DukeConfig.xml";

    public static void main(String[] args) throws IOException, SAXException {
        Configuration config = ConfigLoader.load(DUKE_CONFIG_XML_FILENAME);
        Processor processor = new Processor(config);
        processor.addMatchListener(new PrintMatchListener(true, true,
                true, false, config.getProperties(), true));
        processor.deduplicate();
        processor.close();
    }

}
larsga commented 6 years ago

Why are you closing the ticket? Did you figure out the solution? What was it?

bino013 commented 6 years ago

Hi,

My mistake. Upon checking the value of driver-class on the DukeConfig, I have unconsciously put an excess space right after org.postgresql.Driver.

Thanks for the help anyways.

larsga commented 6 years ago

Thanks for explaining. Duke now trims spaces in class names so that we can avoid a repeat of this.