USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
411 stars 143 forks source link

Removing sparkler-app-0.2.0-SNAPSHOT.jar #159

Closed misterpilou closed 6 years ago

misterpilou commented 6 years ago

First i've got an issue with the sparkler-app-0.2.0-SNAPSHOT.jar so i deleted it, then i couldn't rebuild the project, Is there any particular reason to keep the sparkler-app-0.2.0-SNAPSHOT.jar. Well fyi, removing the:

<dependency>
    <groupId>edu.usc.irds.sparkler</groupId>
    <artifactId>sparkler-api</artifactId>
    <version>0.2.0-SNAPSHOT</version>
</dependency>

from sparkler-app/pom.xml fix the issue, I'll do it myself if you confirm that it is not useful anymore & when i'll finish my work

misterpilou commented 6 years ago

Is there an utility with the sparkler-app/resources/log4j.properties file? As it is already in the conf directory

thammegowda commented 6 years ago

Mr. Pilou @misterpilou

misterpilou commented 6 years ago

okay, and concerning sparkler-app/resources/log4j.properties?

thammegowda commented 6 years ago

oh do you mean sparkler-app/src/main/resources/log4j.properties? That one is set to DEBUG, see log4j.logger.edu.usc.irds=DEBUG, meant for use during development time logs (my IntelliJ IDEA picks up this during development)

Then the one in conf directory, has log4j.logger.edu.usc.irds=INFO. It is meant for the final production deployment, and thus goes into final package.

misterpilou commented 6 years ago

Yes i mean that, okay i got it

thammegowda commented 6 years ago

I am interested in hearing if these log4j.properties files caused any trouble

misterpilou commented 6 years ago

No it didn't made any, i tried to use sparkler-app/src/main/resources dir before knowing that it was actually the conf dir to use as a resources

thammegowda commented 6 years ago

I hope this is resolved. If not please reopen it