USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
410 stars 143 forks source link

Move Sparkler to sbt build #184

Open karanjeets opened 4 years ago

karanjeets commented 4 years ago

Issue Description

Please describe our issue, along with:

How to reproduce it

If you are describing a bug, please describe here how to reproduce it. N/A; Maven scala build plugins are not scalable

An external links for reference

Wondering what others are thinking? - https://www.jetbrains.com/research/devecosystem-2018/scala/

thammegowda commented 3 years ago

@karanjeets good to see you again buddy. whats going on here with this issue?

karanjeets commented 3 years ago

hey @thammegowda, I'm done with most of the work including, move pom to sbt build files, package app & plugins, exclude conflicting transitive dependencies, and resolve build issues. Left with the following items before I make a PR:

This would take another 2 weeks, probably

thammegowda commented 3 years ago

wow that's exciting. Thanks.