search

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
410 stars 143 forks source link

WIP: ISSUE-215 Elasticsearch for Sparkler - Maven Profiles #217

Closed KilometersFan closed 2 years ago

KilometersFan commented 3 years ago

What changes were proposed in this pull request?

Changes to sparkler-app pom.xml:

Is this related to an already existing issue on sparkler?
Related to #215

Will it close an existing issue?
Closes #215.

How was this patch tested?

This was tested by building the project running mvn clean compile package -P solr and mvn clean compile package -P elasticsearch in the sparkler-app directory and running test crawls

lewismc commented 3 years ago

@KilometersFan please update this PR. Thank you

lewismc commented 3 years ago

Let's hold off on merging this PR. We are still working on design of storage modules and need to make sure that work is solid before we progress further with the Maven build.

buggtb commented 2 years ago

Closing down as we no longer use maven.