Elasticsearch for Sparkler - Factory Design Pattern

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

http://irds.usc.edu/sparkler/

Apache License 2.0

410 stars 143 forks source link

Elasticsearch for Sparkler - Factory Design Pattern #218

Closed slhsxcmy closed 3 years ago

slhsxcmy commented 3 years ago

We are implementing a Factory Pattern to extract storage components (Solr and Elasticsearch) from Sparkler implementation. We plan to make sure Solr can still run after the extraction first, then add the connector with Elasticsearch. Currently this is WIP. We'll post further updates soon.

lewismc commented 3 years ago

Thanks