USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

http://irds.usc.edu/sparkler/

Apache License 2.0

411 stars 143 forks source link

Sparkler 127 : Simple plugin framework #142

Closed thammegowda closed 6 years ago

thammegowda commented 6 years ago

What changes were proposed in this pull request?

remove OSGI and Felix
add pf4j
simplify dependencies of sparkler-api
fix the dependency hell that blocked #110
plugins are loaded and enforced strictly based on config. reports conflicts when found (for ex: multiple plugins enabled when only one expected)
build is restructured to make it easy for release.
- everything required to run sparkler will be contained in build directory (including scripts, plugins, configs)

Is this related to an already existing issue on sparkler?

127

Will it close an existing issue?
Closes #127

How was this patch tested?

by running all the test cases