search

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
410 stars 143 forks source link

Not an issue #176

Closed chaitra-rs closed 4 years ago

chaitra-rs commented 4 years ago

Is it possible to run sparkler without docker? I already have a python set up, I want to integrate sparkler

thammegowda commented 4 years ago

Yes. it's possible (we have done that before creating docker image) but requires more effort.

See this dockerfile https://github.com/USCDataScience/sparkler/blob/5c2201310623b70e6bf024e51e521eb4bffc4723/sparkler-deployment/docker/Dockerfile#L38-L55