Finish Juju charm - Githubissues

USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.

http://irds.usc.edu/sparkler/

Apache License 2.0

410 stars 143 forks source link

Open buggtb opened 7 years ago

buggtb commented 7 years ago

Some high level remaining tasks:

[x] Add solr relation
[x] Pick up spark details from relation
[x] Pick up solr details from relation
[x] Finish write to configuration
[x] Compile and add resource
[x] Add actions for remote execution of crawler
[x] Add Kafka relation
[ ] Return id in action output
[ ] Store last ingest id in kv so crawl can pick it up without user looking it up
[ ] Sort out solr cloud
[ ] Finish Docs
[ ] Add charm push to CI for beta branch

buggtb commented 7 years ago

Okay very rough, but deploys the charm with my tweaks: https://jujucharms.com/u/spiculecharms/sparkler/0 bundle and fine tuning to appear over the next week or so.

buggtb commented 7 years ago

Added a bundle today that helps deploy the services. This one is just sparkler and solr, spark support and solr cloud support will come soon-ish

buggtb commented 7 years ago