USCDataScience / sparkler

Spark-Crawler: Apache Nutch-like crawler that runs on Apache Spark.
http://irds.usc.edu/sparkler/
Apache License 2.0
411 stars 143 forks source link

Dashboard setup in non-docker environment #125

Closed C0mmander198 closed 6 years ago

C0mmander198 commented 7 years ago

I followed this wiki article that describes how to setup the dashboard. https://github.com/USCDataScience/sparkler/wiki/Sparkler-Dashboard-Setup

I was able to successfully clone the project, create that app jar and run inject and crawl statements against my local Solr instance through spark-submit. So far so good. But I don't get the dashboard working.

I don't have a "sparkler-dashboard" subfolder in my sparkler-ui folder after running the mvn clean install statement (as described in the above wiki article).

There is only a .war file in the target folder. I tried copying the war file to the $SOLR_HOME/server/solr-webapp/sparkler-dashboard folder (and the jetty context file to the $SOLR_HOME/server/contexts) folder. But this doesn't work.

Which folders do I have to copy to the solr-webapp folder to make this work?

sujen1412 commented 7 years ago

@C0mmander198, just to confirm did you also run git submodule init git submodule update before doing a mvn clean install ?

C0mmander198 commented 7 years ago

Yes, I did that. I followed the exact steps described in the above mentioned wiki link

thammegowda commented 6 years ago

Hey @C0mmander198

There has been some changes recently which were not updated on the wiki page. Please refer to the updated instructions on https://github.com/USCDataScience/sparkler/wiki/Sparkler-Dashboard-Setup

The change is in this step

cp -r ${SPARKLER_GIT_SOURCE_PATH}/sparkler-ui/target/sparkler-ui-*.war ${SOLR_DIRECTORY_PATH}/server/solr-webapp/sparkler 

Let us know if are still stuck here, we are happy to help! Also, you may refer to Dockerfile for additional details: https://github.com/USCDataScience/sparkler/blob/master/sparkler-deployment/docker/Dockerfile#L68

@sujen1412 If you are curious why this issue arouse https://github.com/USCDataScience/sparkler/commit/c6dcde7153f85b58f40224f82916d4b2b1cd1786