actionml / universal-recommender

Highly configurable recommender based on PredictionIO and Mahout's Correlated Cross-Occurrence algorithm
http://actionml.com/universal-recommender
Apache License 2.0
669 stars 172 forks source link

Error on training #71

Closed mick912 closed 4 years ago

mick912 commented 4 years ago

Hi, all. I have installed Harness git clone https://github.com/actionml/harness-docker-compose.git && cd harness-docker-compose && cp .env.sample .env && docker-compose up -d --build And when I try to train I give follow errors

ERROR SparkContextSupport$ - Spark context failed for job JobDescription(321e0ee5-f824-4b72-8589-fcc4e1e2270f,queued,Spark job,Some(Sun Dec 15 08:43:38 GMT 2019),None)
org.apache.spark.SparkException: A master URL must be set in your configuration
at org.apache.spark.SparkContext.(SparkContext.scala:367)
at com.actionml.core.spark.SparkContextSupport$$anonfun$1.apply(SparkContextSupport.scala:133)
at com.actionml.core.spark.SparkContextSupport$$anonfun$1.apply(SparkContextSupport.scala:120)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.liftedTree1$1(Future.scala:24)
at scala.concurrent.impl.Future$PromiseCompletingRunnable.run(Future.scala:24)
at scala.concurrent.impl.ExecutionContextImpl$AdaptedForkJoinTask.exec(ExecutionContextImpl.scala:121)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
mick912 commented 4 years ago

The ERROR is solved by adding "master": "local" according to the documentation:

https://actionml.com/docs/h_ur_config#spark-parameters-codesparkconfcode

{
   "engineId": "ecom_ur",
   "engineFactory": "com.actionml.engines.ur.UREngine",
   "sparkConf": {
       "master": "local",
       "spark.serializer": "org.apache.spark.serializer.KryoSerializer",
       "spark.kryo.registrator": "org.apache.mahout.sparkbindings.io.MahoutKryoRegistrator",
       "spark.kryo.referenceTracking": "false",
       "spark.kryoserializer.buffer": "300m",
       "spark.executor.memory": "20g",
       "spark.driver.memory": "10g",
       "spark.es.index.auto.create": "true",
       "spark.es.nodes": "localhost",
       "spark.es.nodes.wan.only": "true"
   },
   "algorithm":{
       "indicators": [ 
           {
               "name": "buy"
           }
       ]
   }
}

https://github.com/actionml/harness/issues/181#issuecomment-566489961