search

linkedin / photon-ml

A scalable machine learning library on Apache Spark
Other
790 stars 185 forks source link

GameTrainingDriver will not run #468

Open PChambers314 opened 4 years ago

PChambers314 commented 4 years ago

When I use spark-submit class com.linkedin.photon.ml.cli.game.GameTrainingDriver \, it will not run. If I change it to spark-submit \ --class com.linkedin.photon.ml.cli.game.training.GameTrainingDriver \ it runs with an error message: Exception in thread "main" java.lang.AbstractMethodError: com.linkedin.photon.ml.cli.game.training.GameTrainingDriver$.org$apache$spark$ml$param$Params$setter$paramMap_$eq(Lorg/apache/spark/ml/param/ParamMap;)V

Is there any way to fix this so that the GAME model with run?

junshi15 commented 4 years ago

1) you need change to com.linkedin.photon.ml.cli.game.training.GameTrainingDriver, the tutorial needs to be refreshed. 2) can you post the command you used to launch the training? did you use photon-all_2.11/libs/photon-all_2.11-1.0.0.jar? (I compiled it with scala 2.11)

junshi15 commented 4 years ago

I saw the same issue when I launched the job with spark 2.4.5. I suspect you were using a similar version.

The current photon-ml only compiles with spark 2.3 (it complains when compiling with spark 2.4 due to spark backward incompatibility). When you launch the job, make sure you launch with spark 2.3.x.