search

intel-analytics / analytics-zoo

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
https://analytics-zoo.readthedocs.io/
Apache License 2.0
16 stars 3 forks source link

Integration test ( image classification example) failed with BigDL 0.12.0 #548

Closed pinggao187 closed 3 years ago

pinggao187 commented 3 years ago

begin image classification Picked up _JAVA_OPTIONS: -XX:MaxPermSize=3G -Xmx50G Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=3G; support was removed in 8.0 Picked up _JAVA_OPTIONS: -XX:MaxPermSize=3G -Xmx50G Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=3G; support was removed in 8.0 20/11/27 09:42:43 INFO utils.Engine$: Auto detect executor number and executor cores number 20/11/27 09:42:43 INFO utils.Engine$: Executor number is 4 and executor cores number is 8 20/11/27 09:42:44 INFO utils.ThreadPool$: Set mkl threads to 1 on thread 1 20/11/27 09:42:44 INFO utils.Engine$: Find existing spark context. Checking the spark conf... Exception in thread "main" org.apache.commons.lang.SerializationException: java.lang.ClassNotFoundException: com.intel.analytics.bigdl.tensor.QuantizedTensor at org.apache.commons.lang.SerializationUtils.deserialize(SerializationUtils.java:166) at org.apache.commons.lang.SerializationUtils.deserialize(SerializationUtils.java:193) at org.apache.commons.lang.SerializationUtils.clone(SerializationUtils.java:81) at com.intel.analytics.bigdl.utils.Util$.cloneParameters(Util.scala:301) at com.intel.analytics.bigdl.models.utils.ModelBroadcastImp.broadcast(ModelBroadcast.scala:138) at com.intel.analytics.bigdl.models.utils.ModelBroadcastImp.broadcast(ModelBroadcast.scala:251) at com.intel.analytics.bigdl.models.utils.ModelBroadcastImp.broadcast(ModelBroadcast.scala:95) at com.intel.analytics.bigdl.optim.Predictor$.predictImage(Predictor.scala:130) at com.intel.analytics.bigdl.optim.Predictor.predictImage(Predictor.scala:259) at com.intel.analytics.bigdl.nn.abstractnn.AbstractModule.predictImage(AbstractModule.scala:700) at com.intel.analytics.zoo.models.image.common.ImageModel.predictImageSet(ImageModel.scala:71) at com.intel.analytics.zoo.examples.imageclassification.Predict$$anonfun$main$1.apply(Predict.scala:65) at com.intel.analytics.zoo.examples.imageclassification.Predict$$anonfun$main$1.apply(Predict.scala:61) at scala.Option.foreach(Option.scala:257) at com.intel.analytics.zoo.examples.imageclassification.Predict$.main(Predict.scala:61) at com.intel.analytics.zoo.examples.imageclassification.Predict.main(Predict.scala) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Delegatin

jenkins link: http://10.239.47.210:18888/view/ZOO-NB/job/ZOO-NB-Scala-ExampleTests/1139/

Le-Zheng commented 3 years ago

This is strange. Running on local with BigDL 0.12.0 and Spark 2.4.3 is successful. Running on yarn cluster throws the error above.

Le-Zheng commented 3 years ago

This issue has been fixed in BigDL 0.12.1