intel-analytics / analytics-zoo

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray
https://analytics-zoo.readthedocs.io/
Apache License 2.0
16 stars 3 forks source link

submit-examples-on-k8s: vnni BigDL predict - OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /root/opencv/opencv.native/modules/imgproc/src/imgwarp.cpp, line 3496 #262

Open zzti-bsj opened 3 years ago

zzti-bsj commented 3 years ago

vnni BigDL predict - OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /root/opencv/opencv.native/modules/imgproc/src/imgwarp.cpp, line 3496

the example link: https://github.com/intel-analytics/analytics-zoo/blob/master/docker/hyperzoo/submit-examples-on-k8s.md

command

/opt/spark/bin/spark-submit
   --master k8s://https://127.0.0.1:8443
   --deploy-mode cluster
   --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark
   --name analytics-zoo
   --conf spark.kubernetes.container.image=${RUNTIME_K8s_SPARK_IMAGE}
   --conf spark.executor.instances=1
   --conf spark.kubernetes.driver.volumes.persistentVolumeClaim.nfsvolumeclaim.options.claimName=nfsvolumeclaim
   --conf spark.kubernetes.driver.volumes.persistentVolumeClaim.nfsvolumeclaim.mount.path=/zoo
   --conf spark.kubernetes.executor.volumes.persistentVolumeClaim.nfsvolumeclaim.options.claimName=nfsvolumeclaim
   --conf spark.kubernetes.executor.volumes.persistentVolumeClaim.nfsvolumeclaim.mount.path=/zoo
   --conf spark.kubernetes.driver.label.az=true
   --conf spark.kubernetes.executor.label.az=true
   --conf spark.kubernetes.node.selector.spark=true
   --executor-cores 16
   --executor-memory 10g
   --total-executor-cores 64
   --driver-cores 4
   --driver-memory 10g
   --properties-file /opt/analytics-zoo-0.11.0-SNAPSHOT/conf/spark-analytics-zoo.conf
   --py-files /opt/analytics-zoo-0.11.0-SNAPSHOT/lib/analytics-zoo-bigdl_0.12.2-spark_2.4.3-0.11.0-SNAPSHOT-python-api.zip
   --conf spark.driver.extraJavaOptions=-Dderby.stream.error.file=/tmp
   --conf spark.sql.catalogImplementation='in-memory'
   --conf spark.driver.extraClassPath=/opt/analytics-zoo-0.11.0-SNAPSHOT/lib/analytics-zoo-bigdl_0.12.2-spark_2.4.3-0.11.0-SNAPSHOT-jar-with-dependencies.jar
   --conf spark.executor.extraClassPath=/opt/analytics-zoo-0.11.0-SNAPSHOT/lib/analytics-zoo-bigdl_0.12.2-spark_2.4.3-0.11.0-SNAPSHOT-jar-with-dependencies.jar
   --class com.intel.analytics.zoo.examples.vnni.bigdl.Predict   /opt/analytics-zoo-0.11.0-SNAPSHOT/lib/analytics-zoo-bigdl_0.12.2-spark_2.4.3-0.11.0-SNAPSHOT-python-api.zip
   -f /zoo/data2/test/ 
  -m /zoo/data2/analytics-zoo-models/analytics-zoo_resnet-50-int8_imagenet_0.5.0.model

Exception info

2021-05-19 15:51:04 INFO  ImageModel$:142 - Loading an int8 convertible model. Quantize to an int8 model for better performance
2021-05-19 15:51:04 WARN  DistriOptimizer$:160 - Input formats NCHW by default, Please set explicitly if needed
2021-05-19 15:51:04 WARN  DistriOptimizer$:165 - Output formats NC by default, Please set explicitly if needed
2021-05-19 15:51:07 INFO  Predict$:57 - Start inference on images under /zoo/data2/test/...
OpenCV Error: Assertion failed (ssize.width > 0 && ssize.height > 0) in resize, file /root/opencv/opencv.native/modules/imgproc/src/imgwarp.cpp, line 3496
Exception in thread "main" CvException [org.opencv.core.CvException: cv::Exception: /root/opencv/opencv.native/modules/imgproc/src/imgwarp.cpp:3496: error: (-215) ssize.width > 0 && ssize.height > 0 in function resize
]
        at org.opencv.imgproc.Imgproc.resize_0(Native Method)
        at org.opencv.imgproc.Imgproc.resize(Imgproc.java:2761)
        at com.intel.analytics.bigdl.transform.vision.image.augmentation.Resize$.transform(Resize.scala:69)
        at com.intel.analytics.bigdl.transform.vision.image.augmentation.RandomResize.transformMat(RandomResize.scala:46)
        at com.intel.analytics.zoo.feature.image.ImageRandomResize.transformMat(ImageRandomResize.scala:37)
        at com.intel.analytics.bigdl.transform.vision.image.FeatureTransformer.transform(FeatureTransformer.scala:65)
        at com.intel.analytics.zoo.feature.image.ChainedImageProcessing.transform(ImageProcessing.scala:41)
        at com.intel.analytics.zoo.feature.image.ChainedImageProcessing.transform(ImageProcessing.scala:41)
        at com.intel.analytics.zoo.feature.image.ChainedImageProcessing.transform(ImageProcessing.scala:41)
        at com.intel.analytics.zoo.feature.image.ChainedImageProcessing.transform(ImageProcessing.scala:41)
        at com.intel.analytics.bigdl.transform.vision.image.FeatureTransformer$$anonfun$apply$1.apply(FeatureTransformer.scala:91)
        at com.intel.analytics.bigdl.transform.vision.image.FeatureTransformer$$anonfun$apply$1.apply(FeatureTransformer.scala:91)
        at scala.collection.Iterator$$anon$11.next(Iterator.scala:410)
        at scala.collection.Iterator$class.foreach(Iterator.scala:891)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1334)
        at scala.collection.generic.Growable$class.$plus$plus$eq(Growable.scala:59)
        at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:104)
        at scala.collection.mutable.ArrayBuffer.$plus$plus$eq(ArrayBuffer.scala:48)
        at scala.collection.TraversableOnce$class.to(TraversableOnce.scala:310)
        at scala.collection.AbstractIterator.to(Iterator.scala:1334)
        at scala.collection.TraversableOnce$class.toBuffer(TraversableOnce.scala:302)
        at scala.collection.AbstractIterator.toBuffer(Iterator.scala:1334)
        at scala.collection.TraversableOnce$class.toArray(TraversableOnce.scala:289)
        at scala.collection.AbstractIterator.toArray(Iterator.scala:1334)
        at com.intel.analytics.zoo.feature.image.LocalImageSet.transform(ImageSet.scala:101)
        at com.intel.analytics.zoo.feature.image.ImageSet.$minus$greater(ImageSet.scala:60)
        at com.intel.analytics.zoo.models.image.common.ImageModel.predictImageSet(ImageModel.scala:66)
        at com.intel.analytics.zoo.examples.vnni.bigdl.Predict$$anonfun$main$1.apply(Predict.scala:58)
        at com.intel.analytics.zoo.examples.vnni.bigdl.Predict$$anonfun$main$1.apply(Predict.scala:52)
        at scala.Option.map(Option.scala:146)
        at com.intel.analytics.zoo.examples.vnni.bigdl.Predict$.main(Predict.scala:52)
        at com.intel.analytics.zoo.examples.vnni.bigdl.Predict.main(Predict.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Le-Zheng commented 3 years ago

This example can run successfully on my local env. Continue to investigate the error in hyper-zoo image.