search

databricks / spark-deep-learning

Deep Learning Pipelines for Apache Spark
https://databricks.github.io/spark-deep-learning
Apache License 2.0
1.99k stars 494 forks source link

No module named 'PIL' when I need to import sparkdl #224

Closed qxy180009 closed 4 years ago

qxy180009 commented 4 years ago

I want to make a deep learning pipeline. My other libraries are: spark-deep-learning:1.2.0-spark2.3-s_2.11 h5py, keras==2.2.4, tensorflow==1.15.0

When I use from sparkdl import DeepImageFeaturizer, I got the error of ModuleNotFoundError: No module named 'PIL' `--------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last)

in ----> 1 from sparkdl import DeepImageFeaturizer /local_disk0/spark-1213fec9-1a0e-4f1e-a956-199463db5abe/userFiles-dba4d0d3-813d-48cc-94e9-391ddf07f007/addedFile6355333370221159413spark_deep_learning_1_2_0_spark2_3_s_2_11-854c9.jar/sparkdl/__init__.py in 14 # 15 from .graph.input import TFInputGraph ---> 16 from .transformers.keras_image import KerasImageFileTransformer 17 from .transformers.named_image import DeepImagePredictor, DeepImageFeaturizer 18 from .transformers.keras_tensor import KerasTransformer /local_disk0/spark-1213fec9-1a0e-4f1e-a956-199463db5abe/userFiles-dba4d0d3-813d-48cc-94e9-391ddf07f007/addedFile6355333370221159413spark_deep_learning_1_2_0_spark2_3_s_2_11-854c9.jar/sparkdl/transformers/keras_image.py in 15 16 from pyspark.ml import Transformer ---> 17 from sparkdl.param import CanLoadImage, HasInputCol, HasKerasModel, HasOutputCol, HasOutputMode, \ 18 keyword_only 19 from sparkdl.transformers.keras_utils import KSessionWrap /local_disk0/spark-1213fec9-1a0e-4f1e-a956-199463db5abe/userFiles-dba4d0d3-813d-48cc-94e9-391ddf07f007/addedFile6355333370221159413spark_deep_learning_1_2_0_spark2_3_s_2_11-854c9.jar/sparkdl/param/__init__.py in 21 HasKerasModel, HasKerasLoss, HasKerasOptimizer) 22 from sparkdl.param.converters import SparkDLTypeConverters ---> 23 from sparkdl.param.image_params import ( 24 CanLoadImage, HasOutputMode, OUTPUT_MODES) /local_disk0/spark-1213fec9-1a0e-4f1e-a956-199463db5abe/userFiles-dba4d0d3-813d-48cc-94e9-391ddf07f007/addedFile6355333370221159413spark_deep_learning_1_2_0_spark2_3_s_2_11-854c9.jar/sparkdl/param/image_params.py in 23 from pyspark.ml.param import Param, Params 24 from pyspark.sql.functions import udf ---> 25 from sparkdl.image.imageIO import _reverseChannels, imageArrayToStruct 26 from sparkdl.param.converters import SparkDLTypeConverters 27 /local_disk0/spark-1213fec9-1a0e-4f1e-a956-199463db5abe/userFiles-dba4d0d3-813d-48cc-94e9-391ddf07f007/addedFile6355333370221159413spark_deep_learning_1_2_0_spark2_3_s_2_11-854c9.jar/sparkdl/image/imageIO.py in 19 # 3rd party 20 import numpy as np ---> 21 from PIL import Image 22 23 # pyspark ModuleNotFoundError: No module named 'PIL'` The problem is in the `imageIO.py`, where there is a line `from PIL import Image` causing the error., I have tried many library versions.