Traceback (most recent call last):
File "/opt/spark-2.2.1-master/bin/enclosures_pyspark_model.py", line 125, in
build_and_train_model()
File "/opt/spark-2.2.1-master/bin/enclosures_pyspark_model.py", line 118, in build_and_train_model
pmmlBuilder = PMMLBuilder(sc, train_df, pl) \
File "/Users/avnag/Library/Python/2.7/lib/python/site-packages/pyspark2pmml/init.py", line 15, in init
javaPmmlBuilder = javaPmmlBuilderClass(javaSchema, javaPipelineModel)
File "/opt/spark-2.2.1-master/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1401, in call
File "/opt/spark-2.2.1-master/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/spark-2.2.1-master/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 323, in get_return_value
py4j.protocol.Py4JError: An error occurred while calling None.org.jpmml.sparkml.PMMLBuilder. Trace:
py4j.Py4JException: Constructor org.jpmml.sparkml.PMMLBuilder([class org.apache.spark.sql.types.StructType, class org.apache.spark.ml.Pipeline]) does not exist
at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:179)
at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:196)
at py4j.Gateway.invoke(Gateway.java:235)
at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)
=============================================
Is there anything else I need to do?
Why is it not able to find the constructor ? I guess I am using right version
Hi,
My spark version is spark-2.2.1 I am running pyspark2pmml using the following command
./spark-submit --jars /opt/spark-2.2.1-master/bin/jpmml-sparkml-executable-1.3.14.jar enclosures_pyspark_model.py
My code looks like this :
While running it is throwing exception
Traceback (most recent call last): File "/opt/spark-2.2.1-master/bin/enclosures_pyspark_model.py", line 125, in
build_and_train_model()
File "/opt/spark-2.2.1-master/bin/enclosures_pyspark_model.py", line 118, in build_and_train_model
pmmlBuilder = PMMLBuilder(sc, train_df, pl) \
File "/Users/avnag/Library/Python/2.7/lib/python/site-packages/pyspark2pmml/init.py", line 15, in init
javaPmmlBuilder = javaPmmlBuilderClass(javaSchema, javaPipelineModel)
File "/opt/spark-2.2.1-master/python/lib/py4j-0.10.4-src.zip/py4j/java_gateway.py", line 1401, in call
File "/opt/spark-2.2.1-master/python/lib/pyspark.zip/pyspark/sql/utils.py", line 63, in deco
File "/opt/spark-2.2.1-master/python/lib/py4j-0.10.4-src.zip/py4j/protocol.py", line 323, in get_return_value
py4j.protocol.Py4JError: An error occurred while calling None.org.jpmml.sparkml.PMMLBuilder. Trace:
py4j.Py4JException: Constructor org.jpmml.sparkml.PMMLBuilder([class org.apache.spark.sql.types.StructType, class org.apache.spark.ml.Pipeline]) does not exist
at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:179)
at py4j.reflection.ReflectionEngine.getConstructor(ReflectionEngine.java:196)
at py4j.Gateway.invoke(Gateway.java:235)
at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80)
at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69)
at py4j.GatewayConnection.run(GatewayConnection.java:214)
at java.lang.Thread.run(Thread.java:748)
=============================================
Is there anything else I need to do? Why is it not able to find the constructor ? I guess I am using right version