search

perwendel / spark

A simple expressive web framework for java. Spark has a kotlin DSL https://github.com/perwendel/spark-kotlin
Apache License 2.0
9.65k stars 1.56k forks source link

Job aborted due to stage failure #1289

Open NamrataRade opened 10 months ago

NamrataRade commented 10 months ago

Error-org.apache.spark.SparkException: Job aborted due to stage failure: Task 1 in stage 150.0 failed 4 times, most recent failure: Lost task 1.3 in stage 150.0 (TID 426) (172.30.123.114 executor 6): java.lang.NullPointerException sample code: from pyspark.ml.clustering import KMeans kmeans = KMeans(k=4, seed=1) model = kmeans.fit(df_pca.select("features")) df_cluster_out = model.transform(df_pca)

df_pca: unique id features 1 (2,[],[]) 2 (2,[],[]) 3 [0.4,0.8] I have performed scaling+pca before applying to kmeans. Please provide some guidance.

oeph commented 2 weeks ago

This is the wrong spark project - please refer to https://github.com/apache/spark