when I try to fit the model it is giving me an error saying that TypeError: Cannot convert type <class 'pyspark.ml.linalg.DenseVector'> into Vector
from pyspark.mllib.evaluation import MulticlassMetrics
fitted_pipeline = pipeline.fit(final_train) # Fit model to data
prediction = fitted_pipeline.transform(final_train) # Evaluate on train data.
# prediction = fitted_pipeline.transform(test_df) # <-- The same code evaluates test data.
pnl = prediction.select("index_category", "prediction")
pnl.show(100)
Hi Team,
when I try to fit the model it is giving me an error saying that TypeError: Cannot convert type <class 'pyspark.ml.linalg.DenseVector'> into Vector
final_train.printSchema() root |-- category: string (nullable = true) |-- features: vector (nullable = true)
Then it is giving me an error saying that. log.txt
sample df.
The result of indexing and scaling. Each transformation adds new columns to the data frame: +--------+--------------------+--------------+--------------------+ |category| features|index_category| scaled_features| +--------+--------------------+--------------+--------------------+ | 3|[238.0,238.0,238....| 1.0|[0.43949125844258...| | 1|[29.0,25.0,140.0,...| 0.0|[-4.5706653137731...| | 1|[255.0,255.0,255....| 0.0|[0.84701595570415...| | 10|[251.0,251.0,251....| 7.0|[0.75112779164260...| | 1|[192.0,195.0,196....| 0.0|[-0.6632226282651...| | 3|[218.0,218.0,217....| 1.0|[-0.0399495618651...| | 3|[255.0,255.0,255....| 1.0|[0.84701595570415...| | 4|[255.0,255.0,255....| 8.0|[0.84701595570415...| | 1|[221.0,221.0,221....| 0.0|[0.03196656118101...| | 10|[225.0,225.0,225....| 7.0|[0.12785472524256...| +--------+--------------------+--------------+--------------------+