BinaryClassificationMetrics are not working

[fdegier]

When using the BinaryClassificationMetrics there is an error, possible related to conversion of data types. This only occurs when using Sparkmagic and Livy.

Input:

from pyspark.sql import SparkSession   
import pyspark
from pyspark.mllib.evaluation import BinaryClassificationMetrics

a = sc.parallelize([
    (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
    (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), 
    (0.0, 1.0), (0.0, 1.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
    (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), 
    (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0), (0.0, 0.0)
])  

metrics = BinaryClassificationMetrics(a)

Output:

'StructField' object has no attribute '_get_object_id' 

[hnikana]

Any solution ?