search

knockdata / spark-highcharts

Support Highcharts in Apache Zeppelin
Apache License 2.0
81 stars 14 forks source link

StreamingQuery cannot be applied to SeriesHolder #33

Open acn-hendrickacosta opened 5 years ago

acn-hendrickacosta commented 5 years ago

Hi,

I am new with spark and zeppelin, I am trying to create a plot a live chart using data being fetched from kafka stream. I tried to follow the samples I found on the issues but cannot resolve the issue. Can you kindly point out if I am doing something wrong. Thanks!

import java.sql.Timestamp
import org.apache.spark.sql.functions._
import org.apache.spark.sql.types._

val schema: StructType = StructType(Seq(StructField("ECG",StringType),
StructField("activity",StringType)
))

spark.conf.set("spark.sql.streaming.checkpointLocation", "checkpoint")

// Create DataSet representing the stream of input lines from kafka
val lines = spark.readStream.format("kafka").option("kafka.bootstrap.servers", "localhost:9092").option("subscribe", "ecg").option("startingOffsets", "earliest").load()

val data = lines.selectExpr("CAST(value AS STRING) as value", "CAST(timestamp AS TIMESTAMP) as timestamp").select( from_json($"value".cast(StringType), schema=schema).as("data"), $"timestamp").select("data.ECG","data.activity","timestamp").selectExpr("CAST(ECG AS FLOAT) as ECG", "activity","timestamp").as[(Float, String, Timestamp)].toDF("ECG","activity","timestamp")

import com.knockdata.spark.highcharts._
import com.knockdata.spark.highcharts.model._

val query = highcharts(
  data.seriesCol("timestamp")
    .series("y" -> "ECG","x" -> "timestamp")
    .orderBy(col("timestamp")), z, "append")

But it returns this error:

<console>:72: error: overloaded method value apply with alternatives:
  (seriesHolders: com.knockdata.spark.highcharts.SeriesHolder*)com.knockdata.spark.highcharts.model.Highcharts <and>
  (seriesHolder: com.knockdata.spark.highcharts.SeriesHolder,z: org.apache.zeppelin.spark.ZeppelinContext,outputMode: String)org.apache.spark.sql.streaming.StreamingQuery
 cannot be applied to (com.knockdata.spark.highcharts.SeriesHolder, org.apache.zeppelin.spark.SparkZeppelinContext, String)
       val query = highcharts(

The whole log:

import java.sql.Timestamp
import org.apache.spark.sql.functions._
import org.apache.spark.sql.types._
schema: org.apache.spark.sql.types.StructType = StructType(StructField(ECG,StringType,true), StructField(activity,StringType,true))
lines: org.apache.spark.sql.DataFrame = [key: binary, value: binary ... 5 more fields]
data: org.apache.spark.sql.DataFrame = [ECG: float, activity: string ... 1 more field]
import com.knockdata.spark.highcharts._
import com.knockdata.spark.highcharts.model._
<console>:72: error: overloaded method value apply with alternatives:
  (seriesHolders: com.knockdata.spark.highcharts.SeriesHolder*)com.knockdata.spark.highcharts.model.Highcharts <and>
  (seriesHolder: com.knockdata.spark.highcharts.SeriesHolder,z: org.apache.zeppelin.spark.ZeppelinContext,outputMode: String)org.apache.spark.sql.streaming.StreamingQuery
 cannot be applied to (com.knockdata.spark.highcharts.SeriesHolder, org.apache.zeppelin.spark.SparkZeppelinContext, String)
       val query = highcharts(

Thanks!