sparklyr / sparklyr

R interface for Apache Spark
https://spark.rstudio.com/
Apache License 2.0
945 stars 306 forks source link

Support for compute sinks #2061

Open javierluraschi opened 5 years ago

javierluraschi commented 5 years ago
Foreach sink - Runs arbitrary computation on the records in the output. See later in the section for more details.

https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html#output-sinks

harryprince commented 5 years ago

spark streaming is very meaningful for the online anti-spam machine learning scenario, wish to see better streaming pipeline support from Kafka reading to HBase writing.