search

Kotlin / kotlin-spark-api

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x
Apache License 2.0
456 stars 34 forks source link

Exploring streaming #133

Closed Jolanrensen closed 2 years ago

Jolanrensen commented 2 years ago

As mentioned here: https://github.com/JetBrains/kotlin-spark-api/issues/122 This WIP pr creates a withSparkStreaming() {} function and KSparkStreaming scope. It makes it easier to work with Apache Spark Streaming since the boilerplate code is already handled. See Streaming.kt for an example.

Jolanrensen commented 2 years ago

@asm0dey What do you think of the notation like this? I'll add some more tests for the tuple functions btw. Also, are there any other streaming examples you know of that I can test to see whether we support them?

Jolanrensen commented 2 years ago

@asm0dey checkmarks all around, finally. Any other tests or checks I need to do before merging streaming support?