search

Kotlin / kotlin-spark-api

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x
Apache License 2.0
459 stars 35 forks source link

added encoders 3.2 #134

Closed Jolanrensen closed 2 years ago

Jolanrensen commented 2 years ago

Duration, Period, ByteArray (Binary, now actually working)

added tests for: Instant datatype, duration, period, binary, and others added datatypes: ByteArray (BinaryType), CalendarInterval, Nothing (All three not yet working)

metioned: https://github.com/JetBrains/kotlin-spark-api/issues/131

WIP

Jolanrensen commented 2 years ago

Not sure why serializing ByteArrays with knownTypes doesn't work actually... it should just work

Jolanrensen commented 2 years ago

okay the Period encoder seems to be broken by Spark itself. Other than that, all 3.2 stuff seems to be working and the tests actually test the contents now instead of just showing it

Jolanrensen commented 2 years ago

Oh right and the KotlinReflection scala file is now more readable :)