Kotlin / kotlin-spark-api

This projects gives Kotlin bindings and several extensions for Apache Spark. We are looking to have this as a part of Apache Spark 3.x
Apache License 2.0
456 stars 34 forks source link

Tuple copy method with changing types #156

Open Jolanrensen opened 2 years ago

Jolanrensen commented 2 years ago

Currently when calling copy(_x = ...) on a Tuple, the new values need to have the same type as the original Tuple. This might however not be what the user expects or needs.

Here is my suggestion with an example for Tuple2:

@JvmName("copy_1_2")
fun <R1, R2> Tuple2<*, *>.copy(_1: R1, _2: R2): Tuple2<R1, R2> = Tuple2<R1, R2>(_1, _2)
@JvmName("copy_1")
fun <T2, R1> Tuple2<*, T2>.copy(_1: R1, _2: T2 = this._2): Tuple2<R1, T2> = Tuple2<R1, T2>(_1, _2)
@JvmName("copy_2")
fun <T1, R2> Tuple2<T1, *>.copy(_1: T1 = this._1, _2: R2): Tuple2<T1, R2> = Tuple2<T1, R2>(_1, _2)
fun <T1, T2> Tuple2<T1, T2>.copy(_1: T1 = this._1, _2: T2 = this._2): Tuple2<T1, T2> = Tuple2<T1, T2>(_1, _2)

val tuple: Tuple2<Int, String> = 1 X "2"

val a: Tuple2<Double, Long> = tuple.copy(_1 = 2.0, _2 = 3L)
val b: Tuple2<Double, String> = tuple.copy(_1 = 1.0)
val c: Tuple2<Int, Double> = tuple.copy(_2 = 1.0)
val d: Tuple2<Int, String> = tuple.copy(_1 = 2, _2 = "3")
val e: Tuple2<Int, String> = tuple.copy()
Jolanrensen commented 2 years ago

Well... image

It can be generated like this, but the file becomes 7+ GB... so enjoy I guess... But for real, the only way to do this would be to create a compiler plugin that generates the necessary functions on the fly.

private fun MutableList<Int>.addOne(base: Int): MutableList<Int> {
    for (i in indices.reversed()) {
        if (this[i] < base - 1) {
            this[i]++
            break
        }
        this[i] = 0
    }
    return this
}

private fun Int.toBoolean(): Boolean {
    require(this in 0..1) { "int should be either 0 or 1" }
    return this == 1
}

private fun main() {
    val alphabet = (2..22).toList()

    val file =
        File("/data/Projects/kotlin-spark-api (copy)/scala-tuples-in-kotlin/src/main/kotlin/org/jetbrains/kotlinx/spark/api/tuples/NewTupleCopy.kt")

    if (file.exists()) file.delete()
    file.createNewFile()

    file.outputStream().bufferedWriter().use { writer ->

        @Language("kt")
        val _1 = writer.write(
            """
            |package org.jetbrains.kotlinx.spark.api.tuples
            |
            |import scala.*
            |
            |fun EmptyTuple.copy(): EmptyTuple = EmptyTuple
            |
            |fun <T1> Tuple1<T1>.copy(_1: T1 = this._1()): Tuple1<T1> = Tuple1<T1>(_1)
            |@JvmName("copy_1") fun <R1> Tuple1<*>.copy(_1: R1): Tuple1<R1> = Tuple1<R1>(_1)
            |
            """.trimMargin()
        )

        for (i in alphabet) {
            val typeMap = MutableList(i) { 0 }
            val types = (1..i).toList()
            do {
                val booleanTypeMap = typeMap.map(Int::toBoolean)

                // T1, *, T3 etc.
                val inputTupleTypes = (types zip booleanTypeMap).map { (it, keep) -> if (keep) "T$it" else "*" }

                // T1, R2, T3 etc.
                val outputTupleTypes = (types zip booleanTypeMap).map { (it, isT) -> if (isT) "T$it" else "R$it" }

                // copy_2 etc.
                val copyName = "copy" + outputTupleTypes
                    .filter { 'R' in it }
                    .joinToString("") { "_" + it.removePrefix("R") }

                // _1, _2, _3 etc.
                val argumentNames = types.map { "_$it" }

                val arguments = (types zip booleanTypeMap).map { (it, isT) ->
                    "_$it: ${if (isT) "T$it = this._$it" else "R$it"}"
                }

                @Language("kt")
                val _2 = writer.write(
                    """
                    |@JvmName("$copyName") fun <${outputTupleTypes.joinToString()}> Tuple$i<${inputTupleTypes.joinToString()}>.copy(${arguments.joinToString()}): Tuple$i<${outputTupleTypes.joinToString()}> = Tuple$i<${outputTupleTypes.joinToString()}>(${argumentNames.joinToString()})
                    |
                    """.trimMargin()
                )

                typeMap.addOne(2)
            } while (typeMap.any { it.toBoolean() })
        }
    }
}
Jolanrensen commented 2 years ago

It's strange, Scala cán do this. It allows the type to switch within the copy method, keeping the original types when a parameter is not supplied. It feels like it's possible to do this in Kotlin as well, but I just don't know how

Jolanrensen commented 2 years ago

Related youtracks issue: https://youtrack.jetbrains.com/issue/KT-52519