Kotlin / dataframe

Structured data processing in Kotlin
https://kotlin.github.io/dataframe/overview.html
Apache License 2.0
772 stars 48 forks source link

add infer types parameter in DSL functions #579

Closed koperagen closed 4 months ago

koperagen commented 5 months ago

I believe tests that i added speak for themselves. Type can often be erased to Any and we need an easy way to get a meaningful result

Jolanrensen commented 4 months ago

I think this requires more explanation, as expr already has the ability to generate a type. However, this type is only known if it is supplied to expr as reified parameter. If a column needs to be added using a normal generic function, this type is not known and the newly added column will always be Any/Any?. This PR adds the option to infer the (schema) type from the data even if the reified type is too broad.

koperagen commented 4 months ago

This summarizes the issue how i initially found it. There was no way to create anything other than Any column from my list of Any that i got from a library. Changes in AddDsl were made for consistency of expr functions

 val data: List<Any> = listOf(1, 2, 3)
val res = data.toDataFrame {
      "d" from { it } // Any :( 
      "e" from inferType { it } // Int
       expr(infer = Infer.Type) { it } into "d" // Int
}