Perf : question

[Quafadas]

I can't see discussions, which is where this should be, as it's not an issue, sorry for that.

case class User(name: String, age: Int)

val users = UserOf[Array](
 name = Array("Albert", "Benedict", "Christopher", "Diane"), 
 age = Array(59, 42, 22, 36))

val plus1 = users.age.map(_+1)

val agedUsers = users.copy(age = plus1)

I ask, because arrays are quite performant, and I'm curious about whether this could be used to preserve metadata during some computationally intense transformations. Could this pseudocode be possible / reasonably performant?

[tschuchortdev]

Did you declare UsersOf manually or are you using the HkdFor type? If UsersOf is declared manually, then it is just a plain old case class containing arrays and no library code is even involved in your example. The performance characteristics then come down to access patterns: Compared to an Array[User], sequential accesses to all ages would likely be faster due to CPU cache lines and the map(_+1) operation could even be vectorized (in theory). Accessing name and age sequentially for each user would in turn be slower because the fields are further apart in memory and are thus not in the CPU cache. What you have there is essentially a dataframe with columnar storage, typically used in data analysis use cases.

If you are using functions from my library (like mapK) on a manually declared UserOf case class, then the performance of that mostly comes down to Shapeless 3. Shapeless is also using untyped arrays everywhere internally and has been carefully optimized, so I would expect pretty good performance here. Still, you are copying case classes whenever you modify them and doing it through Shapeless might possibly impede copy elision optimizations in the compiler. I don't know how smart the compiler is in that regard (it is a big problem in Haskell). Thus you should always benchmark, as you can never be sure if finicky compiler optimizations may lead to unexpected results.

If you are using HkdFor to generate the UsersOf type, then things become more complicated as it is no longer a real case class. Still, the fields of HkdFor are internally stored in an untyped array and field accesses are resolved to an array index by a macro at compile time. Therefore I would also expect reasonable performance here, similar to a regular case class.