I wanted to evaluate use of Scala 3 in our project that uses spark and came across your project and found it very useful.
However code generation did not work entirely, since we also rely on collections in our case classes used in spark.
Hence I've tried my best to add most of the datatypes that are supported in spark sql.
Maybe you can have a look at the PR - sorry it is quite big. I also added sbt-scalafmt which makes it possible to use scalafmtAll and scalafmtSbt inside sbt.
There is one long-ish and quirky matching expression at the end of Serializer and Deserializer in summonAll which makes it possible now to encode and decoder very long data types. I know that it is generally possible to do so when increasing "-Xmax-inlines:256" for example - but with this "bulk-processing" of the tuples that I added: I don't need that setting in our project and it doesn't crash the compiler with a stackoverflow. I tried to fix that stackoverflow but it seems the best way to do so is to restrict the stack size of that summonAll expression otherwise the compiler will crash occasionally in our project, especially when clean building.
I also added myself as a developer - I hope that is OK.
I also hope that this PR is OK - or give me your thoughts and please review. Thanks!
Hello @vincenzobaz
I wanted to evaluate use of Scala 3 in our project that uses spark and came across your project and found it very useful. However code generation did not work entirely, since we also rely on collections in our case classes used in spark. Hence I've tried my best to add most of the datatypes that are supported in spark sql.
Maybe you can have a look at the PR - sorry it is quite big. I also added
sbt-scalafmtwhich makes it possible to usescalafmtAllandscalafmtSbtinside sbt.There is one long-ish and quirky matching expression at the end of Serializer and Deserializer in
summonAllwhich makes it possible now to encode and decoder very long data types. I know that it is generally possible to do so when increasing "-Xmax-inlines:256" for example - but with this "bulk-processing" of the tuples that I added: I don't need that setting in our project and it doesn't crash the compiler with a stackoverflow. I tried to fix that stackoverflow but it seems the best way to do so is to restrict the stack size of thatsummonAllexpression otherwise the compiler will crash occasionally in our project, especially when clean building.I also added myself as a developer - I hope that is OK.
I also hope that this PR is OK - or give me your thoughts and please review. Thanks!
Regards Michael