search

mjakubowski84 / parquet4s

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.
https://mjakubowski84.github.io/parquet4s/
MIT License
283 stars 65 forks source link

fix: proper resolution of custom types #250

Closed mjakubowski84 closed 2 years ago

mjakubowski84 commented 2 years ago
  1. Fixes resolution of implicit codecs and schemas for custom types provided by users of the library. Especially, allows using classes and case classes as a custom type - by giving higher priority to implicits provided by a user.
  2. Custom schema is not processed by ParquetSchemaResolver during partitioning and its fields are not filtered (partition fields are nor removed) - user has to take partitioning into account when defining a schema.
  3. Several minor library upgrades
  4. Compatibility classes are moved to dedicated compat package
vreuter commented 2 years ago

Thanks again @mjakubowski84 , just tested with the new version and looks to be working great!