search

mjakubowski84 / parquet4s

Read and write Parquet in Scala. Use Scala classes as schema. No need to start a cluster.
https://mjakubowski84.github.io/parquet4s/
MIT License
283 stars 65 forks source link

Feature Request: scalapb support #303

Closed huajiang-tubi closed 1 year ago

huajiang-tubi commented 1 year ago

Is there existing support for scalapb in this repo or anywhere else?

mjakubowski84 commented 1 year ago

I am not sure what you mean, but here you have an example of a writer with Protobuf schema using sbt-protobuf:

https://github.com/mjakubowski84/parquet4s/blob/master/examples/src/main/scala/com/github/mjakubowski84/parquet4s/fs2/CustomParquetWriterFS2App.scala

huajiang-tubi commented 1 year ago

parquet-protobuf only works with generated messages in Java. We are using akka-grpc and want to write the generated scala messages to parquet. Here is a proposal for the feature: #305

mjakubowski84 commented 1 year ago

Hi @huajiang-tubi, That's a nice add-on and quite an effort to implement it!

mjakubowski84 commented 1 year ago

Support for ScalaPB released in https://github.com/mjakubowski84/parquet4s/releases/tag/v2.13.0