search

higherkindness / skeuomorph

skema morphisms
https://higherkindness.github.io/skeuomorph/
Apache License 2.0
89 stars 15 forks source link

support avro logical type? #323

Open chenharryhua opened 4 years ago

dmarticus commented 4 years ago

@chenharryhua can you provide more information about what you mean by this issue? Is there a specific use case you have in mind that skeuomorph doesn't cover?

chenharryhua commented 4 years ago

My bad. I try to use skeuomorph in my project to transform avro schema to/from spark struct-type. my attempt looks like this: https://github.com/chenharryhua/nanjin/blob/master/spark/src/main/scala/com/github/chenharryhua/nanjin/spark/NJDataTypeF.scala . skeuomorph looks like a more general/idiomatic way to do it. but seems there are two missing parts in AvroF, one is how to transform avro logical type to AvroF and another is how to deal with decimal.

dmarticus commented 4 years ago

No problem at all! Thanks for sharing your example; I'll look into this more tomorrow and let you know what I find.

dmarticus commented 4 years ago

Confirmed that your assessment around the missing parts of AvroF is correct. Great catch! We'll triage this issue in our backlog and would happily welcome a PR if you have a good idea for the implementation :)

chenharryhua commented 4 years ago

Thank you, @dmarticus. I will have a try later. I am uncertain about how to transform AvroF to MuF once the logical type is added and its impact on Probuf schema which I thought also need to transform to MuF