timonviola
commented
2 weeks ago Hey, did you check this part of the docs: https://doc.arroyo.dev/connectors/confluent#avro
Just to understand you are seeing the expected behavior (quoting the docs):
For Avro, there are some features that cannot be converted to SQL types:
- Unions (aside from unions that are a single type and null, which are converted to nullable columns) ... Fields with these types will be re-encoded as JSON and stored in a JSON column, in which form they can be processed via SQL JSON functions.
Is this what you are referring to "source shows as JSON"?
kvedes
Hi
I'm trying to use Arroyo on a Kafka topic from Confluent. I can successfully set up the connection and get the Avro schema which is stored in the Schema Registry. However, when I try to create a pipeline my source shows as JSON instead of Avro. I suspect this is due to my Avro schema containing Union types. I cannot find any documentation related to union types, so I suppose it might not be supported. Will this be coming the with the Protobuf support? - Protobuf has similar semantics just called oneof. Otherwise I would like to request it.
P.S. I'm brand new to Arroyo