search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
10.51k stars 3.03k forks source link

Kafka connector: support for JSON, Protobuf formats with Confluent Schema Registry #23736

Open alexwirehead opened 1 month ago

alexwirehead commented 1 month ago

Hi. Are there any plans to add support for JSON, Protobuf formats in Kafka connector together with Confluent Schema Registry?

At the moment, according to the documentation, there is only AVRO support.

We tried to configure with JSON format but got an error:

2024-10-09T15:09:10.409Z WARN SplitRunner-20241009\_150909\_00009\_wcea6.2.0.0-1-236 io.trino.metadata.MetadataManager Failed to get metadata for table: kafka_test.default.trino-test ││ io.trino.spi.TrinoException: Not supported schema: JSON

wendigo commented 1 month ago

@alexwirehead not at the moment but contributions are welcome :)

alexwirehead commented 1 month ago

@wendigo Thank you for your reply. I would love to contribute, but unfortunately I have no experience in Java :(

Praveen2112 commented 1 month ago

But for schema registry integration we do have issues with Confluent as schema registry binaries for protobuf are backed by Confluent Community License https://www.confluent.io/confluent-community-license-faq/ - @mosabua/ @martint what's our plan for integrating such libraries ?