search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
9.83k stars 2.85k forks source link

Add event-listener plugin with Kafka sink #22426

Open marton-bod opened 1 week ago

marton-bod commented 1 week ago

Currently open source Trino has only MySQL and http event listener plugins. These are not always scalable, especially when several larger Trino clusters are sending hundreds of thousands of events to the same centralized store. Kafka is equipped to handle that sort of load and therefore is used by several companies to sink their QueryCreatedEvents and QueryCompletedEvents. I would propose to create a new Trino plugin called trino-kafka-event-listener.

mosabua commented 1 week ago

That would be awesome .. I heard a mention at Trino Fest 2024 that you might know where such a plugin already exists ..