search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.47k stars 1.28k forks source link

[Discussion] Event listener to gather query stats/information #7391

Open dharakk opened 3 years ago

dharakk commented 3 years ago

Hi, We have been looking into doing query analysis in order to pin point a specific scenario and combing through server and broker logs is a difficult task. An event listener interface which enable us to add a hook into query execution and gather query info/stats would be really helpful. For example Presto provides the following interface to hook into query execution. https://github.com/prestodb/presto/blob/master/presto-spi/src/main/java/com/facebook/presto/spi/eventlistener/EventListener.java

Do we already have a similar mechanism in Pinot? Have we ever considered doing so as I can imagine it wont be as simple as Presto because of the query volume and latency sensitive nature of Pinot?

mcvsubbu commented 3 years ago

Yes, see https://github.com/apache/pinot/blob/master/pinot-broker/src/main/java/org/apache/pinot/broker/api/RequestStatistics.java

You will need to plug in your own mechanism to emit this from the broker

kishoreg commented 3 years ago

Good idea. As Subbu mentioned, we have all the information but we lack the listener interface. It's definitely worth adding one in our spi.