search

apache / pinot

Apache Pinot - A realtime distributed OLAP datastore
https://pinot.apache.org/
Apache License 2.0
5.45k stars 1.28k forks source link

Pinot Broker Cache #656

Open xiangfu0 opened 8 years ago

xiangfu0 commented 8 years ago

Start this session to discuss the broker cache implementation. Some rough idea about this includes:

  1. We can try to cache partial results from composite key<query& segmentList> to corresponding server response. Next query will not hit those already queried segments and only hit segments not in cache.
  2. In request/response to indicate if cache is used or not.
  3. Cache TTL based on segment expiration and also the configurable cache size.
  4. Routing mapping will be online computed.
  5. This is per broker local cache for now, no global cache in picture for now.
raviarin commented 8 years ago

This is a pretty good feature to add to broker caching. It would be great if you can put up a design doc wiki page that folks can comment.