search

trinodb / trino

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)
https://trino.io
Apache License 2.0
9.88k stars 2.86k forks source link

implement local cache and query result cache in trino #13932

Open Jay-ju opened 1 year ago

Jay-ju commented 1 year ago

ref， i want to introduce local cache

  1. rubix is not available for current version， in addition， its architecture must require install a rubix cluster， so it's heavy for cache. here i advice that cache is lib in worker.
  2. query result cache is import for bi scene. condition in ui is relatively same. reuse is necessary.

Local Cache Frame ref just like this, can we reuse those codes? in addition, we can solve some questions like multi disk，consistency

Query Result we will use (sql text + ConnectSplit id)'s md5 as cache key, sql result as cache value

martint commented 1 year ago

Trino already has a built in local cache for the hive connector. Please see