Eviction - Githubissues

xtdb / xtdb

An immutable database for application development and time-travel data compliance, with SQL and XTQL. Developed by @juxt

https://xtdb.com

Mozilla Public License 2.0

2.5k stars 161 forks source link

Eviction #1950

Open jarohen opened 3 years ago

jarohen commented 3 years ago

potential spec:

removed immediately from the k-d tree, as if it were an all-vt all-tt delete
removed eventually from all other indices
can split into two parts - maybe the first part can submit a 'work item' to the object store, and maybe that's mergeable in its own right

thoughts:

better to do xtdb/xtdb#2036 first?
also need to evict/not store eids in the log index
shouldn't change row-ids
consider evicting for a time range?
check for prior art in academia
use object store to co-ordinate work - cache headers/etags etc.
related to xtdb/xtdb#1944

refset commented 1 year ago

Noting a link to the ~recent ADR: https://github.com/xtdb/core2/blob/63c98af7bfabb5fa961594f55f6facd52e79163f/doc/adr/0004-eviction.adoc