scylladb / scylladb

NoSQL data store using the seastar framework, compatible with Apache Cassandra
http://scylladb.com
GNU Affero General Public License v3.0
13.53k stars 1.29k forks source link

Add hidden static column to a partition. #20252

Open gleb-cloudius opened 2 months ago

gleb-cloudius commented 2 months ago

It is useful to have a place to store metadata per partition. One such example is paxos state for LWT operations. Having a static column in a partition that is not accessible from regular CQL queries (at least with a standard syntax), but can only be manipulated by internal APIs can be used to store such metadata.

Since the metadata is local per node repair should skip it. CDC should ignore it as well. It should be omitted from describe output and table metadata available to the drivers (since it is hidden).

kbr-scylla commented 2 months ago

cc @kostja @avikivity @bhalevy

from the meeting it seems there are multiple ways to approach this: implementing new type of column on storage level, or somehow reuse existing storage capabilities and add special handling to query layer level / repair etc. (based on the column name?), or some mix of both.

kbr-scylla commented 2 months ago

Required for https://github.com/scylladb/scylladb/issues/18068