search

cockroachdb / cockroach

CockroachDB — the cloud native, distributed SQL database designed for high availability, effortless scale, and control over data placement.
https://www.cockroachlabs.com
Other
30.11k stars 3.81k forks source link

sql: investigate auto-splitting large values across KV pairs (adaptive column families) #13195

Open RaduBerinde opened 7 years ago

RaduBerinde commented 7 years ago

I think there is some value to think about what it would take to have adaptive column families. Specifically, allow large values to be "pulled out" of a family to a separate KV upon insertion, on a row-by-row basis. If it helps, we could leave a special marker inside the family which indicates that the value was separated out.

Updates will get trickier, but this would allow us to have large column families by default while still avoiding very large KVs (and thus slow updates).

CC @danhhz @petermattis

Jira issue: CRDB-6118

petermattis commented 7 years ago

Other database systems do this. We should too. For example, https://www.postgresql.org/docs/9.5/static/storage-toast.html.

knz commented 6 years ago

@jordanlewis in which project should this land do you think?

knz commented 6 years ago

Quoting @petermattis

While we could implement support for large values entirely via column families, I think it would be better to plumb that knowledge down into the Core layer.

jordanlewis commented 4 years ago

We might want to consider this again in the not-so-distant-future to help people who are trying to store large values in rows, and having memory usage and other issues from doing so.

github-actions[bot] commented 1 year ago

We have marked this issue as stale because it has been inactive for 18 months. If this issue is still relevant, removing the stale label or adding a comment will keep it active. Otherwise, we'll close it in 10 days to keep the issue queue tidy. Thank you for your contribution to CockroachDB!