search

k3s-io / kine

Run Kubernetes on MySQL, Postgres, sqlite, dqlite, not etcd.
Apache License 2.0
1.56k stars 233 forks source link

[feature request] go-memdb backend #97

Open tao12345666333 opened 3 years ago

tao12345666333 commented 3 years ago

I want to know if we can add a new backend using go-memdb as in-memory database.

https://github.com/hashicorp/go-memdb

Provides the memdb package that implements a simple in-memory database built on immutable radix trees. The database provides Atomicity, Consistency and Isolation from ACID. Being that it is in-memory, it does not provide durability. The database is instantiated with a schema that specifies the tables and indices that exist and allows transactions to be executed.

The database provides the following:

  • Multi-Version Concurrency Control (MVCC) - By leveraging immutable radix trees the database is able to support any number of concurrent readers without locking, and allows a writer to make progress.

  • Transaction Support - The database allows for rich transactions, in which multiple objects are inserted, updated or deleted. The transactions can span multiple tables, and are applied atomically. The database provides atomicity and isolation in ACID terminology, such that until commit the updates are not visible.

  • Rich Indexing - Tables can support any number of indexes, which can be simple like a single field index, or more advanced compound field indexes. Certain types like UUID can be efficiently compressed from strings into byte indexes for reduced storage requirements.

  • Watches - Callers can populate a watch set as part of a query, which can be used to detect when a modification has been made to the database which affects the query results. This lets callers easily watch for changes in the database in a very general way.

brandond commented 3 years ago

That might be hard, since kine expects a SQL compatible storage engine. Seems like if you want an ephemeral dataatore you could get the same result out of sqlite with the special :memory: database file path?

tao12345666333 commented 3 years ago

yep, at the same time I also hope not to introduce other components to accomplish this.

tao12345666333 commented 3 years ago

maybe we can using https://github.com/araddon/qlbridge to translate it. https://github.com/araddon/qlbridge/tree/master/datasource/memdb

tokers commented 3 years ago

Introduce an SQLite with the in-memory file path is still a bit heavy for us, we need some further discussions.

brandond commented 3 years ago

What specifically are you trying to achieve?

brandond commented 3 years ago

fwiw, I tried using sqlite in memory mode and it's pretty flakey, due to changes in table locking semantics.

docker run --rm -it --privileged --name k3s-server-1 rancher/k3s server --datastore-endpoint 'sqlite://file:memdb1?mode=memory&cache=shared'
akutz commented 2 years ago

fwiw, I tried using sqlite in memory mode and it's pretty flakey, due to changes in table locking semantics.

Is this still the case? I've recently started exploring https://github.com/kcp-dev/kcp/issues/54#issuecomment-1074142302 how to use kine with kcp as a means to support a light-weight binary CLI program that uses controllers. I was hoping to leverage an ephemeral, in-memory DB like sqlite in-mem for this purpose.

brandond commented 2 years ago

You could just use kine but store the SQLite database in a temporary directory that's not persistent?

akutz commented 2 years ago

You could just use kine but store the SQLite database in a temporary directory that's not persistent?

True, but part of the goal is a super portable, self-contained means of implementing controllers in CLI programs. I'll take your suggestion under advisement. Based on the response, can I assume that in-memory sqlite still does not work well?

brandond commented 2 years ago

You're welcome to try it and see if it works for you. It didn't work very well for a whole Kubernetes cluster using the example command I gave.

akutz commented 2 years ago

Ack. Thank you. So just to be clear, to your knowledge no one has a) tried it since you did last July or b) worked on it since then either?

brandond commented 2 years ago

yep.

Some of the requirements to use the sqlite in-memory store are described here: https://www.sqlite.org/inmemorydb.html - I don't remember specifically what I ran into with locking, but it should be easy enough to reproduce.

brandond commented 2 years ago

Seems to be the same issue described here: https://github.com/mattn/go-sqlite3/issues/50#issuecomment-87811607

mhmxs commented 5 months ago

That might be hard, since kine expects a SQL compatible storage engine.

@brandond Is this statement still valid?

The reason I think it may not, is: here https://github.com/k3s-io/kine/blob/master/pkg/endpoint/endpoint.go#L59 kine creates a raw etcd listener without any sql specific wrapper. It looks simple task to create any backend, not just SQL. As i see a backend only has to implement this interface: https://github.com/k3s-io/kine/blob/master/pkg/server/types.go#L20 and has to start it's own listener.

:pray:

brandond commented 5 months ago

We now have a non-sql backend in https://github.com/k3s-io/kine/tree/master/pkg/drivers/nats, but it was fairly non-trivial for the Synadia folks to implemement. You might look at that as an example though.