search

philippgille / gokv

Simple key-value store abstraction and implementations for Go (Redis, Consul, etcd, bbolt, BadgerDB, LevelDB, Memcached, DynamoDB, S3, PostgreSQL, MongoDB, CockroachDB and many more)
Mozilla Public License 2.0
749 stars 70 forks source link

Multi user #72

Open ghost opened 5 years ago

ghost commented 5 years ago

Embedded stores and non embedded stores ( like my SQL ) are run differently.

I had an idea to make switching adnostic, in that boltdb or badger can be used from many processes, by wrapping access with a connection poller

Also using this lib with the Google go- cloud project and wire IOC mechanism would make this much cleaner too.

philippgille commented 5 years ago

I had an idea to make switching adnostic, in that boltdb or badger can be used from many processes, by wrapping access with a connection poller

I'm not sure if I understand you correctly.

bbolt isn't connection-based by itself, so that would require an actual server with its own API that uses bbolt as an underlying store, just like etcd does (see here). I'm not sure how this would fit into gokv, because gokv so far only contains the gokv.Store interface and multiple implementations and a connection-based server wouldn't implement that interface. Also creating a server has a much broader scope:

I think that's a project of its own, because of its size and also because it's not an implementation of the gokv.Store interface. It will also be extremely hard to achieve the performance and stability of existing server-based key-value stores like etcd, which has many people working on it with backing by a large company. A plain key-value store with less features (maybe without cluster capabilities) might be a little simpler, but still. But of course, assuming such a server exists, then gokv would be the right place to add a client implementation to.

Until then maybe check out projects like LedisDB which is a server that supports the Redis protocol and supports multiple underlying key-value stores: LevelDB, RocksDB, in-memory. It probably has an architecture that makes it easy to implement even more underlying stores here, like bbolt and BadgerDB. I didn't try LedisDB out yet, but it might already work with gokv's redis package.

Does this address your point or did I completely misunderstand you?

Also using this lib with the Google go- cloud project and wire IOC mechanism would make this much cleaner too.

go-cloud doesn't support any key-value store yet. There is an open issue for adding key-value stores and I already asked if I could help with implementing it a while ago, because the result could look similar to gokv. See here.

Regarding the existing features, see here. Of these features only the "Unstructured binary (blob) storage" and "Connecting to MySQL and PostgreSQL databases (mysql, postgres)" features are relevant to gokv. go-cloud doesn't make any guarantees regarding compatibility with other databases, so it probably can't or shouldn't be used with MySQL/PostreSQL protocol-compatible databases like CockroachDB or TiDB. But gokv already supports CockroachDB and will add support for TiDB in the future, so that would mean we have to keep gokv's SQL abstraction package sql anyway, and then maintain two abstractions instead of one.

Regarding dependency injection with wire: When I checked out wire the last time it was still living in the go-cloud package as an internal tool, and there was a proposal to move it to its own repo (see here, I think the issue was moved from the go-cloud repo, or maybe there was another one), so it didn't seem to be ready to be used by other projects. This has changed two months ago (see here), which makes it more interesting.

I'm not sure if it makes using gokv any easier, considering there is code generation involved, but I'll have another look at it. I created issue #74 for the evaluation.