Adding functionality to satisfy v0.itworks

[uncleDecart]

This commit removes NatsClient and also introduces proper subscribe mechanism, which you can actually use to receive new values.

There's a bunch of things to do till version v0.itworks , we can slowly continue moving with multiple PRs or I'll just create commit for each point in TODO list

TODO:

• [x] Gracefully handle unsubscribe when we delete value
• [x] For subscriber handle server response in case if subscription is not found
• [x] remove NATS from GHA
• [x] make nkv persistent, i.e. try load from checkpoints when create values
• [x] Add id tracking for requests, right now all requests are with id 0 which is not useful, we need this for any kind of debugging
• [x] Introduce some graphs
• [x] write basic benchmarking
• [x] Introduce proper errors and their returns
• [x] Introduce graceful shutdown for server and notifier via channels (probably oneshot ones)

[uncleDecart]

@deitch PR is ready for review, I'll do rest of todo in separate PR(s)

Edit: I tried to make it easier to review commit-by-commit, i.e. each commit serves one purpose

[deitch]

I put some comments inline. One other important thing that I didn't address is the "keyspace". What is our keyspace? I see a few options:

• Global, every put/get/delete/subscribe uses a single universal key space, so no difference between keys named "foo", "bar", "a.b.c", "this*1me"
• Global but with character restrictions (no *, etc.)
• Namespaced, either explicitly via put(namespace, key, value) or implicit, like NATS, via . separated, as in put(namespace1.key1, value) which might be stored and thus searched differently. Both of these imply some ability to do get("namespace1.*") or getAll("namespace1"), similar for delete and subscribe (doesn't make sense for put()

[uncleDecart]

Regarding "key space" I believe our best option would be to have it implicit like NATS, via separated dot. Because technically it won't change the way we store things and the only thing that would be added is to get all by a regex as a special case, we can then update documentation

[deitch]

Are you sure that implicit doesn't change storage? There can be significant performance differences between get("foo.*") when it's all in a single table and you need to check each key, vs separate tables.

[uncleDecart]

Are you sure that implicit doesn't change storage? There can be significant performance differences between get("foo.*") when it's all in a single table and you need to check each key, vs separate tables.

True, performance will be different, but depends on how many keys will we have? Say we are talking about 20 services which are namespaces of first order, then there are subscriptions/publications of 25 in each which are namespaces of 2nd order and each of them would contain say N values for each Edge App (say 25 as well). So it's ~12500 entries in total. How much it would increase our performance to iterate over 625 entries say instead of 12500 for each request? Well, I need to add benchmarks so that we could see it in microseconds :D

[deitch]

You're thinking eve. I'm thinking, you're building a generic high performance low footprint kv store.

[uncleDecart]

You're thinking eve. I'm thinking, you're building a generic high performance low footprint kv store.

Question is then, how many layers of nested namespaces should we allow in such generic store?

[deitch]

Truth is, it doesn't matter. Define the API day one, you always can optimize the back end later. You just need to think about what the API would constrain later.

[uncleDecart]

Then making it implicit makes sense, we can change implementation later, so client can do wild masks on dot patterns, wdyt?

[deitch]

Sure. Start with that, use regex for linear search. Split it in the future.

[uncleDecart]

Put description in #6 will address it in separate PR, it'll need some love

[uncleDecart]

Actually got too into it so I finished all the todo list, @deitch I see one unresolved discussion about documentation, once it's done I think we're safe to merge this and proceed with other issues