search

damus-io / nostrdb

The unfairly fast embedded nostr database backed by lmdb
Other
87 stars 15 forks source link

Consider Cap'n Proto optimizations #17

Closed vRobM closed 8 months ago

vRobM commented 10 months ago

Since you're familiar with flatbuffers.. ;-)

jb55 commented 10 months ago

On Mon, Sep 11, 2023 at 12:17:51PM -0700, Rob wrote:

Since you're familiar with flatbuffers.. ;-)

I'm not set on flatbuffers, it's just what I'm using right now. I was looking at the capn' proto comparison to flatbuffers the other day and there doesn't seem that much difference that is relevant to nostrdb. do you have something specific in mind?

vRobM commented 10 months ago

Understood.

Not an expert on it, so on the surface yes, fairly similar, however it's a pretty deep subject depending on which layer you look at which can be optimized.

Not sure what the data format considerations are in nostrdb, for example the binary data format would be more compact compared to JSON. How that would affect KV needs or other transformations & manipulations IDK. Depends where the overhead is acceptable.

https://www.youtube.com/watch?v=6V_lVZzV6AE This guy delves into the depths of Cap'n Proto which may provide useful perspectives, however limited to the example app he builds.

A learning opportunity for sure, and we can connect with Kenton who designed it if desired.

jb55 commented 8 months ago

closing this until we have some specific thing we are trying to optimize for