search

qoollo / bob

Distributed BLOB storage
MIT License
31 stars 3 forks source link

Use array as a backing data type for Pearl Key #515

Open ikopylov opened 2 years ago

ikopylov commented 2 years ago

Currently, Pearl Key wraps Vec (https://github.com/qoollo/bob/blob/master/bob-backend/src/pearl/data.rs#L6), which is inefficient for key sizes less then or equal to 16. At the same time we have BobKey struct (https://github.com/qoollo/bob/blob/master/bob-common/src/data.rs#L15) that already wraps a constant size array, which is more efficient. I suggest to change backing type of pearl Key to constant size array

ikopylov commented 2 years ago

Additionally, it would be good to replace trait From<Vec<u8>> to From<&[u8]>

ikopylov commented 2 years ago

I think we should do the opposite: BobKey should use Vec under the hood. The reason is that we already get the key as Vec from the gRPC API, and we can avoid copying the key through code, so the actual overhead of using Vec becomes zero. Moreover in future the large size keys will work better with Vec

ikopylov commented 1 year ago

The best option is to use conditional compilation. For key sizes <= 16 it is better to store them in arrays, for >16 it is better to store them in Box<[u8]> (Box<u8> by itself occupies 16 bytes of memory, Vec<u8> on the other hand occupies 24 bytes)