iondbproject / iondb

IonDB, a key-value datastore for resource constrained systems.
BSD 3-Clause "New" or "Revised" License
587 stars 48 forks source link

Fixed size of keys and values per DB? #2

Closed pfalcon closed 8 years ago

pfalcon commented 8 years ago

Do I understand it right that size of keys and values is fixed per DB ("dictionary")? But that doesn't scale at all!

iondbproject commented 8 years ago

Cutting any overhead possible was one of the primary goals of IonDB. The design choice to only allow fixed size keys and values was made in light of the resource constrained nature of our target platform.

pfalcon commented 8 years ago

If it's resource-constrained, how it helps to store a VARCHAR(255) field (using SQL terminology) in 255 bytes of storage, knowing that in reality length of data is logarithmically distributed? How it helps to waste 90% of storage space, how it helps to move 255 bytes around, instead of average 10, how it helps to write 255 bytes instead of 10+N (N is small) to flash?

That said, not arguing of design choices, just pointing that it would help to make this fact explicit in README.

graemedouglas commented 8 years ago

My comment here would be that the design decision was more drastic: you don't get varchars. In that sense, sure, its lacking, but in most cases, I predict you won't be storing variably sized strings on your microcontroller.

pfalcon commented 8 years ago

I predict you won't be storing variably sized strings on your microcontroller.

That's as true as saying that people won't want to use SQL on their microcontrollers ;-). (With reference to LittleD.) But again, I may understand a need to apply some constraints to design (regardless whether I think a particular constraint is reasonable (though it's hard to argue that free space management for variable-sized data, and thus deletion support, is rather more complicated than for fixed-size)). Again, it just would be helpful to know these constraints right away.

iondbproject commented 8 years ago

Again, it just would be helpful to know these constraints right away.

We definitely agree on this point, and the README has been updated to explicitly state this.

Thank you pfalcon!