search

isar / hive

Lightweight and blazing fast key-value database written in pure Dart.
Apache License 2.0
4.06k stars 403 forks source link

How much & how data can be stored in Hive #536

Open vimalmistry opened 3 years ago

vimalmistry commented 3 years ago

Question App like whatsapp. If app have large messages data. And I create messages box. My question is.

How much data can be stored in single box? Does hive use file splitting as per size for offline data store? For example chunks of 100mbs rather then 1gb file ?

Code sample No Code

Version Latest version

themisir commented 3 years ago

How much data can be stored in single box?

I'm not sure if there's a limit, but storing too much will affect performance.

Does hive use file splitting as per size for offline data store? For example chunks of 100mbs rather then 1gb file ?

No, it doesn't.

vimalmistry commented 3 years ago

@TheMisir So each box have it's own file?

themisir commented 3 years ago

@TheMisir So each box have it's own file?

Yes.

vimalmistry commented 3 years ago

@TheMisir . Does hive support lazy data load from disk to memory? or its all loaded in memory by default?

themisir commented 3 years ago

Hive supports lazy loading. Please check Lazy box section on hive docs.

DhavalRKansara commented 2 years ago

So Is it possible to store the large data in a single box with size of 1 gb or greater?

themisir commented 2 years ago

So Is it possible to store the large data in a single box with size of 1 gb or greater?

I haven't done specific benchmarks to test how much does performance gets compromised at which size. So I can't say with specific numbers, but I would suggest using more mature DBMS like SQLite or others for working with bigger data sources. They'll also let you create indices which might increase your query speed. But depending on your use case Hive might also work for you. It really depends on what you're using your data store for. Each database have its own strengths and weaknesses.