Large scale database - Githubissues

gjeanmart commented 6 years ago

Hi, I'm currently doing research on serverless databases and OrbitDB seems to be an amazing solution. Thanks for all this work.

However, I'm thinking about implementing a massive document store that could contains thousand and thousand of documents and make it searchable as much as possible. I looks like storing such content in a single OrbitDB docstore could potentially overload the node (browser) and make it crash ...

So I would like to know if a splitted implementation would make more sense and would scale better.

every document is a single OrbitDB docstore database such as "entity-01", "entity-02", ...
after inserting/updating a document, we update an index file (OrbitDB database) that only contains the IPFS hash of each document and a few searchable index fields.

Do you think this type of implementation can make sense and scale ?

Thanks. Greg

abh1manyu commented 5 years ago

Did you find an answer to this? I'm also trying to understand best way to approach similar problem.

RichardLitt commented 5 years ago

Thanks @gjeanmart and @abh1manyu. I think this question may be better asked in the main orbit-db repo; I'm afraid it may have gotten lost in here! Another thing to would be to ask on Gitter and to come along to our fortnightly community calls - there's one tomorrow.

I'll keep this open. Hopefully someone can get to it, soon. I'd encourage reposting it, though. I wish I personally knew the answers so that I could help!

aphelionz commented 4 years ago

Moving to the Field Manual repo for more details / discussion

orbitdb / field-manual

Large scale database #89