Research and test implications of using a single IPFS datastore and orbitdb-cache

3box / 3box-pinning-node

A node.js implementation of the 3box-pinning-node

MIT License

17 stars 5 forks source link

Research and test implications of using a single IPFS datastore and orbitdb-cache #280

Open msterle opened 4 years ago

msterle commented 4 years ago

Document answers to the following questions (with reproducible tests where suitable):

ipfs:

What purpose do locks serve for ipfs repos?
Why is it considered safe to not use locks on S3 ipfs repos, as appears to be shown by https://github.com/ipfs/js-datastore-s3/blob/master/src/s3-repo.js ?
Is it "safe" to ignore locks on other datastore types?
Are there conditions that should be met for not using locks, and if so, what are they? Do we currently satisfy these conditions, and will we continue to do so?
If it is safe, are there other disadvantage to have a single datastore? What is the motivation behind the ipfs-cluster project in this case?

orbitdb:

Under what conditions will an in-memory cache fail? (elasticache/redis)
Can concurrency issues arise from a shared orbitdb cache, and if so, under what conditions?
How can these issues be mitigated?

links:

https://github.com/ipfs/go-ipfs/issues/6556

KickedDroid commented 4 years ago

Why is it considered safe to not use locks on S3 ipfs repos

I asked this question before and the answer I got was s3 doesn't handle deduplication. So two writes can be the same but will be two different entries. If you want to minimize bandwidth and storage usage, locks would prevent this? And I suppose you would have to deduplicate the datastore yourself. FYI take this with a grain of salt I'm just repeating what I was told.

On a side note how do you get ipfs nodes to share a single datastore?

zachferland commented 4 years ago

@AIDXNZ currently we run ipfs-js nodes configured with an ipfs-repo with shared storage backends (for root, blocks, keys, datastore), currently that is s3, could pass anything that implements ipfs datastore interface.

Depending on features/modules enabled in ipfs there maybe some issues with no locks for storage backends besides blocks (thats what this issue was more for, and its not necessarily recommend to run it this way). But for blocks have not had any concerns sharing across instances since all keys in s3 (CID) -> data.

for first question, yeah i think that is true in s3 if you are not keying the the writes yourself. But for the blocks store the CID is the key, and same keys are overwritten (unless you have versioning enabled). For bandwidth, pretty sure ipfs wont try to write blocks it already has (not sure).

KickedDroid commented 4 years ago

For bandwidth, pretty sure ipfs wont try to write blocks it already has (not sure).

I should have been more clear my bad. I was thinking more like if two nodes were to write the same block to s3, that would be inefficient right?

But that's really cool! Thanks for the great answer! Are the any problems with using a single datastore that you are aware of yet?