search

iost-official / go-iost

Official Go implementation of the IOST blockchain
https://developers.iost.io/
GNU Lesser General Public License v3.0
564 stars 112 forks source link

IPFS integration #358

Open jerkyrs opened 6 years ago

jerkyrs commented 6 years ago

Consider using IPFS to store block data or file storage requirements so all nodes do not need to store all data. IPFS could also be used to store other data related to Dapps and could be closely linked to v8vm

https://ipfs.io/ https://ipld.io/ https://github.com/ipfs/js-ipfs

It is unclear from the whitepaper how exactly the past blockchain will be stored in its entirety

We mitigate this issue by moving the storage responsibility from past blockchains to the client. Since latest epoch’s blocks is retained, clients can ask the validators of the shard to create existence proof for transactions validated in epoch e during the next epoch.

How will the data actually be stored in what format

kaidiren commented 6 years ago

until now only some part of p2p use ipfs https://github.com/iost-official/go-iost/search?q=ipfs&unscoped_q=ipfs , the other has not yet been made public

jerkyrs commented 6 years ago

Is there a plan to use the Lambda PoST to index the blockchain to ensure durability?

publicfrenemy commented 5 years ago

IPFS is trash in its current state. Not even close to suitable for blockchain storage, or even storage refrence. Look at all the problems that Dtube has with files missing and load times

jerkyrs commented 5 years ago

@publicfrenemy your probably right, having experience with SDS deployments like CEPH which are permission-ed based these are incredibly complex and especially in relation to backfilling when a node or two goes offline meaning the durability of data has reduced until backfill is complete.

I think IOST are eventually going to use what is being designed by Lambda (not Amazon Lambda but an incubator project sponsored by Bluehill) which is PoST = Proof of Space Time , see https://medium.com/@Lambdaim/lambda-faq-ver-4-0-fb9a391908c2 . If you are going to store your blockchain on some type of permissionless distributed storage you are going to need to make sure it is not lost (like if a seed or to many nodes goes off a bittorrent i assume same is possible today in IPFS)

Longer term it is not going to be practical to store entire blockchain on single storage device, the time it takes for example to synchronize Ethereum is somewhat needless if you were able to just goto the head block and start participating by storing partial amounts of the chain or producing blocks, for the short term (1-3 year) and without high amount of transactions probably unnecessary.