search

ipfs / notes

IPFS Collaborative Notebook for Research
MIT License
401 stars 30 forks source link

A weird suggestion, just saying - data preservation #361

Open deavmi opened 7 years ago

deavmi commented 7 years ago

I was thinking about something that would really help data preservation.

I was thinking that all IPFS nodes could have a feature to watch the network (the DHT?) and look for new objects that have been added, then check how many people are seeding it (gave it pinned) and if it makes the minumum cut then it leaves it alone but if it does not then it pins it.

This will allow all data to survive as say now there is cat.jpg nand no one has pinned it, it won't survive if the original hoster of it goes down.

Anyway that is just an idea in it's basic form. You'd also need to figure out how to get the minumum seeder count working as it is not necessarily true that in the first second node one will pin it and then then the next second anothr pins it. The global counting will be hard as more than one (probably a lot more) will pin it the second it comes onto the network (the DHT), so yeah. I guess you could do something where nodes negotiate on who should release the pinning of the file where you could do a coin flip and see which node does so.

Didn't mean to disturb the other things on the issue tracker, this just came to my mind. Sorry if I did, I did not mean to. :smile:

hsanjuan commented 7 years ago

Hello, we are working on https://github.com/ipfs/ipfs-cluster/ to orchestrate multi-pinning.

deavmi commented 7 years ago

Okay. Awesome. Glad to hear it is being worked on.

reit-c commented 7 years ago

Another thing to look at is the eventual Filecoin, which is designed to incentivize the network to keep files up.

deavmi commented 7 years ago

I know FileCoin is off-topic here but If I am correct with Filecoin all files are stored on all machines or are they distributed in a different way with say now x many have file b and x many have file d?

On 2017-02-05 10:49 AM, reit-c wrote:

Another thing to look at is the eventual Filecoin, which is designed to incentivize the network to keep files up.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ipfs/ipfs/issues/220#issuecomment-277505529, or mute the thread https://github.com/notifications/unsubscribe-auth/AE92vDnUA-XNgErnh0WyO19BdsnKONcnks5rZY0NgaJpZM4LYEwM.

reit-c commented 7 years ago

It is a mystery. Filecoin isn't in development yet.

deavmi commented 7 years ago

I see. Is there a specification for FileCoin though?

On 2017-02-08 06:02 AM, reit-c wrote:

It is a mystery. Filecoin isn't in development yet.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ipfs/ipfs/issues/220#issuecomment-278223341, or mute the thread https://github.com/notifications/unsubscribe-auth/AE92vHZQHmYyiD4v_ltGFJ8sLE_asvUUks5raT5ngaJpZM4LYEwM.

misrab commented 7 years ago

There's a whitepaper here I believe, but not sure if that's what you're looking for / up-to-date.

deavmi commented 7 years ago

Was just wondering if one existed, but thanks :)

On 2017-02-14 08:21 AM, misrab wrote:

There's a whitepaper here http://filecoin.io/filecoin.pdf I believe, but not sure if that's what you're looking for / up-to-date.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ipfs/ipfs/issues/220#issuecomment-279618714, or mute the thread https://github.com/notifications/unsubscribe-auth/AE92vL06lHCzaNwL3HTAH1o4zcNuXP_2ks5rcUfTgaJpZM4LYEwM.

bertrandfalguiere commented 6 years ago

I know FileCoin is off-topic here but If I am correct with Filecoin all files are stored on all machines or are they distributed in a different way with say now x many have file b and x many have file d?

It would be impossible to store EVERY FILE on EVERY MACHINE. (Or I misunderstood the question). Users will pay for storage, so if you want x copies of a file, there would be x copies encrypted differently (because of how its Proof-of-Replication works), and they will need to pay x times. So I guess it will be configurable and people will choose how many replicas they want. If Alice wants x copies of her file d and Bob wants x' copies for file d, there will be different numbers of copies for different files. Note that Proof-of-Replication, as described in Filecoin's whitepaper, means losing deduplication.