pipermerriam / ipfs-persistence-consortium

A consortium to help persist IPFS objects until Swarm arrives
73 stars 15 forks source link

Hey Piper! #3

Open GriffGreen opened 7 years ago

GriffGreen commented 7 years ago

Hey @pipermerriam :-D! Giveth and Swarm City are looking at building something similar to this (and once we get it working we will likely include others (First on the list is Aragon, then we will start on boarding more groups) @sophiii is leading it with help from @sponnet and @jbaylina

A few brief questions:

  1. Did you end up going forward with this? If not, why not?!
  2. Are their other repos we should look at?
  3. Have others done this?
  4. Any words of wisdom?

Much love, Griff

pipermerriam commented 7 years ago

This specific implementation is generally dead/dormant. I'd be interested in participating if a new group sprung up to do this. I think that doing it with a simple smart contract would be a good easy way to do it. We need something like this for ethpm and I imagine there are now more others who also need similar persistence. Lets do something.

sophiii commented 7 years ago

Cool, We have a contract for this https://github.com/swarmcity/IPFSProxy next we will build a docker container that caches all of the hashes that are published. We will soft launch with 2 or 3 nodes and add other project organically.

GriffGreen commented 7 years ago

@pipermerriam we are going for a proof of concept with Swarm City, if you want to follow the progress we are working on the giveth slack in the #ipfs_together channel. You can sign up for the Giveth Slack at http://slack.giveth.io if you want to follow the progress... but you can probably just save your attention for other things, while we try to get this done without getting too distracted ourselves ;-)

pipermerriam commented 7 years ago

@sophiii looks like the contract you link exposes a single API for adding a single hash. Assuming that is correct, do you have any plans for how to mitigate the cost of adding many hashes so that it doesn't require a transaction per hash?

I believe this is possible using the IPFS protocol itself. If memory serves, there should be a way to create a single hash which represents an arbitrary set of other IPFS hashes.

Either way, nice to see this is happening and we will definitely use it in the coming months if it's functional to help add extra mirrors to ethpm data (and help others mirror their data in the process)

sophiii commented 7 years ago

Yes IPFS lets you upload folders as a single IPFS hash. That is what we plan to use for now. We plan to do a single update a day. Where you make all the data persistence. Think nightly backup to tape in data center.

Other solutions we explored;

  1. Use whisper to add ipfs hashes to persitance consortium Tho that comes at the cost of extra complexity and for now we would like to maintain the simpler version and investigate further optimizations once we have a solid POC.
  2. Have a contract that tracked other contract addresses and each node watches these addresses for a predefined event. This event would contain the IPFS hash the we add to persitance. That way we could add files to IPFS for the low gas cost of an event.

Most of the spec is discussed https://giveth.slack.com/messages/C6AAF4MHU/details/ if you would like to join and advocate one of these other options. But for now our preference is simplicity.

akaihola commented 5 years ago

This specific implementation is generally dead/dormant.

@pipermerriam, could you state this in the repository description?

sponnet commented 5 years ago

We built further on these concepts in this repos https://github.com/ipfsconsortium And https://github.com/dappnode/DAppNode

On Sun, Aug 12, 2018, 14:59 Antti Kaihola notifications@github.com wrote:

This specific implementation is generally dead/dormant.

@pipermerriam https://github.com/pipermerriam, could you state this in the repository description?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/pipermerriam/ipfs-persistence-consortium/issues/3#issuecomment-412341304, or mute the thread https://github.com/notifications/unsubscribe-auth/AAka9tXyx4PbPkntv4zUBPGzG-V93Swdks5uQCa1gaJpZM4OZBV6 .