Use Case: Pinning Buddy System

lidel commented 6 years ago

Together with @olizilla we've identified some UX challenges related to sharing files via IPFS (#34), and one of ideas was to provide an opt-in trust-based way for a group of friends to participate in a "pinning buddy system".

Disclaimer:

This may exist in some shape and form already (see Prior Art below), this issue is just a placeholder to investigate UX around making it work for casual users with minimal friction possible.

Use Case

"How to keep shared content around when you shut down your laptop?"

Things like Filecoin or commercial pinning services are one way to solve pinning, but hobbyist users and developers may not want to go that route.

UX of Pinning Buddy System

Rough idea is to provide a super easy way for group of people to opt-in to a scheme where they pin each others pins without any money/goods being exchanged in the process. Think "group of friends", "small software house" or "members of local hackerspace".

How we can make the processs as simple as possible? Ideally, it would be one-time opt-in that looks roughly like this:

generate unique secret/key (pubsub room id?) for the group
share it between your friends (IM, email)
everyone adds it to the config of own IPFS node
from now every pin made by a member of the group will be automatically advertised (eg via pubsub) and all group members will preload/pin it in best-effort

Security Considerations

It should be explicitly documented as trust-based alternative to things like commercial pinning services. It means, you trust every group member with the knowledge what are your pins.
Should opt-in have an open or closed characteristic?
- open: new members just add the same key (probably better UX)
- closed: adding new member requires everyone to switch to a new key (better security, as it would enable things like key being a product of PeerIDs of all "whitelisted" members)

Prior art

https://github.com/ipfs/ipfs-cluster (Collective pinning and composition for IPFS)
http://mars.i.ipfs.team:9444/ (manual gx package pinning)
https://github.com/BrendanBenshoof/cachewarmer/ (opt-in cache warming, seems to be dead, but provided "buddy system" feel we aim for)
https://github.com/ipfs/pinbot-irc (a bot for the ipfs irc channel that pins things)
https://github.com/tableflip/ipfs-friends

olizilla commented 6 years ago

The user story went like this:

I want to share a file with a friend. I don't want to use [insert advert heavy centralised file sharing website name here], I just want to use my computer to send a friend a file and I want them to receive it, whenever they decide to click the link.

I don't want to pay for this privilege, but I do have a group of like minded friends who are happy to help out, reciprocate, and generally be nice peers. I'd like to be able to add my friends peerIDs to a list, and when I use companion or desktop to share a file with someone, I'd feel 💯warm fuzzies knowing that my friends ipfs nodes would notice and pin that hash, and that they'd be helping me make sure that the intended recipient of the file would be able to get it, even if i've had to close my laptop and head home. I'd gladly do the same for them.

JonKrone commented 6 years ago

+1, love this idea.

As I understand, when a node restarts it doesn't re-subscribe to the pubsub channels it was previously listening to. In the user story that @olizilla told, once the user turned off their laptop, they'd have to reconnect to the topic. It would quickly become frustrating to re-connect to their shared channel. Perhaps we'd want an option for persistent subscriptions in the pubsub implementation.

When generating this pubsub channel we could also generate a webpage with a shareable URL that would automatically, via window.ipfs if available, subscribe a local node to the pinning topic. This simplifies the UX a bit.

I'm unfamiliar with ipfs-cluster, but it would be super cool to buddy up with a cluster. It could enable broader use cases for groups that have lots of data like small businesses, academics with research datasets, movie libraries, etc.

//cc https://github.com/ipfs-shipyard/peer-pad/issues/90 because there's potential overlap.

hsanjuan commented 6 years ago

Having this usecase supported is in Cluster roadmap in the midterm (6 months). We call it "collaborative pinsets". The use-case described thing is the simplest one to implement for cluster so it will come first. We will need however to figure out which is the best UX flow for the user.

Say IPFS Cluster can provide this (pubsub based distribution of pins along with a persistance so that you can catch up after being away). What would you expect to be the required configuration to set it up?

For reference, discussion on collaborative pinsets: https://github.com/ipfs/ipfs-cluster/pull/467/files

The setup I thought of was:


Ideally it will have a super-easy setup:

1. Run the go-ipfs daemon
2. `ipfs-cluster-service init --template /ipns/Qmxxx`
3. `ipfs-cluster-service daemon`

The config is initialized with a template which has the right values to make everything work.

akavel commented 5 years ago

I'm not sure if that's the exactly same usecase or slightly different, but personally I'd be super interested in a possibility to have a shared backup between a few machines through IPFS, where I could control which files are pinned on which machines. For example, I have photos on my laptop, but I want to also backup them through IPFS to my Raspberry Pi or NAS, and also to a VPS or cloud server I am paying for. While for some older photos, I would like to remove them from the laptop, and keep only on the RPi/NAS + VPS/cloud.

bertrandfalguiere commented 5 years ago

hsanjuan commented 5 years ago

Having this usecase supported is in Cluster roadmap in the midterm (6 months). We call it "collaborative pinsets".

I'm happy to report that this is supported now, since v0.11.0 release.

jessicaschilling commented 4 years ago

Closing -- let's move further work on this into Cluster's collaborative pinsets as needed.

akavel commented 4 years ago

@jessicaschilling Given that this is an issue in IPFS GUI, is this now considered complete in the GUI? Can you help me understand how can I use this feature in the GUI? Or, does it mean that because of the "collaborative pinsets" feature is in ipfs-cluster, it is for some political reasons not planned to be supported by the IPFS-GUI? Or maybe is there a new issue in ipfs-cluster that is somehow discussing the GUI aspect? I'd be very grateful if you could help me understand how can I further track the status of the GUI aspect of this feature! TIA

edit: Also, as a side note, I must admit after reading the "collaborative pinsets" docs of ipfs-cluster, I unfortunately still don't really understand how to use them for this use case :( I'd be super grateful if someone could help me try and understand that...

jessicaschilling commented 4 years ago

Hi @akavel -- thanks for reaching out with your comments!

This definitely isn't considered complete in the GUI-based apps (Desktop, WebUI), and the intent is to better integrate Cluster as a whole into these apps. Issue https://github.com/ipfs/ipfs-gui/issues/59 is, at present, the master issue for discussing this, though at the moment active development isn't happening simply due to allocation of resources (that said, contributors are always more than welcome!).

Just as a note -- we're currently in the process of examining all open issues in the 12 GUI-related repos for the sake of aligning and prioritizing work. That's why you saw labeling and notes on this particular issue. 😊

As for support in working with IPFS Cluster -- that's a bit out of my wheelhouse, but we do have folks on https://discuss.ipfs.io/ happy to discuss. We're really trying to centralize on using the forums as primary means of support and discussion, so please open a thread there! Thanks so much.

ipfs / ipfs-gui