lit review: data replication / personal data back-ups

[gitaaron]

• current work

• Bitswap

• ipfs-cluster

• pinset

• pnet

• CRDTs

  • orbitdb crdt

  • SE Daily interview

  • designing data-intensive applications

• FIM

• zfs

  • zfs copy-on-write

[gitaaron]

I don't think BitSwap will help with this use case since it seems that the entire contents of the file (root CID only) will be transferred and thus consume resources/bandwidth unnecessarily for each change.

We also want to do the opposite - instead of asking all peers if they have a CID we want to advertise to all peers that there is a newly created CID and they should download it.

There are probably learnings on protocol/wire transfer so it could be worth it to dig deeper.

[gitaaron]

Reading a bit further - looks like ipfs-cluster is promising and they mention using BitSwap so perhaps my previous comment is incorrect.

[gitaaron]

One potential issue with ipfs-cluster is a lack of relays for NAT hole punching.

This network does not interact with the main IPFS network, nor with other private IPFS networks and is solely used so that Cluster peers can communicate and operate....

This also means that Cluster peers operate separately from IPFS with regards to NAT hole punching, ports etc.

One thought is possibly using an ad hoc VPN instead? Problem with that is it would incur a master/slave architecture where a single BOX would have to operate as the VPN server. Still - perhaps something to look into in the future if NAT traversal becomes a sticking point?

[farhoud]

One potential issue with ipfs-cluster is a lack of relays for NAT hole punching.

This network does not interact with the main IPFS network, nor with other private IPFS networks and is solely used so that Cluster peers can communicate and operate....

This also means that Cluster peers operate separately from IPFS with regards to NAT hole punching, ports etc.

One thought is possibly using an ad hoc VPN instead? Problem with that is it would incur a master/slave architecture where a single BOX would have to operate as the VPN server. Still - perhaps something to look into in the future if NAT traversal becomes a sticking point?

Base on my experience we can still use Rely v2 and webrtc signaling server for hole punching

[gitaaron]

Base on my experience we can still use Rely v2 and webrtc signaling server for hole punching

Moving this convo over to - https://github.com/functionland/docs/issues/63#issuecomment-1073932513