search

openworm / owmeta-core

Core library for owmeta
MIT License
3 stars 1 forks source link

Peer-to-peer file sharing for OpenWorm data set distribution #13

Open mwatts15 opened 5 years ago

mwatts15 commented 5 years ago

Parent of: openworm/owmeta#403, openworm/owmeta#407, openworm/gsoc-2019#2

clbarnes commented 5 years ago

I think bittorrent was mentioned in one of these issues, but I think it's worth mentioning both IPFS and dat in case they haven't come up already. dat, in particular, is built for exactly this, although it looks like the only mature implementation is in javascript, which is pretty useless for doing FFI with.

jaideep-seth commented 5 years ago

Hello @clbarnes , while researching for the project i found that:

  1. Information is available on the IPFS only as long as the individual node servers uploading the contents are running i.e information does not persist on the file system. Project specifications require contents to be accessible even if the host node is offline.
  2. I looked into BitTorrent, Kademlia and IPFS for Python implementations, will look into 'dat' right away!

This is the link to my proposal where i have compared some P2P protocols & described the suitability of BitTorrent for Project 16.1, please give me your suggestions.

Thank you so much for your interest! will reach out to you for feedback and testing in further development phases of the project(if thats alright)!

clbarnes commented 5 years ago

Looks good - didn't know that about IPFS, glad you understand it better than I do!

One more thing to think about with bittorrent is that some institutes try their hardest to block the entire protocol. You can get around most of that with non-standard ports, SSL and so on, but that may be more overhead for users.

It's a shame the rust implementation of dat doesn't seem to have taken off, as that would have been pretty easy to throw a python wrapper around.