What should bridge together the device swarms?

[Adrien-Luxey]

The DHT is not our contribution, but we are still going to need storage capacity.

We could use a centralized service, or a production-ready distributed KV-store, or a real DHT.

Centralized service

A file, SQLite, SQL, No-SQL, whatever. We just need to explain that this is not our contribution in the paper.

KV-Store

• Bolt is the Golang KV-store implementation. It's quite feature-rich, which makes it bloated for our corner case. It's not distributed.

DHT

• There are shitloads of Kad/Mainline implementations around. They are tailored for BitTorrent, thus accepting only IP as values. We could use this, but complying with the BitTorrent BEPs would take some effort;
• I just stumbled upon OpenDHT, a C++ DHT that seems to have several properties we desire (cryptography, notably). A cool blog post explains its features ;
• Octopus: A Secure and Anonymous DHT Lookup

I will check out how OpenDHT works, for the moment.

[dbosk]

NebuloStore can be used as a DHT with lots of redundancy. It has the properties I need for certain privacy properties. It has some good publications behind it:

• Krzysztof Rzadca, Anwitaman Datta, Sonja Buchegger. Replica Placement in P2P Storage: Complexity and Game Theoretic Analyses. In Proceedings of ICDCS 2010, Genoa, Italy, June 2010.

Most relevant summary: http://naan.mimuw.edu.pl/trac/raw-attachment/wiki/ObjectApi/nebulolists.pdf

And it has an implementation, see: http://nebulostore.org/software.html You have in the same page the implementation of the P2P backup system where the average availability was 13%.

[Adrien-Luxey]

Alas, we need more complex queries that DHT allow for routes creation (i.e. peers selection).

I was thinking of Cassandra, that has a DHT backend, but I'm sure its network cost is not negligible.

---

The two basics operations we need from the DHT are:

• Put nodes and file addresses for routes: should not require more functionality than basic KV-store;

• Route creation, that requires queries like "Give me all nodes connected next round": requires SQL-like "WHERE round=t+1" functionality.

[dbosk]

On Mon 09 Apr 2018 15:51:50 GMT, Adrien Luxey wrote:

The two basics operations we need from the DHT are:

We don't necessarily need a DHT. We can just say we need a distributed (preferably decentralized) algorithm that provides the following two operations. (I've started phrasing it like this in the paper.)

Then we can simply plug in any algorithm which provides these properties.

• Put nodes and file addresses for routes: should not require more functionality than basic KV-store;

• Route creation, that requires queries like "Give me all nodes connected next round": requires SQL-like "WHERE round=t+1" functionality.

[Adrien-Luxey]

I would keep the DHT. If we only use predictions at t+1, we don't need SQL-like queries. The only query would be "get random peers". We would also put file exchange information at precise locations that expire.

Literature is abundant on the topic, it will add value to our paper, and some references.

[Adrien-Luxey]

Ok, I don't find anything for getting a random sample of data from a DHT. So I'm starting to wonder about using a Random Peer Sampling service (RPS) to select peers when creating routes. It's a well-studied topic in distributed computing: to have every node keep an up-to-date list of random peers in the network (stale data is evicted reasonably fast).

A good article on the topic is CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays. Doesn't tackle privacy, but newer articles do.

There is still another case where we needed a DHT: providing introductory points (IPs) to the file requester. We could do this out of band. Need to think about it.

[dbosk]

We use random peer sampling for now, but keep this for future work when we might consider using something else.

[dbosk]

Our black box use of RPS is general enough. We can plug in anything.