Riffle torrent implementation

[charles2910]

As one part of allium, we will implement the anonymous file sharing system proposed in the Riffle article. It's described in section 5 of the paper and it consists of 3 main parts:

• Block request
• Block upload
• Block Download

[charles2910]

Upon further analysis, we can break down this work in a handful of parts. These parts are listed below with important points of each part.

• [x] Create Torrent File - Done in #10
  • Use libtorrent? No, use custom lib - torf
  • Implement a custom lib? (Out of necessity or just to understand it better)
• [ ] Distribute Torrent File - issue #14
  • Custom implementation needed
  • Broadcast to all machines in Riffle Group
• [ ] Distribute Torrent Blocks
  • Custom implementation needed
  • Distribution algorithm would be a good thing (
• [ ] Get Torrent Blocks
  • Custom implementation needed
  • Use the PIR to Download the wanted block

[charles2910]

Research about Create Torrent File:

The torrent file uses a specific enconding known as bencode and it has mandatory fields as well as some optional ones.

The explanation and text below were adapted from the unoficial BitTorrent Specification.

Bencoding

Bencoding is a way to specify and organize data in a concise format. It supports the following types: byte strings, integers, lists, and dictionaries.

Byte Strings

Byte strings are encoded as follows: :

• Example: 'spam' is represented as '4:spam'.

Integers

Integers are encoded as follows: ie

• Example: 3 is represented as 'i3e'.

Lists

Lists are encoded as follows: le

• Example: ["spam", "eggs", 3] is encoded as 'l4:spam4:eggsi3ee'.

Dictionaries

Dictionaries are encoded as follows: de

• Example: d4:spaml1:a1:bee represents the dictionary { "spam" => [ "a", "b" ] }.

Metainfo File Structure

The content of a metainfo file (the file ending in ".torrent") is a bencoded dictionary, containing the keys listed below. All character string values are UTF-8 encoded.

• info: a dictionary that describes the file(s) of the torrent. It's structure is described below for the case of a single file torrent;
• announce: The announce URL of the tracker (string);
• creation date: (optional) the creation time of the torrent, in standard UNIX epoch format (integer);
• comment: (optional) free-form textual comments of the author (string);
• created by: (optional) name and version of the program used to create the .torrent (string);

Note that some fields were left off for brevity.

Info Dictionary

This section contains the dictionary structure for the "single file" mode.

• name: the filename. This is purely advisory. (string)
• length: length of the file in bytes (integer)
• piece length: number of bytes in each piece (integer)
• pieces: string consisting of the concatenation of all 20-byte SHA1 hash values, one hash per piece. (byte string, i.e. not urlencoded)

Note:

• Again some fields were left off.
• Each piece has a corresponding SHA1 hash of the data contained within that piece. These hashes are concatenated to form the pieces value in the above info dictionary. Note that this is not a list but rather a single string. The length of the string must be a multiple of 20.

[charles2910]

I've found 2 libraries that can be used for generating the torrent files. They are:

• torf: newer, has better manipulation of fields, GPL-3+.
• dottorrent: base implementation, MIT.

To give a better example of the torrent file, I've created a test torrent. The high level (fields and values) will be shown together with the bencoded resulting file.

{
    'info': {
        'length': 223,
        'name': 'teste.c',
        'piece length': 16384,
        'pieces': b'\x9dz\x12\x90\xe4\xfdK\xec\x1b_\xb28\xc6/\xab\x87\xd1\x9c\x1cc'
    },
    'created by': 'Charles',
    'comment': 'This is a test torrent file',
    'creation date': 1635279024
}

d7:comment27:This is a test torrent file10:created by7:Charles13:creation datei1635279024e4:infod6:lengthi223e4:name7:teste.c12:piece lengthi16384e6:pieces20:\x9dz\x12\x90\xe4\xfdK\xec\x1b_\xb28\xc6/\xab\x87\xd1\x9c\x1ccee