[Preliminary] DHT Tracker

[akinwale]

Problem Statement

Within the context of media playback in the app, especially video, there can be a significant delay from the point in time when the Play button was clicked to the when playback actually starts. There are also instances where playback ultimately fails to start after the user has waited during this period where nothing happens. This results in a poor user experience.

The delay is attributed to the fact that it takes some time for the distributed hash table (DHT) implementation to find peers that have a particular file available, and even when the peers have been found, the peers may actually be offline, which prevents the file from being downloaded at that point in time.

While content on the discover page is guaranteed to work at least 90 – 95% of the time, the same cannot be said for videos picked at random from search results or other avenues for discovery.

Approaching this from the user’s perspective, we can simplify the problem like so.

• It takes too long for a video to start playing after clicking on the Play button.
• In a number of instances, after waiting so long for a video to play, it doesn’t play at all, but instead fails with an error.

From a technical standpoint, the reason it takes a significant amount of time for appears to be a result of several things.

• Poor DHT implementation
• Not having a significant number of active DHT nodes
• General limitations on typical DHT performance (citation needed)

The proposed solution to alleviate this problem would be to implement a tracker which caches active peers and useful metadata corresponding to the files that they have available. A round-trip request to a server running a tracker should easily take 2 seconds or less which essentially leads to the file download starting much earlier, and prevents the unnecessary wait time in the case of a file not actually being available to download.

Requirements

• Anyone should be able to set up their own tracker server.
• Users should be given the option to opt-out of using trackers.
• With trackers enabled, DHT will be used as a fallback if no trackers are available.
• Data related to non-active (or offline) peers should not be retained. The tracker DB should only have peers that have announced to the tracker over the last n minutes or hours as the case may be.

Owner

akinwale

[lyoshenka]

Our TTFB time is significantly lower than it used to be, so I'm closing this for now. A tracker may still be the right move in the future as our network grows.