search

quchen / amoeba

Amœba is a distributed network.
Other
18 stars 1 forks source link

Speed up pruning? #40

Closed quchen closed 10 years ago

quchen commented 10 years ago

Currently, prune requests are sent to single random DSNs once per housekeeping tick. This leads to an exponential probability of reaching that one node that actually allows the pruning instead of rejecting it.

On the other hand, sending prune requests to all DSNs might be a little too much; if multiple OKs come back concurrently, the node might prune away more nodes than necessary (i.e. more than allowed, even).

Is there some way of speeding up the pruning process without running into trouble? I don't like how the comparatively long timing of the housekeeping loop is responsible for the integrity of pruning behaviour anyway.

(Idea: maybe pruneH could remove the nodes from the DB itself in the atomic block, which would solve the concurrency problems. The additional removal when the worker terminates would be redundant in this case, but that is a pretty small price to pay.)

quchen commented 10 years ago

80f122c43d45e5a47f5f964280b4ec095021fef2 implements a nicer intermediate solution: for each excess connection, one pruning request is sent out (instead of just one as it was previously).

Since nodes handle pruning requests on a server basis, once the pruning signal can't be amended by the issuing USN. The worst case when sending out lots of pruning signals (as mentioned in the original text in #40) would be that all DSNs accept the request and drop the connection, leaving the USN without DSNs, which means it's half-dead.