katzenpost / mailproxy

POP/SMTP to Katzenpost proxy server library
GNU Affero General Public License v3.0
4 stars 5 forks source link

send: Figure out a better send scheduler. #14

Open Yawning opened 6 years ago

Yawning commented 6 years ago

The transmit scheduler is somewhat primitive and is heavily impacted by the receive polling interval. It would be nice if this could be made more intelligent. In particular a better scheduler should:

  1. Provide good anonymity properties.
  2. Provide a good user experience.
  3. Not overload the network.
  4. Minimize spurious re-transmissions.

For reference I think the current one is more oriented towards satisfying criterias 3 and 4, thus the need for improvement. This may be something that should be defered till someone comes up with the finalized cover traffic design.

Yawning commented 6 years ago

The scheduler is a lot more intelligent and aggressive about transmitting now, but it can still be improved.

In particular, it will send entire messages at least once in FIFO order, so the client side mail spool can have HOL problems if there is a large message followed by a smaller one, a problem that is significantly exacerbated by the message amplification when there is multiple recipients (#8).

Making it more fair across messages in the spool requires tracking a bit more state, though I don't think it will be overly hard or complicated, especially since the state doesn't need to be persistent.

Yawning commented 6 years ago

Ok, I changed the half assed random delay scheduler to use Poisson sampling as per the Loopix paper. I have no idea what a good value for lambdaP is at this point, kimchi uses 15.0 for now, and it works. The "right thing" will happen (with a few limitations that I need to address) when a more sensible value of lambdaP is set.

david415 commented 6 years ago

This LambdaP is NOT correct as per the Loopix design because it should be transmitting forward messages from the user via a FIFO queue or drop decoy messages and NOT loop decoys. The Loopix design specifies that the loop decoys must be send out on a separate Poisson process. In fact, we should have a scheduler with not one but 3 timers one for each client lambda value as specified in our PKI document.