Problem: there is no protection against DoS and amplicifation attacks in case of UDP and public K

[1082008]

SHS is a very nice protocol if used over TCP but what if I need to use it over UDP? If K is kept in secret than it can be enough to prevent DoS attacks but what if K is public? With K an attacker can send large amount of challenge messages to the server, easily exhausing its memory. A DDoS attack also can exhaust CPU resources which could be mitigated somehow.

My ideas are:

1. SHS can be protected against memory exhaustion attacks using the magic cookie mechanism seen in CurveCP. This needs adding this magic cookie to server challenge (step 2) which the client must repeat in auth (step 3).
2. Making server challenge bigger opens the door for amplification attacks, so client challenge (step 1) needs to be padded in order to be at least as big as server challenge. The server side must enforce this size for client challenge messages.
3. Attacks against CPU resources could be mitigated using some proof-of-work system. If the server is low on CPU, it should give the client some puzzle in the server challenge. The client must solve that puzzle and include its solution in auth. It should be considered that this type of protection worths it. I am not sure at all because the server can check solution earliest only in step 3 but at that point it can also check the auth message which proves that the client is a legitim party because of its signature.

Without the proof-of-work mechanism the modified protocol may look like this:

1. ? → ?: a_p, hmac_K(a_p), padding
2. C = Box_[T](a_p, b_psecret) ? ← ?: b_p, hmac_K(b_p), C
3. H = A_p|Sign_A(K|B_p|hash(a·b)) A → B: Box_{[K|a·b|a·B]}(H), C
4. A ← B: Box_{[K|a·b|a·B|A·b]}(Sign_B(K|H|hash(a·b)))

where:

• padding has the same size as C and its content doesn't matter,
• C is the magic cookie
• T is a minute key only known by B. In step 3 B accepts A's message only when it contains a valid C encrypted by the current or the previous minute key.

Note that using magic cookie B doesn't need to allocate any memory until step 3, when A authenticated herself for him.

@dominictarr please review

[dominictarr]

please lets use a name for the "magic cookie" that actually makes sense. What is it? Is it the ephemeral private key encrypted back to a symmetric key that the server only knows?

[dominictarr]

One thing that concerns me here is that A and B both send C in the clear, which means C could be replayed.

Can you describe the situation you are imagining where this DoS happens? what sort of application are you imagening deploying this to?

[dominictarr]

I mean, I guess, if we are gonna talk about mitigating DoS, how do we model this? currently, the security properties we claim about shs are all things you can verify cryptographically - but this kind of DoS depends on the implementation, so what is our model of DoS that we are defending against?

[1082008]

Yes, "magic cookie" contains the ephemeral private key (I used b_psecret for notation but maybe b_s is more appropriate) encrypted back to a symmetric key that only the server knows. That cookie doesn't only contain b_s but also a_p therefore the cookie is bound to this actual handshake. An attacker can replay the cookie but is useless because he doesn't know a_s, he is unable to create a valid auth message with a replayed cookie.

I think the name cookie is a descriptive one because the "magic cookie" described here is very much like HTTP cookies. Server sends a cookie which must be included in the next request of the client. This is a client side state storage. In this case the cookie must be encrypted because only the server should be able to read its content. I guess that's why its called magic cookie in CurveCP but we can call it simply cookie. What it contains is the same as the minimal amount of information the server would need to store after received client challenge and sent server challenge.

The DoS situation where C is useful is very easy to trigger. In case of TCP transport every client connection causes memory use in the server's network stack and also in the server application with the current implementation. That means some attacker can open unbounded number of connections and send a client challenge message on each of them, triggering memory allocations on the server side. Memory will be exhausted and legitimate clients won't be able to initiate connections to the server under attack. The attack doesn't have to be a distributed one, only one evil host can take down the whole service.

There is not much we can do while staying on TCP because its relatively high per connection resource use at the level of network stack. Magic cookies can help on the server application but not on the network stack. For a real protection UDP should be used instead of TCP. That way there would be no overhead caused by connections because UDP is a connectionless protocol. A new client won't cause any memory use in the OS network stack. But it will cause some memory use in the server application without using cookies. (A secret K can provide the same protection but K cannot be secret in some cases.)