Extend the threading platform abstraction layer

[yanesca]

Suggested enhancement

After some team discussion, we've decided to rely on a new threading abstraction which mimics C11 (i.e. mbedtls_fff where fff is the C11 function name, having the same parameters and return type, with default implementations for C11, pthreads and Windows). We'll use condition variables in addition to mutexes.

Justification

Mbed TLS needs this because without it we can't implement the improvements to the threading support we were planning. (See the threading design document).

[tom-cosgrove-arm]

I'd actually prefer that we move to reader-writer locks after mutexes

[yanesca]

I prefer condition variables as

• they are present in C11 threads - not because it would be necessary to translate 1-1 to any API, but in the hope that the C11 standards guys did a good job (they definitely put more thought into this than we can hope to) and this falls in the least common denominator for threading on a very wide range of platforms
• are more powerful - I mean I know how to implement reader-writer locks with condition variables, but don't know how to do it the way around

Also, if we decide to seriously consider reader-writer locks, we will need to plan/write up the design at least to the extent we have for condition variables.

[tom-cosgrove-arm]

RW locks are present in Pthreads and WIndows. As you say, they are a higher-level abstraction (can be built from mutexes and condition variables) so code size would be reduced and maintainability higher (i.e. don't use a low-level abstraction when there's a more appropriate one for the problem to be solved).

In this case, I think reader-writer locks do map well to what we're trying to do: once correct thread safety is implemented with mutexes, each mutex acquisition/release code could be replaced with either read or write acquire/release, and I would expect concurrency to increase and correctness remain.

[paul-elliott-arm]

I must admit I am unsure precisely what we need condition variables for other than implementing RW Locks, but given most major platforms bigger than an RTOS implement them, we should probably try and use those implementations (which are probably much more efficent) rather than implement our own unless we have to. We could also set this up under a flag so that we fall back to just using mutex'es if whoever is using the library doesn't want to use them or cannot implement them.

I do think from experience that the performance pain from using mutex'es alone is going to cause significant enough friction that we are going to have to do this, though.

[gilles-peskine-arm]

@paul-elliott-arm Condition variables are more general than reader-writer locks (I don't think it's possible to implement condition variables with reader-writer locks without busy-wait). I think mutex plus condition variables are theoretically equivalent to semaphores, but semaphores are harder to use.

A typical use case is a producer-consumer queue: multiple producer threads can add objects to a queue, and multiple consumer threads can take object out of a queue. You can use a mutex to protect the queue, but how do consumers wake up producers? With a condition variable, the producer signals the condition, which wakes up consumers.

Condition variables are pretty common in concurrency APIs, and where they're missing there are usually semaphores. For example Zephyr has condition variables, but if you want a reader-writer lock, you need to build your own.

[gilles-peskine-arm]

If you're curious about the choice of synchronization primitive, you might want to read Jouni Leppäjärvi's thesis: A pragmatic, historically oriented survey on the universality of synchronization primitives. Leppäjärvi presents (local) synchronization as being mutual exclusion plus wait-signal, which can be satisfactorily fulfilled by either semaphores or mutexes+condition variables.

[paul-elliott-arm]

I know what condition variables are, and I'm not questioning that we could bring them in, they are, as you say, pretty common. What I am saying here, is that in most cases we need something that looks very much almost in every way like a reader-writer lock (something, btw, which is in every single multithreaded implementation I have ever worked with).

I agree that we don't have RW locks on every platform, as I said above we should use them on platforms where we have them, and then do what we can for the ones that do not basically. You can construct a RW lock with care from a condition variable and a mutex.

So in short, I am saying we need both.

[yanesca]

As far as I remember the main argument for reader/writer locks was performance: taking a reader lock doesn't mean a full context switch if the OS implements the reader/writer lock. It takes a full context switch if we implement it on top of other primitives.