userspace mutex behavior

[Orycterope]

I'm trying to design how the kernel will implement userspace mutex.

Here's a few properties for a mutex, from the kernel's perspective:

• A mutex is process local.
• When there is contention the kernel manages a queue (FIFO) of waiting threads, to prevent starvation.
• A mutex is just a cookie. The usual implementation is: the user gives the userspace address of its mutex structure as an argument to the lock and unlock syscalls, and the kernel uses it as an ID for the mutex, and never interprets it in any way.

From there implementing mutex seems quite simple, but I still need to figure out what should be the behavior in some edge-cases:

1. Is a mutex just a virtual address for the kernel ? What happens if the address is aliased, i.e. the page it lives in is mapped several times, creating several addresses pointing to the same data. Do we treat it as several mutexes ? This makes no difference for the kernel, but the userspace might be very angry about this. We should make sure that it is impossible to create an alias for a mutex in safe rust. What if the mutex lives in a shared mapping ? Do each process have its own independent mutex, pointing to the same data ? Do we forbid mutex in shared mappings ?
2. What do we do when the thread owning a mutex crashes ? When it safely panics, the userspace thread poisons the mutex while unwinding, unlocks it (waking up other threads), and dies, and the kernel does not have to be aware of this, poisoning is a userspace concept. But when it crashes ?
   • If the kernel does nothing, waiting threads will sleep forever. This is yet another way to create an immortal process, as its last thread will never be dropped.
   • Does it wake up all threads, and have the syscall return an error, letting the userspace handle it ? But horizon doesn't seem to return errors for this syscall. Also, the architecture-specific implementation in std doesn't seem to have access to the poison structure, so it couldn't poison it if the syscall returned an error.
   • Do we ... kill the whole process ?

[roblabla]

Is a mutex just a virtual address for the kernel ?

yes. You are not supposed to use a mutex that's aliased. Very UB, bad idea.

What if the mutex lives in a shared mapping ?

Same, very UB. Mutex is process-local. The tags they use (thread handles) are process-local too, and can conflict in a cross-process scenario. (unlikely, but y'know, it can happen).

We should make sure that it is impossible to create an alias for a mutex in safe rust.

I mean, it should already be impossible because all the functions to create shared mapping from existing memory should be unsafe. If they aren't, that's a problem.

What do we do when the thread owning a mutex crashes ?

An excellent question. First of all, a thread that crashes in unexpected ways is supposed to kill the whole process, so it's not a big issue. But what about a thread that locks a mutex, and then calls svcExitThread() without unlocking? I spent some time reversing, and I am fairly sure the answer is "nothing happens" still, so yeah, the mutex will stay locked forever. Should probably test on hardware though, just to make sure.