This change fundamentally changes how change callbacks work on Dynamics. Prior to this change, callbacks executed on the thread that was performing the change. This could lead to situations where multiple threads were executing callback chains which leads to unpredictable locking patterns on the dynamics. The basic deadlock detection was not enough.
This change defers callbacks to a single callback thread. This thread ensures that no dynamic can have callbacks enqueued more than once. By limiting execution to one set of callbacks at any given time, this greatly reduces the surface for locks to contend with each other.
The next issue was how tuple-related functions like for_each/map_each were acquiring their locks. By calling a.read() then b.read(), this was causing a to be held in a locked state while b was being aquired. If users are careful to always acquire their locks in this order, everything is fine. But with Cushy there can be unexpected situations where these locks are being held.
This change also refactors lock acquisition for tuples to try to acquire all the locks in a non-blocking way. If any lock woould block, the initial locks are dropped while the lock that would block is waited on. After this is acquired the process starts over again to gain all the locks. This isn't perfect, but it doesn't require unsafe. With unsafe, we could in theory create a ring of callbacks that handles acquiring all of the locks into MaybeUninits. Upon successfully calling all callbacks, the values can be assumed init. But writing all of this in macro_rules isn't fun, and the current solution alleviates the main problem
This change fundamentally changes how change callbacks work on Dynamics. Prior to this change, callbacks executed on the thread that was performing the change. This could lead to situations where multiple threads were executing callback chains which leads to unpredictable locking patterns on the dynamics. The basic deadlock detection was not enough.
This change defers callbacks to a single callback thread. This thread ensures that no dynamic can have callbacks enqueued more than once. By limiting execution to one set of callbacks at any given time, this greatly reduces the surface for locks to contend with each other.
The next issue was how tuple-related functions like for_each/map_each were acquiring their locks. By calling a.read() then b.read(), this was causing a to be held in a locked state while b was being aquired. If users are careful to always acquire their locks in this order, everything is fine. But with Cushy there can be unexpected situations where these locks are being held.
This change also refactors lock acquisition for tuples to try to acquire all the locks in a non-blocking way. If any lock woould block, the initial locks are dropped while the lock that would block is waited on. After this is acquired the process starts over again to gain all the locks. This isn't perfect, but it doesn't require unsafe. With unsafe, we could in theory create a ring of callbacks that handles acquiring all of the locks into MaybeUninits. Upon successfully calling all callbacks, the values can be assumed init. But writing all of this in macro_rules isn't fun, and the current solution alleviates the main problem