Transactional side effect mutations (from thesis paper)

[GregoryConrad]

I can think of two ways to implement this.

1. Similar way to Dart (i.e., opt in). We can add a new field to the write txn: txn_modified_capsules: Option<HashSet<Id>>, where if txn_modified_capsules is Some, no rebuilds will take place until it is committed (i.e. they are ignored), or something like that. If None, uses the normal rebuild on each call behavior. Need some more thought here. Note that this must be in a transaction or something behind a Mutex, because this is not possible to do in parallel safely (competing transactions would wreak havoc).
2. Explicit/forced. We can make a container.run(increment_count) and completely remove the CapsuleRebuilder structure. Like container.run(|txn| { increment_count(txn); set_state(txn, 1234); });

[GregoryConrad]

I’ve put some thought to it, and the opt-in approach really feels like a hack on the implementation side, but manual txns can be less ergonomic in many scenarios.

Alas, here are some links that may be handy if we go with the opt-in approach:

• https://doc.rust-lang.org/std/macro.thread_local.html
• https://stackoverflow.com/questions/61070398/how-to-create-a-thread-local-variable-inside-of-a-rust-struct
• https://doc.rust-lang.org/std/thread/struct.ThreadId.html

Idea is that we have an atomic Option or similar representing the thread of a current txn. If we’re in such a thread, we do not rebuild but instead add ourselves to list to be rebuilt. In another thread, we would lock until we can make our own txn.

I may try to implement both approaches to see how each looks/feels from both an API and implementation perspective and pick based on that.

[GregoryConrad]

Got a good chunk of implementation work done on the side-effect-txns branch for the explicit txn approach (like container.run(|txn| set_state(txn, 1234));). Probably won't finish the implementation until I'm sure I want to go down that road because it'll take a good chunk of time.

Explicit transaction approach:

• Pros: forces you to know exactly what is happening at all times
• Cons: forces you to know exactly what is happening at all times

Jokes aside, the implementation is fairly clean and easy, which is a good sign. Issue is that it is atrocious to work with. Aside from manually passing around txn everywhere, Rust has a really annoying bug in closures that follow the form |foo: &Foo| -> &Bar {foo.bar} because the lifetimes aren't properly related. And of course, explicitly writing out the lifetimes to fix this requires nightly. The other workaround that I used in my branch is a function that helps the compiler out with lifetimes, but forcing users into a workaround like that is 🫤

I am going to try to figure out the non-explicit (like rearch-dart) way next, if it is even possible. Frankly not sure, but I'll give it a go. I will likely need a CapsuleRebuilder like structure in a thread-local variable of some sort.

[GregoryConrad]

For the dart-like approach:

Was originally going with thread_local: this is wrong because each container needs its own independent transactions (versus just a thread).

also, I was a bit stumped with initial impl since locking a mutex in the same thread isn’t allowed (possible panic or deadlock). I actually learned about the “reentrant mutex” in this process and that should do the trick when put in the ContainerStore (ReentrantMutex<RefCell<HashSet>). https://docs.rs/parking_lot/latest/parking_lot/type.ReentrantMutex.html

we can keep the write txn within its own normal mutex. Since the reentrant mutex will lock the entire txn process, only one thread at a time will access the data/nodes mutex. Also, continued locks of those mutexes in the same thread should be quick since there won’t be any contention and they should pick it up instantly since it was last held by the same thread.

[GregoryConrad]

Pushed up some kinda-working code (side effects broken) for the Dart-like transactions into dart-like-side-effect-txn. I like this approach to the side effect transactions more than the explicit approach, so I will likely push this one to completion and see how it goes.

[GregoryConrad]

In case the current side effect txn model, for whatever reason, needs to be changed, see the below patch off of commit 92ed98c92d70e45d1257f66cfb271b0098381697 that starts the implementation of explicit side effect transactions.

explicit-side-effect-txns.patch